Stay Competitive! Mastering the Electronic Job Search (Westchester Library System)

Outcome-Based Evaluation
Best Practices in New York State

(Example illustrating the use of a Complete OBE Package, including a logic model)

This is a classic OBE project. Predictions about user outcomes were made and tracked with carefully constructed follow-up activities. This is another good example of how a system training program writes outcomes and measures that reach the patron level. Note that often when projects are conceived with outcomes in mind, the percentage of successful participants exceeds predictions.

Outcome #1: Librarians and counselors search electronic career and education resources.

Indicators: It is predicted that 201 or 80% of all librarians and counselors who complete the WLS training sessions will be able to describe how to conduct an effective electronic search for career and educational information assessed by trained observers during the training session. 176 or 70% of librarians and counselors who complete the WLS training sessions will score 80 or better on three searches prescribed in a quiz during the training session.176 or 70% of librarians and counselors who complete the WLS training sessions will be able to navigate two commercial online job banks assessed by trained observers during the training session.

Results Outcome #1: It was demonstrated through observation that 55 staff out of 61 who participated (91%) were able to describe how to conduct an effective electronic search for career and educational information during the training sessions (prediction was 80%). A short written quiz indicated that 56 staff (92%) were able to successfully carry out three prescribed searches during the training sessions (prediction was 70%. Trained observers determined that 52 staff (85% were able to successfully navigate two commercial online job banks (prediction was 70%. These results clearly show that the librarians and counselors increased their skills using career-related resources. Year 2 Results: 92% or 150 of 163 participants in workshops obtained skills in using online career resources (prediction was 80%)

Outcome #2: Librarians and counselors teach/counsel targeted groups of jobseekers to use electronic career resources effectively.

Indicators: It is predicted that 20 or 33% of library staff who completed the project training sessions will report training community groups to use electronic career resources as measured by participant reports by the end of the project. 75 or 70% of community patrons/clients who attend the library trainings can describe how to conduct an effective electronic search for career and educational information assessed by trained observers at the end of the workshop.

Results of Outcome #2: Only 12 librarians (20% out of 61 who completed the eight training sessions reported that they trained groups of patrons in their home libraries (prediction was 33%). Not all participants reported back so the actual number may be higher, but this outcome was disappointing. Some librarians indicated that they would try to conduct training sessions in the future but that the demands on their time had made it impossible for now. In the training sessions that were conducted in the home libraries for a total of 92 patrons, the librarians estimated that 64 patrons (70%) were able to describe how to conduct an electronic search for career and education information (prediction was right on target here). It was demonstrated that staff improved their ability to help the target group use electronic resources and to teach targeted groups of jobseekers to use electronic resources through the project training sessions. We will continue to encourage the trained librarians to conduct more classes in their libraries. Year 2 Results: A sample of professional who were trained through this project agreed to track the number of times they provided individual assistance to clients on electronic job search and education resources, and they were contacted …monthly…many are college counselors who see hundreds of students. It is estimated that over 3000 users were assisted. In a follow-up survey of participating libraries who hosted the four jobseeker workshops in Year 2, it was reported that at least 350 searches/individual assistance to the target population occurred. The total number of users served in Year 2 is approximately 3621, which when combined with the 733 users served in Year 1, totals 4454. (prediction was 3752)

The report states that not all participants filed reports. It would be legitimate to extrapolate what the actual number might have been. Projects such as this one should be encouraged to continual to ask participants beyond the period of the grant about their activities because, often significant program impact occurs after report deadline dates. The follow-up survey that sampled assistance to clients was an excellent measure to add to what is known about impact.

Outcome #3: Jobseekers search electronic career and education resources.

Indicators: It is predicted that: 112 or 80% of jobseekers who complete the 6-week career workshop (estimate that 140 out of 160 participants will complete) will prepare an electronic resume as measured by program records submitted by the counselors at workshop end. 98 or 70% of jobseekers who complete the 6-week career workshop will search a business database independently assessed by trained observers during the workshop. 112 or 80% of jobseekers who complete the 6-week career workshop will search the WLS education database assessed by trained observers during the workshop.

Results of  Outcome #3: Records submitted by the counselors for the jobseeker workshops (80 participants) indicated that 54 jobseekers who attended (68% prepared and submitted an electronic resume by the end of the six-week workshops (prediction was 80%). The counselors reported observing that approximately 72 jobseekers (90%) were able to search a business database independently during the workshops  (prediction was 70%) They also reported that about 76 jobseekers (95%) had searched the WLS online education database  (prediction was 80%). These results showed that the targeted jobseekers gained new knowledge of electronic career and education resources and developed the ability to use them independently. Year 2 Results: Counselors who conducted four 6-week workshops for 80 participants reported that 57 jobseekers who attended (71%) prepared and submitted and electronic resume by the end of the six-week period (prediction was 80%). The counselors reported that approximately 72 jobseekers (90 %) were able to search a business database independently during the workshops  (prediction was 70%). 60 jobseekers (75%) had searched the WLS online education database (prediction was 80%).

This grant is a good example of how programs have cumulative impact, first on people trained for a service and secondly on people served.

Outcome #4: Jobseekers experience successful job searches and/or a return to school.

Indicators: It is predicted that: 42 or 30% of jobseekers who complete the 6-week career workshop will report either obtaining employment and/or returning to school assessed by their answers to a survey sent to them four months after service. 40 or 25% of jobseekers who complete the 6-week career workshop and report reemployment and/or a return to school on the survey sent to them four months after service will indicate that the workshop was instrumental in their successes as assessed by their answers.

Results of Outcome #4: Results on the progress of the two fall groups (40 people) who attended the jobseeker workshops are available since we started receiving completed evaluation surveys that we sent out four months after the workshops were completed. It was determined that 18 people (45%) successfully obtained employment or returned to school (prediction was (30%); and 11 of these clients (61%) indicated that the workshop was instrumental in their success (prediction was 25%). Since the two winter jobseeker workshops recently ended, there wasn’t time to determine client progress and report it in this narrative. It will be reported in the Year 2 interim report. Year 2 Results: 28 out of 40 returned four-month follow-up evaluation surveys. 14 people (35%) successfully obtained employment or returned to school (prediction was 30%; 9 of these clients (22%) indicated that the workshop was instrumental in their success (prediction was 25%). Data on winter workshops were not yet available.

Again it would be advisable to continue to follow-up with people who attended jobseeker workshops because such data is the ultimate in impact data. Both years yielded information that jobseekers were truly helped but timing limited the availability of follow-up information.

Anecdotes: (a). Jeff A, a man in his late 50’s, attended the winter jobseeker workshop at the Mt. Kisco Public Library. He had just closed down a small pharmaceutical supply business that was on doing well, and he needed to find a job. He became fascinated with the amount of company and industry information available on the electronic databases and Websites that were being reviewed in the workshop, and he came to the library regularly to do searches, sometimes seeking assistance from a librarian. Be the end of the workshop he reported finding an interesting sales rep opportunity on Craig’s List, a site that lists local community classified and forums—a great place to find jobs, housing, goods and services for free. As the workshop ended, he was scheduled to be interviewed by a large pharmaceutical firm for this position. He later called to hay he had been hired. (b). One of the librarians who attended an electronic resources training session at WLS became very enthusiastic about these resources and decided to broaden the services that his library offered to jobseekers. He offered one class (with more to come) on electronic job search at the library and also put together an excellent Webpage on his library’s Website that offers links to a variety of Internet sites that would interest people in career/job transition. He said that the training experience offered through this grant opened up a whole new area of professional interest that has not only enabled him to assist many patrons, but that has provided a great deal of personal satisfaction.


OBE Activities

Training Manuals

Certified OBE Trainers

OBE Training Plan

Links

OBE Best Practices home

OBE home

Go to Library Development home page


Last Updated: June 3, 2009