Toolbox for Outcome Based Evaluation/Extension of OBE Training (Queens Borough Public Library)

Outcome-Based Evaluation
Best Practices in New York State

(Example illustrating the use of a Complete OBE Package, including a logic model)

Outcome #1: The participants will gain a basic knowledge of outcome-based evaluation techniques and be able to apply them to the planning of their technology training sessions and other programs to ensure reportable, measurable results.

Indicators: # and % of participants who scored more than 70% in the classroom knowledge test.

Data Sources: Knowledge test from training sessions.

Results Summary: We collected knowledge test forms completed during the classroom training sessions. The average score was 87% (standard being 70%). Also, in order to get the certificate for OBE Toolbox, the on line course, the participants should take a quiz – practically all participants passed the quiz – 100%.

Outcome 1:The outcome clearly states what the participants will be able to do when they attend the workshop and measures the results effectively with a test. 

Outcome #2: The participants who organize and coordinate programs will plan, design and evaluate their programs using a scientific approach to Outcome Based Evaluation.

Indicators: # of instruments used by participants in evaluating their training programs; # of instruments used in planning (gap analysis, needs, etc)

Data Sources: Evaluation reports for their public programs.

Results Summary: We identified, until now, 3 Community Libraries that used the OBE approach in planning and developing their programs. Also, the 6 Adult Learning Centers we have in the system started to use OBE in their programs. We need more time to collect additional data for these indicators.

Outcome 2: The stated outcome is excellent. Better indicators would be the # and % of participants whose subsequent OBE approaches to planning and development were truly OBE in nature and yielded genuine impact data. 

Outcome #3: The participants who write competitive grant applications for funding will use the OBE in their applications.

Indicators: Ratio approved grant versus submitted grant applications.

Data Sources: # of approved grants; # of submitted grant applications.

Results Summary: We need more time to collect data for these indicators. We will try to gather these data starting next week, after the end of the financial year; we will have available data from the annual reports.

Outcome 3: Changes in grants submitted and received may or may not be attributable to use of OBE. Subsequent grants that use OBE to prove program impact on participants would demonstrate the value of OBE to participants.

Outcome #4: OBE will contribute to a shift in the library culture.

Indicators: % increased in employee cultural assessment

Data Sources: Voice of Employee – annual employee survey.

Results Summary: We did second employees’ survey in June 2007and compared the results with 2006. We noticed improvement in several areas: the learning climate got a 4% increase, personal initiative increased by 5%, and the way QL values its employees increased by 4%.

We intend to keep contact with the community libraries from our system in order to assess how they are implementing the skills and knowledge and to serve as a resource for applying all the tools in planning, running, and evaluating their programs. We will develop an online survey to gather information about the way they are applying the gained knowledge and to find out about other training needs they might have.

Outcome 4: It is not clear how OBE training and use contributes to the categories of employee climate listed. Follow-up on how OBE has become part of standard operating procedure in the organization, is valued by the employees, and is going to be continued would indicate a climate change attributable to OBE.

In general this whole grant was very well conceived for using OBE. The plan for future data collection is excellent.

 

OBE Activities

Training Manuals

Certified OBE Trainers

OBE Training Plan

Links

OBE Best Practices home

OBE home

Go to Library Development home page


Last Updated: June 3, 2009