Exploring New Technologies (Mohawk Valley Library System)

Outcome-Based Evaluation
Best Practices in New York State

(Example illustrating Qualitative Results)

Outcome #1: Library staff will feel confident and competent teaching patrons how to use the NetLibrary downloadable audiobooks service.

Indicators: # of patrons who create accounts and use the NetLibrary service; # and % of staff who report feeling confident in their ability to assist patrons in using the service.

Data Sources: usage statistics from NetLibrary; survey of library staff

Results Summary: Our system began offering the NetLibrary downloadable audiobook service on January 1, 2007.  We offered 3 training sessions to library staff, where they learned how to use the service.  Since January 1, 2007, 709 user accounts have been created, and over 2,400 audiobooks downloaded by patrons.  We developed a web page with instructions, frequently asked questions, and troubleshooting tips for both library staff and the public to refer to when using the service.  In a recent survey, 74% of staff who responded report that they feel confident in their ability to assist patrons in the use of the NetLibrary downloadable audiobook service.

Outcome 1: While this is a “confidence” outcome, it was selected because imbedded in the outcome was a way to see if workshop participants used what they learned. The creation of new accounts and downloading activity show impact. Asking participants about how the web page helped them and asking patrons about their use of audio books and the value of the help they received from library staff would have also have captured program impact. It is not clear what the predictions were and how close the program came to realizing expectations.

Outcome #3: Library staff will become familiar with emerging technologies (ex. blogs, wikis, social software sites, etc.) and identify ways to apply these technologies in their libraries.

Indicators: # of staff who attend emerging technologies training sessions; # of training sessions held; % of library staff who are aware of emerging technologies before and after the training; # and % of libraries that use a blog or other emerging technology as part of their library’scommunication efforts before and after the training. DataSources: attendance sheets from workshops; feedback forms from workshops; technology training assessment; feedback from library staff.

Results Summary: We identified a need for staff training on emerging technologies based on the results of our technology training needs assessment conducted in December 2006.  We partnered with Southern Adirondack Library System, Upper Hudson Library System, and the Capital District Library Council to hold 2 workshops on Library 2.0 technologies in May 2007.  These workshops were presented by Michael Stephens, a national expert on Library 2.0 technologies, author of the tametheweb.com blog, and an instructor in the Graduate School of Library and Information Science at Dominican University in Illinois.  18 MVLS system and member library staff attended one of these workshops.

Outcome 3: While this is basically a satisfaction outcome, a portion of the results, namely how those who learned about blogs are now using them to communicate does show that there is evidence that some learners did put what they learned into practice. Asking those learners about the success of the new form of communication would yield program impact. The surveys also show that workshops taught new concepts. Finding out about how the new concepts were applied in the workplace would also yield program impact.
Feedback from these workshops:: Of a total of 132 evaluations returned, 118 attendees (89%) gave the program an overall rating of excellent.  24 attendees (18%) reported that they knew only 0-25% of the workshop content prior to the workshop; 53 attendees (40%) knew 25-50% of the content prior to the workshopFollowing these programs, we re-issued the technology training assessment that we conducted in December 2006, with hopes that the number of staff who had never heard of a technology in December were now at least marginally familiar with it.  In December, 77% of staff that completed the assessment had never heard of RSS feeds; following the training, that number had decreased to 56%.  Similarly, 74% of respondents in December had never heard of shared bookmarking sites like Del.icio.us; that number decreased to 53% after the training. We held 1 hands-on session to teach library staff how to create and manage a blog in June.  17 MVLS system and member library staff attended that workshop.  Since the workshop on June 14, 6 member libraries (60% of the member library buildings represented at the workshop) have begun using a blog to communicate with their patrons.

OBE Activities

Training Manuals

Certified OBE Trainers

OBE Training Plan

OBE Best Practices home

OBE home

Go to Library Development home page

Last Updated: June 3, 2009