Plan of Service Evaluation Project Summary
This document also available in 
.PDF format [138k]
CONTENT
Summary of Results
- Online Survey Findings
- Respondents
- Purpose
- Sections and Elements: Public Library Systems
- Sections and Elements: Reference and Research Library Resources Systems
- Sections and Elements: School Library Systems
- Revisions
- Other Comments
- Follow-Up Telephone Interview Findings
- Interviewees
- Levels of Service
- Revisions
- Public Library Systems: Central Library Plan and Direct Access Plan
- Reference and Research Library Resources Systems: Some Overlap Among the Elements
- School Library Systems: Member Plan and Cooperative Collection Development Plan
- Other Comments
APPENDICES
Appendix A: 2006-2011 Plan of Service
Appendix B: 2006-2011 Plan of Service FAQ for Public Library Systems
Appendix C: 2006-2011 Plan of Service FAQ for School Library Systems
BACKGROUND
All library systems (Public Library Systems, Reference and Research Library Resources Systems, and School Library Systems) are required by New York State Education Law and Commissioner’s Regulations to submit a Plan of Service to the New York State Education Department, New York State Library, Division of Library Development. The 2006-2011 Plan of Service has a basic, shared structure for all library systems, as well as unique elements for each type of system (see Appendix A: 2006-2011 Plan of Service, Appendix B: 2006-2011 Plan of Service FAQ for Public Library Systems, and Appendix C: 2006-2011 Plan of Service FAQ for School Library Systems). The planning document has four purposes:
- Articulate the basic agreement between the State and the governing board of the system, which makes possible the payment of State Aid to the system
- Enumerate the mutual commitments, responsibilities, and obligations of the system and its members
- Share the system's mission, goals, intended results, and evaluation methods with its members
- Determine how the system meets the service needs of its community, region, and the State
With the next Plan of Service due in 2011, the New York State Library’s Division of Library Development formed an internal Plan of Service Implementation Team and hired an independent evaluator to evaluate the content and format of the 2006-2011 plan. The findings from the evaluation (summarized in this document) will serve to inform revisions to the 2011-2016 Plan of Service.
EVALUATION PROCESS
In August 2008, Amanda R. Latreille (AmaLat Consulting) responded to a Request for Quote from the New York State Library to assist the Plan of Service Implementation Team with an evaluation of the content and format of the 2006-2011 Plan of Service. Mrs. Latreille was the successful vendor, and she began work in November 2008.
The evaluation of the Plan of Service was conducted using two methods:
- An online survey
- Follow-up telephone interviews
The online survey was distributed to all library system directors in March 2009 via email. The comprehensive survey covered the Plan of Service’s purpose, sections and elements, and revision process. The survey provided an opportunity for respondents to share comments and/or suggestions for improvement.
The survey’s response rate was high, with 59 out of the 78 library system directors (or 76%) completing the survey.
The follow-up telephone interviews were conducted in April and May 2009 with a sample of survey respondents. These respondents self-identified in the survey as willing to participate in a follow-up interview, and 18 were selected based on system type, geographic location, and other criteria.
The interviews included standard follow-up questions (about levels of service and revisions, for example), but also allowed for discussion and further exploration of individual survey responses. Interviewees were again encouraged to share any additional comments and/or suggestions for improvement regarding the Plan of Service.
Survey and interview responses were anonymous. Mrs. Latreille collected survey responses and conducted the interviews. Aggregated information was shared with the Plan of Service Implementation Team.
SUMMARY OF RESULTS
Online Survey Findings
Respondents
As mentioned previously, the response rate for the survey was high (76%), with 59 out of 78 library system directors completing the survey. Details regarding the 59 respondents, as collected in the survey, are as follows:
- 16 Public Library System directors responded (out of a 23 or 70%)
- 7 Reference and Research Library Resources System directors responded (out of 9 or 78%)
- 36 School Library System directors responded (out of 46 or 78%)
- Respondents’ experience in developing a Plan of Service ranged from none to 5 times or more
Purpose
The first portion of the online survey covered the purpose of the Plan of Service. Respondents were asked if the current format fulfilled the purpose (as outlined in the introduction of this document). The vast majority of survey respondents answered yes (56 respondents out of 59 or 95%).
When asked how the Plan of Service is used with the library system’s member libraries, respondents’ most common responses included:
- As a guide or planning tool (36 respondents out of 59 or 61%)
- At Board, Council, Liaison, and other meetings (14 respondents out of 59 or 24%)
- The majority of respondents refer to their system’s Plan of Service a few times a year (41 out of 59 or 69%)
Sample comments:
- We use it as a framework when developing services with our libraries, to guide the work and to insure that it fits within our goals and objectives.
- It outlines the services we provide to our member libraries and who is responsible for what. It is used at every Council and Liaison meeting to track our progress.
- It is shared with Council and available on our web site. It is a basic guide for our activities and services.
Sections and Elements: Public Library Systems
The sections and elements portion of the online survey was customized for each type of library system, as each type is required to complete a different set of elements in the Plan of Service based on their specific roles, services, and programs, and statutory and regulatory requirements.
The following are highlights from the Public Library Systems’ sections and elements portion of the survey:
- 13 respondents (out of 16 or 81%) stated that the sections and elements allow for adequate description of the system's structure and projected services and programs
- No major trends in respondents’ suggestions for adding or removing elements were noted
- 13 respondents (out of 16 or 81%) stated that the Central Library Plan link should be submitted with the Plan of Service
- 16 respondents (out of 16 or 100%) stated that the Direct Access Plan link should be submitted with the Plan of Service
Note: View the Public Library System elements.
Sections and Elements: Reference and Research Library Resources Systems
The following are highlights from the Reference and Research Library Resources Systems’ sections and elements portion of the survey:
- 4 respondents (out of 7 or 57%) stated that the sections and elements allow for adequate description of the system's structure and projected services and programs
- Some overlap among the elements was a major trend noted by respondents
Sample comments:
- Overlap of many areas (i.e. Element 1 and 2). So much is related to Technology Services. Doesn't include services that are for community created through library collaboration. Element 9 is cooperative efforts with other library systems - what about coop efforts among libraries that are not union catalogs, resource sharing, coordinated services, etc?
- There is considerable overlap between the sections and elements. One question that I always ask is: who views this information and how is it used? Who needs the information we provide? A streamlined plan could contain four elements: member services; resource-sharing and outreach; administration; and advocacy and external relations.
- Technology: should be integrated in other areas and not identified as its own category. The topics identified above under technology are all resource sharing initiatives. So the question is which category do I put them in - both? Seems more work than necessary. Digitization is a technology but also a resource sharing initiative. Where do I put it?
Note: View the Reference and Research Library Resources System elements.
Sections and Elements: School Library Systems
The following are highlights from the School Library Systems’ sections and elements portion of the survey:
- 28 respondents (out of 36 or 78%) stated that the sections and elements allow for adequate description of the system's structure and projected services and programs
- No major trends in respondents’ suggestions for adding or removing elements were noted
- 26 respondents (out of 36 or 72%) stated that the Member Plan link should be submitted with the Plan of Service
- 22 respondents (out of 36 or 61%) stated that the Cooperative Collection Development Plan link should be submitted with the Plan of Service
Note: View the School Library System elements.
Revisions
The last portion of the online survey covered the Plan of Service revision process. Respondents were asked if they were aware that revisions can be submitted to the Plan of Service, and 53 respondents (out of 59 or 90%) answered yes. Seventeen respondents (out of 59 or 29%) also stated that they have submitted revisions to their current Plan of Service, and 31 (out of 59 or 53%) expected to submit revisions. No major trends were noted in respondents’ comments about the revision process.
Other Comments
Throughout the online survey, respondents had several opportunities to answer open-ended questions and provide comments and/or suggestions for improvement. Though there was a great deal of inconsistency in these comments, a few additional trends (beyond those already discussed) did emerge. These trends include:
No Changes
Nineteen respondents (out of 59 or 32%) did not suggest any changes to the Plan of Service.
Sample comments:
- Keep it exactly as it is. Do not change anything.
- Frankly, it actually works pretty well for us (all things considered).
- The current plan is effective. Please don't fix it if it isn't broken! With the financial difficulties we'll all be facing, it is important to have a document that doesn't make additional work or add to an already overloaded day.
Vendor Software
Although it was made clear that the functionality of the Plan of Service software was not part of the online survey, some respondents chose to comment. The vendor has made improvements to the software, which will be phased in as applications are released.
Ten respondents (out of 59 or 17%) stated that the Plan of Service software is difficult to use. Six respondents (out of 59 or 10%) stated that the format could be more user-friendly and easier to print and share with members and others.
Duration and Frequency
Eight respondents (out of 59 or 14%) stated that the Plan of Service time period should be shortened. Seven stated it should be shortened to three years, and one stated it should be updated annually. Respondents pointed to the economy and/or changes in technology as reasons to make this change.
Sample comments:
- The Plan of Service should cover three years not five.
- The time frame of five years, given the economy, both on a state and national level, as well as the changes in technology, make it extremely difficult to plan in this period of time.
- In this economy, a five year plan is difficult to develop, a shorter - three year plan would be advisable.
Follow-Up Telephone Interview Findings
Interviewees
The 18 follow-up telephone interviewees consisted of:
- 7 Public Library System directors
- 3 Reference and Research Library Resources System directors
- 8 School Library System directors
Levels of Service
The Plan of Service Implementation Team asked the evaluator to include a question in the interviews about levels of service. This was a new component added to the 2006-2011 Plan of Service format to ascertain what services, if any, were being provided by library systems to members for a fee.
In recent years with systems experiencing funding constraints and cuts, some systems have had to charge for certain services to cover costs.
When interviewees were asked about the levels of service question, 12 interviewees (out of 18 or 67%) stated that the language needed to change. The most common suggestion was to remove the “levels of service” language, and use the terms “free” and “fee-based” (7 interviewees out of 18 or 39%).
Sample comments:
- "Levels" is negative. It suggests that some members are getting better services than others.
- "Free" and "fee-based" is easier to understand. "Levels of service" is confusing, especially since some systems do have different membership levels (full members, sub-members, for example) which are not necessarily related to fees.
- Just ask "Are there services for which you are charging a fee?"
Revisions
The Plan of Service revision process was explored further during the interviews. The following trends were noted:
- 11 interviewees (out of 18 or 61%) stated that they were not aware that simple changes such as a change in address, members, or website URL should be reported
- 5 interviewees (out of 18 or 28%) stated that it is easier to report these simple changes via email or on the Annual Report
- 9 interviewees (out of 18 or 50%) suggested that the New York State Library send email reminders regarding the revision process and due dates
Public Library Systems: Central Library Plan and Direct Access Plan
The Plan of Service Implementation Team decided to consider the submission of the Central Library Plan and Direct Access Plan required for Public Library Systems. Interviewees were asked if these plans should be reviewed and updated every five years with the Plan of Service.
Six interviewees (out of 7 or 86%) answered yes that the Central Library Plan and Direct Access Plan should be reviewed and updated with the Plan of Service. Three interviewees (out of 7 or 43%) added that the Direct Access Plan should be reviewed, but changed only as needed.
Sample comments:
- The plans should be done together. We wouldn’t know how else we would do it.
- We should do the plans all at the same time. We are trying to review them every year as well.
- Our Central Library Plan is looked at every year, but our Direct Access Plan is looked at and revised only as needed.
Reference and Research Library Resources Systems: Some Overlap Among the Elements
Since some overlap among the elements was a trend noted in the online survey, the topic was revisited in the interviews. All three interviewees (or 100%) stated that there is overlap and redundancy among the elements for Reference and Research Library Resources Systems. Suggestions for improvement focused on consolidation of elements.
Sample comments:
- It should be down to 4 or 5 elements. Merge Elements 1 and 2 (Resource Sharing, and Technology Services). Merge Elements 4 and 5 (Continuing Education and Training, and Consulting and Technical Assistance Services). Merge Elements 6 and 9 (Coordinated Services, and Cooperative Efforts with Other Library Systems). Merge Elements 7 and 8 (Awareness and Advocacy, and Communications Among Member Libraries and Library Systems).
- Remove Technology and make a part of all elements. Remove Special Client Groups…call Member Service Programs. Same things I said in the survey.
- Technology and Resource Sharing really overlap...lots of cross referencing is what we do now. Special Clients is hard, but we make it fit. It’s difficult to know where to put things…World Cat, for example, could go under Resource Sharing, Technology, or Coordinated Services. How do we know where to put it?
School Library Systems: Member Plan and Cooperative Collection Development Plan
As with Public Library Systems, the Plan of Service Implementation Team decided to revisit the School Library Systems’ additional plans, the Member Plan and Cooperative Collection Development Plan. Interviewees were asked if these plans should be reviewed and updated every five years with the Plan of Service.
Five interviewees (out of 8 or 63%) stated that the deadlines for the Member Plan and Cooperative Collection Development Plan should be staggered and not due when the Plan of Service is due. Three (or 38%) stated that the plans should all be due at the same time.
Sample comments:
- Staggering the due dates is better. Could be before or after…it doesn’t matter.
- Staggering would help. We have two bosses, and it’s just too much. We need to simplify.
- I think the plans should be integrated and done at once. We use the Member Plan to develop our Plan of Service, and I think the CCD Plan should be part of the Plan of Service, too. We could use clearer expectations for the CCD Plan.
Other Comments
As with the online survey, there was little consistency in the additional comments made by the interviewees. No trends were noted beyond those already discussed.
CONCLUSION
Though the evaluation indicated that the current Plan of Service does fulfill its purpose (as outlined in the introduction of this document), it also showed that there is room for improvement with content and format. Specific areas to be considered include:
- Language related to levels of service
- Submission of the Central Library Plan and Direct Access Plan by Public Library Systems
- Some overlap among the elements for Reference and Research Library Resources Systems
- Submission of the Member Plan and Cooperative Collection Development Plan by School Library Systems
The Plan of Service Implementation Team will be reviewing the findings identified in this evaluation during the coming months. All comments will be reviewed in light of statutory and regulatory requirements. The Project Summary will be shared with the Coordinator of Statewide Library Services, the State Librarian, and Division of Library Development staff.
The Plan of Service Implementation Team thanks all of the library system directors who took the time to complete the online survey and participate in the follow-up telephone interviews. Your feedback made the evaluation of the current Plan of Service possible.
[Report prepared by AmaLat Consulting, Elbridge, NY]