Library Services and Technology Act Five-Year Evaluation Report, October 1, 1997 September 2002

"Libraries empower people and library services must be dynamic and responsive to the changing needs of people of all ages and all abilities. A diverse State such as New York, which has both large numbers of people to be served in densely- populated urban areas, as well as large geographic areas with smaller numbers of people, requires a complex network of library services to assist all citizens of the State to locate and use information and services that will help to advance their lives in many ways." -- New York State's first LSTA Five-Year Plan.

The University of the State of New York
The State Education Department
The New York State Library
Cultural Education Center
Albany, New York 12230
2002

Go to Table of Contents This document in .PDF format (822K)

THE UNIVERSITY OF THE STATE OF NEW YORK

Regents of The University

Robert M. Bennett, Chancellor, B.A., M.S. Tonawanda
Adelaide L. Sanford, Vice Chancellor, B.A., M.A., P.D. Hollis
Diane ONeill McGivern, B.S.N., M.A., Ph.D. . Staten Island
Saul B. Cohen, B.A., M.A., Ph.D. New Rochelle
James C. Dawson, A.A., B.A., M.S., Ph.D. Peru
Robert M. Johnson, B.S., J.D. Huntington
Anthony S. Bottar, B.A., J.D. North Syracuse
Merryl H. Tisch, B.A., M.A. New York
Geraldine D. Chapey, B.A., M.A., Ed.D. Belle Harbor
Arnold B. Gardner, B.A., LL.B. Buffalo
Harry Phillips, 3rd, B.A., M.S.F.S. Hartsdale
Joseph E. Bowman, Jr., B.A., M.L.S., M.A., M.Ed., Ed.D Albany
Lorraine A. Cortés-Vázquez, B.A., M.P.A. Bronx
Judith O. Rubin, A.B. New York
James R. Tallon, jr., B.A., M.A. Binghamton
Milton L. Cofield, B.S., M.B.A., Ph.D. Rochester

President of The University and Commissioner of Education
Richard P. Mills

Chief Operating Officer
Richard H. Cate

Deputy Commissioner for Cultural Education
Carole F. Huxley

Assistant Commissioner for Libraries and State Librarian
Janet M. Welch

The State Education Department does not discriminate on the basis of age, color, religion, creed, disability, marital status, veteran status, national origin, race, gender, genetic predisposition or carrier status, or sexual orientation in its educational programs, services and activities. Portions of this publication can be made available in a variety of formats, including braille, large print or audio tape, upon request. Inquiries concerning this policy of nondiscrimination should be directed to the Departments Office for Diversity, Ethics, and Access, Room 530, Education Building, Albany, NY 12234. Requests for additional copies of this publication may be made by contacting the Publications Sales Desk, Room 309, Education Building, Albany, NY 12234.


Table of Contents

  1. Executive Summary
  2. Overall Results
    1. Information Access through Technology
      1. Statewide ServicesLibrary Development
      2. Statewide ServicesResearch Library
      3. Technology Grants Program
    2. Information Access through Special Services
      1. Statewide ServiceLibrary Development
      2. Statewide ServicesResearch Library
      3. Special Services Grants Program
  3. Results of In-Depth Evaluations
    1. Technology
    2. Special Services
  4. Lessons Learned
  5. Brief Review of Evaluation Process

Appendices

Appendix A: Meeting the Needs of All New Yorkers: Library Service in the New Century
Appendix B: NOVEL Implementation Plan
Appendix C: CDA Corp., Final Evaluation Report for the Library Services and Technology Act Program in New York State

I. Executive Summary

The New York State Library used Federal Library Services and Technology Act (LSTA) funds to carry out the goals and objectives of the LSTA Five-Year Plan for the period October 1, 1997 to September 30, 2002. This report contains an overall evaluation of the impact of LSTA funded programs and services conducted by New York State Library staff and an evaluation facilitator, results of an in-depth evaluation performed by an outside consultant, and lessons learned through both evaluation processes. The evaluation commenced in December 1999 with the planning of an RFP for an evaluation project and was completed in April 2002 with this overall report.

Overall Results

The report highlights considerable impact of LSTA funded programs and services toward accomplishing the two main goals of the five-year plan, providing Information Access through Technology and Information Access through Special Services.

Information Access through Technology

LSTA funds supported statewide services and grants aimed at library user access to electronic information locally, statewide, nationally, and worldwide. The following summarizes the impressive progress on the technology goal.

Statewide Services

  • New Yorks EmpireLink project gives free access to four major databases covering newspapers, health information, general reference, and business information to over 3,600 registered libraries and library systems. Overall usage is significant and growing, currently reaching over 650,000 searches per year for only the first two databases mounted.
  • The State Library has recognized 3491 of 6300 libraries as Electronic Doorway Libraries (EDL), an increase of 35% since 1997.
    • 475 libraries and library systems have obtained over $56.9 million in E-Rate discounts since 1997.
    • 329 public libraries, assisted by the public library systems and the State Library secured $16.6 million in Gates Library Foundation grants for equipment, software, and training.
    • From 1997 to 2002, $21 million in State and Federal funds was awarded to libraries and library systems for technology purposes. Of this total, $10 million was from the Regional Bibliographic Databases Program, $7 million from LSTA, and $4 million from the EDL Initiative. The funding supported such projects as access to electronic resources, virtual catalogs, digitization, training, and retrospective conversion.
  • The New York State Government Information Locator Service (GILS) is one of the top five pages accessed on the State Librarys web site. There were over 1.4 million hits in 2001.
  • Interlibrary loan clients of the State Library surveyed reported 91.1% satisfaction in obtaining interlibrary loans; 90.1%, rating service as timely; 93.8%, praise for interactions with State Library staff; 95% satisfaction with related health information services.
  • In comparison with research libraries nation-wide, the State Library is a strong net lender of materials.
  • 89.9% of users of State Library reference services reported they were very satisfied.
  • Registered borrowers (including libraries) increased by 36.2% over four years; registered borrowers (excluding libraries) more than doubled in four years.
  • The State Librarys web site recorded over 6.6 million web hits in 2001, an increase of 509 percent since 1997.

Technology Grants

During the period October 1, 1997September 30, 2001, the New York State Library administered four annual competitive grant cycles.

  • The main purposes of Technology grants were to:
    • Emphasize the development of electronic content, including both new content and linkages to other content;
    • Support linkages between library catalogs and provide access to a variety of information resources through regional and statewide gateways; and
    • Increase the availability and quality of technology training so that library staff will be able to effectively deliver electronic doorway library services, as well as train their customers in locating, evaluating and using electronic information.
  • Library systems and their member libraries developed a wide range of Electronic Content projects, from designing web pages, digitizing small and large collections, retrospective conversion, enhancing interlibrary loan through delivery of digital content, to developing virtual catalogs. Over the four years, 116 grants were awarded in this category for a total of $3.8 million.
  • Library systems and their member libraries maintained a steady and high interest in using LSTA grants to develop their library staffs skills in searching the Internet, developing web sites and home pages, and in managing their technology infrastructure. During the four years, 100 grants were made for Training projects, for a total of $3.28 million. Some 39,626 users participated in 3848 training sessions.

Information Access through Special Services

Both the grants program and the statewide services work of the State Library provided many opportunities to serve the specific library needs of New Yorkers in the areas of outreach, improving literacy and job readiness skills, and to expand the roles that libraries can play in contributing to the general economic improvement of New York State. The following summarizes overall progress on the special services goal.

Statewide Services

  • The Division of Library Development staff
    • Provided expert advisory and training services, including grant-writing workshops, to libraries and library systems offering special education and outreach library services.
    • Led a partnership effort with representatives of the three types of library systems and member libraries to reinvent and streamline system strategic planning processes for the 21st century. All 74 library systems now have new five-year plans of service.
    • Partnered with the 23 public library systems to assist the States 750 public libraries in meeting new minimum standards of public library service between 1994 and 1999.
    • Set goals for the statewide summer reading program beginning in 1998. The program exceeded the 2001 goal for participation by 18 percent and the number of participants from 1999 to 2001 increased by 335 percent over three years.
    • Organized the Great New York ReadAloud celebrations for 1998, 1999, 2000 and 2001 which reached over 750,000 persons in New York State.
  • In June 1998, the Board of Regents charged a Regents Commission on Library Services to take a visionary look at library services in New York State and to develop a comprehensive set of policy recommendations to improve library services. To address the needs of the resident of New York, the Commission put forth 10 policy recommendations to improve and transform library services in the state. The Board of Regents adopted the report of the Commission on Library Services as official policy in July 2000 and subsequently developed a robust $105 million New Century Libraries budget and legislative initiative.
  • As part of the transition from LSCA to LSTA the Research Library participated for the first time in the State Librarys meetings across the state to discuss with customers recommendations on the services and programs that should be funded through the LSTA program. EmpireLink was an outgrowth of those coalition-building activities.
  • The Research Library served special populations including prisoners in State correctional facilities.

Special Services Grants

  • The main purposes of Special Services grants was to emphasize the role that public libraries and public library systems play in providing access to library resources and services that advance and enhance the lives of all New Yorkers.
  • Public library systems used Adult Literacy grants to assist their member libraries to provide library-based adult literacy programs and services for basic reading and English as a second language. In the four years, 26 grants were made for Adult Literacy projects, for a total of $1.28 million.
  • Public library systems used Family Literacy grants to help their members to assist parents and caregivers to understand the importance of reading to children and to help them with their own literacy skills improvement so that they could easily read with their children. Thirty-two Family Literacy grants, including the Statewide Summer Reading Program, over the four years totaled $1.12 million. Some 78,838 users of 208 libraries participated in family literacy projects.
  • Public library systems used Economic Opportunity grants to enable their member libraries to assist individuals to develop job-readiness skills and help small business owners and individuals starting their own find the business information resources they need to succeed. Forty-six projects over the four years totaled $1.84 million. In addition, in FY 2000 and FY 2001, a special invitational grant program was offered to public library systems to collaborate with the New York State Department of Labor (DOL) to train library staff to use specific DOL job-related products and to evaluate them for effectiveness with the public. Five public library systems received a total of $364,439 in grants for this two-year period.

Results of In-Depth Evaluation

The Executive Summary of the outside consultants report noted that "findings from this evaluation indicate that the use of LSTA funds has made a positive contribution to the development of the system for library service delivery at the local, regional and State levels in New York State." (p. iv)

The consultant used six key informant groups to gather data for the in-depth evaluation. They were directors of public library systems, reference and research library resources systems, school library systems, central and co-central libraries, LSTA Grant Project Directors and members of the staff of the New York State Library. The data gathering included interviews, written surveys, focus forums and analysis of data logs maintained by State Library staff.

Working with the State Library staff, the evaluator identified four major long-term outcomes in the LSTA Plan. The following section summarizes some of the findings from the outside evaluation by each of those outcomes.

  1. All New Yorkers will have statewide electronic access to information resources.
    • "The forum participants indicated that among the most important things done by the State Library using LSTA funds has been the development of EmpireLink." (Evaluation of the LSTA Library Services and Technology Act Program in New York State) These findings from the focus forums are consonant with the data provided by the State Library regarding EmpireLink. The reported positive impact of this program are supported by the heavy usage data collected on EmpireLink.
      • "Library Services and Technology Act funds have been used by New York State to further both the digitization of bibliographic and of unique and historical records and to encourage broad access to information. The number of New Yorkers with electronic access to information resources has grown at a strong pace over the past five years." (Evaluation...) The survey respondents indicated there is on-going need for electronic content support. The consultant reported that, "Library systems that have completed conversion are now turning their attention to other tasks related to the electronic medium, for example, the purchase of electronic full-text journals and reference materials, the digitization of historical documents and exploring the issues surrounding e-books." (Evaluation...)
      • Library systems conducted a substantial amount of technology training activities with LSTA funds. This confirms the data reported from the annual reports on the grants. Both the surveys and the focus forms indicate a continuing need for ongoing and increased technology training.
      • The in-depth study yielded insight into an area of need that has grown out of the technology activities undertaken by New Yorks libraries with various means of financial support including use of LSTA funds. Tracking of public use of electronic resources whether electronic reference, web activity, or database use is a critical issue because it is important to capture the value to the public and because a substantial amount of professional work goes into provision of the services.
      • Substantial numbers of directors of all library system types reported a concern about telecommunications support for library technology threatening system ability to continue to deliver high quality technology services. "Focus forum participants noted that it is this area [telecommunications] in which they need the most technical assistance, and it is this area in which expert help is unlikely to be resident at the New York State Library, and where consultant experts are extremely expensive." (Evaluation...)
  2. Libraries and library systems will deliver programs that meet and anticipate constantly changing needs for library services.
    • Both in the pre-survey interviews and the focus forums carried out as part of the in-depth evaluation, library system directors and the directors of member libraries indicated that there had been a significant change in services for users and professional practice over the five year period covered by the present LSTA Five Year Plan.
    • "The seventy-six focus forum participants were clear in their reporting that their library systems and system member libraries provide extensive services to new populations. libraries are increasingly perceived by other community agencies as a neutral environment in which information for job seekers, and training in work-readiness skills can be delivered without undue stress to the information seekers." (Evaluation...)
    • "The effect of the Statewide Services is to support the work of the library systems in New York as they work closely with their member libraries to meet the needs of the residents of the state." (Evaluation...) Activity logs completed by members of the New York State Library staff during May 2001 and information collected by the evaluators during interviews of the New York State Library staff in April 2001 confirm the findings of the in-depth study. 67% of the log entries reported interaction with a system director or library system staff person.
  3. New York State libraries, library systems, other library organizations and the New York State Library will strengthen public policy support for libraries.
    • The New York State Library is seen as the source of policy level information for the library service delivery system in the State. In addition, both the survey responses and the focus forums indicated that the need for information regarding public policy issues and proposed legislation both for managers at the three types of library system in the State and for advocates of library services is growing.
  4. 4. All New Yorkers will have access to library resources and services that advance and enhance their lives as workers, citizens, family members and lifelong learners.
    • More New Yorkers have electronic access than had access five years ago, and the quality of that access has improved along with the magnitude of the access provided. Informants to this evaluation reported that they have increased the amount of access in their areas, by using EmpireLink, for example, and they have increased the quality of the services they provide, for example, by collaborating with other libraries and/or service delivery agencies in their regions.
    • Library systems report high levels of collaboration with other libraries both in and outside of their regions to provide greater cooperation and resource sharing.
    • Libraries have become members of community coalitions in order to play an important role in the provision of seamless service delivery at the community level in New York State.

Lessons Learned

This evaluation study has confirmed the value of LSTA funding to New Yorkers. It has also pointed out important lessons about need for change and improvement and for future directions.

Findings and Recommendations

  • Restrictions on the grants categories have somewhat reduced the ability of libraries and library systems to further their established pattern of innovative programming.
  • Focus forum participants noted that they could use better information about the quality and results of past grant-funded programs. They expressed the belief that if the State is to move forward in the innovative environment the program developers at library system and member library level have to receive clearer information regarding what innovations have been tried and tested and with what result.
  • While the communication of grant application rules and grant management procedures is applauded, the evaluation pointed to a need for a more tightly coordinated change process for the LSTA Grants Program and the regular communication of that process as well as any changes in emphasis or objective.
  • In terms of "general communication within the library services system in the State" focus forum participants noted that the use of a listserv to disseminate information to them is not efficient." (Evaluation...) The evaluators recommended "the exploration of new technologies to allow for both asynchronous and synchronous communications." (Evaluation...)
  • The one-year length for grant projects was perceived as a barrier to institutionalization of grant funded innovations.
  • The restriction on paying for existing staff to work on LSTA grant-funded projects was seen by forum participants to negatively impact on the quality of the work being done and on the ability of the systems to institutionalize the work of many of the grants. (Evaluation...)
  • There is a need to strengthen program evaluation. The evaluator who conducted the in-depth study, the evaluation facilitator who analyzed grant reports, State Library data, and the State Library staff all reported on the shortcomings of LSTA program evaluation. The outside consultant made a variety of recommendations for ongoing evaluation of both the Grants Program and the Statewide Services, noting particularly that "measurement of the impact of LSTA supported Statewide Services on the library experience of residents of New York is complex." (Evaluation...)

Summary Statement

The findings of the outside consultant confirmed and reinforced the data gathered and analyzed in the overall review of results and also reinforced perceptions about issues for future consideration.

II. Overall Results

The New York State Library's LSTA Five-Year Plan for the period October 1, 1997 to September 30, 2002 was organized by its two main goals relating to Information Access through Technology and Information Empowerment through Special Services. The first part of this report addresses the overall results in achieving the goals through both Grants and Statewide Services. Under each goal information about the Statewide Services program is presented first organized by the two units of the State Library, the Division of Library Development and the Research Library. The Grants Program information for each goal follows the Statewide Services section.

A. INFORMATION ACCESS THROUGH TECHNOLOGY

1. STATEWIDE SERVICES--LIBRARY DEVELOPMENT

The Division of Library Development provided expert statewide leadership and guidance for the planning and coordinated development of library services for a networked electronic environment. As a part of its ongoing commitment to assist all libraries to become electronic doorways for their users, the Library Development Team, using its strategic plan, Improving Library Services for All, focused on five major activities as outlined in the LSTA Plan.

a. Work with the library community and its supporters throughout the State to ensure that all libraries have the financial support to serve their users as electronic doorways to information locally, statewide, nationally and worldwide.

Results:

1) The State Library initiated EmpireLink, a statewide project to provide access to full-text electronic databases for libraries and their users in 1999. Working with a statewide committee of representatives from all types of libraries, the State Library identified the priority content areas of newspapers, health, general reference, and business. New York's EmpireLink project now provides free access to four databases covering all these content areas, as well as a general one targeted to junior high and middle school students. It also provides free training to over 3,600 registered libraries and library systems. The EmpireLink Project provides a foundation for the New York Online Virtual Electronic Library (NOVEL), providing access to high quality, reliable digital information to all New Yorkers. Access to information through Empire Link is available in some of the following ways:

  • Residents of any community can review the opinions of medical experts.
  • Local business people can access the latest business research from their offices.
  • Students can obtain the different perspectives from newspapers across the nation and the world.
  • Libraries can make all of these databases available on a 24/7 basis to their registered borrowers who can access the information from their homes and offices.

Sample information about types of libraries registered and their usage of some of the databases follows in the next three tables.

  • Gales Health Reference Center Academic (HRCA)--Available since January 1999, HRCA is a source for both medical care professionals and consumers about important health information. The table below shows that many public, hospital/medical centers and school libraries are registered to use this database.

EmpireLink Accounts Using Health Reference Center Databases

June 1999-May 2000

Public

Systems/

Regions

Libraries

Schools

Academic

Hospitals

Consortia

Other

Capital District

41

90

26

16

10

13

Central

86

175

15

11

8

5

Long Island

69

75

25

11

11

4

Metro

77

137

47

34

10

21

North Country

65

97

12

4

12

3

Rochester

41

95

11

7

11

3

South Central

90

138

20

17

11

7

Southeastern

76

48

17

11

10

3

Western

60

75

21

15

14

0

 Totals

605

930

197

126

105

59

The usage statistics, currently summarized for June 1999-May 2000 show that this database is heavily used.

Session: each time a user enters a database
Connect time: in minutes
View: on-screen displays of articles
Retrievals: each article printed or e-mailed through the service

Health Reference Center Usage June 1999-May 2000

Connect

Average

Regions

Sessions

Time

Time

Views

Retrievals

Capital District

27,638

290,433

11

76,045

14,394

Central

26,168

268,914

10

68,198

10,414

Long Island

38,534

471,621

12

115,692

14,156

Metro

80,664

848,848

11

211,174

27,726

North Country

16,721

168,474

10

39,834

4,751

Rochester

36,630

369,971

10

92,532

14,998

South Central

33,742

350,849

10

95,169

12,638

Southeastern

27,795

294,413

11

69,659

8,713

Western

30,358

324,755

11

80,800

10,700

Totals

318,250

3,388,278

10.6

849,103

118,490

  • Dialog @ CARLBasic Collection. Available since January 2000, this database contains, among other databases, about 75 national and international newspaper and newswire databases. The usage statistics show that many high school, academic and public libraries make heavy use of this database. The following chart illustrates the growth in usage of the Dialog database.

EmpireLink: Dialog Statistics (bar chart)

Searches w/ Hits +15.4%
Searches w/Zero Hits +11.6%
Total Searches +13.9%
Total Sessions +24.8%

  • EBSCOhostAvailable since April 2001, this source contains over 750 journals on a broad range of topics including general reference, social sciences, business, health, general science and current events. New Yorks contract also includes Primary Search, reference materials geared to elementary and middle school age users, and Topic Search, a full-text resource for middle school students and above. Statistics on usage of this database are just beginning to be collected, but there are many positive comments on the content from school, public, and academic libraries.
  • Gale Business & Company Resource CenterAvailable since March, 2002, this database provides detailed company and individual news and information. Most of EmpireLinks over 3,600 registered users are expected to sign up for this new database. Also available since March, 2002, Gale InfoTrac Junior features titles most critical to school library media centers in support of curriculum. It includes selected full-text newspaper articles and maps.

Overall the usage of the first two databases is significant. Using a sample of usage data for the Health Reference database for 2001, it appears that usage of that database has increased by 10 percent. Dialog usage increased by 24.8 percent. The following chart shows actual growth in Dialog usage and extrapolated growth in Health Reference usage.

Growth in EmpireLink Usage bar chart

Overall the number of libraries taking advantage of EmpireLink is also increasing. The State Librarys goal was to reach 3625 libraries by July 2002. The following illustrates progress toward that goal. As of April 2002, the State Library exceeded the goal with 3674 libraries registered.

graph shows progress towared goal of 3625 libraries registered for EmpireLink

EmpireLink "train the trainer" sessions are provided for each new database in all regions of the State. More than 70 training sessions have been held for over 1400 people. The State Library maintains an EmpireLink Help Desk with toll-free service and a listserv to respond to questions and concerns about EmpireLink service. Following are sample comments received from staff of various libraries:

  • "Ive worked in college libraries where dozens of databases are available. None of that is as exciting and amazing to me as having Health Reference Center and Dialog available in the Bayville Public Library. The ability to provide this to our customer is fantastic. This is a good thing for this library and this community." Public Library
  • "I want to take a minute to thank you and your staff for all of your assistance with EmpireLink. In a rural area like ours, these databases provide information access to students that would otherwise have none. Without the State Library and your efforts, these small school districts could not afford to subscribe. Thank you so much. Whenever a new project is begun, there will always be some naysayers. Please know that there are a great number of silent librarians out there who are most appreciative, but are just too busy to write and tell you so. I hope you will remember this small token of thanks when you are experiencing one of the endless technology bugs in getting all 7000 of New York States libraries connected. Perhaps it will help you to understand that we really do appreciate all your efforts." School Library System
  • In a letter to the director of South Central Regional Library Council: "Thanks again for helping us get access to the HRC-Academic database, and for scheduling the users workshop so soon after its availability. We have already used it several times. I know you sometimes collect success stories, so I thought Id relate and example. A woman in Apalachin was referred to us through her local fire and ambulance squad. She was worried about a neighbor who has lung cancer and related respiratory problems. The patients husband insists on heating the home with an unvented kerosene heater located in the room she spends most of her day. He refused to check with her doctor about it, and dismissed concerned friends with Show me where it says Although the information we found on InfoTrac wont necessarily change his mind, hopefully it will at least raise enough doubts so that they check with a doctor. Thanks again for this great resource!" Special Library

2) Library Development implemented 19 of 23 recommendations in the Third Statewide Library Technology Plan, Doorways to Information in the 21st Century: Every New York Library an Electronic Doorway Library, issued in April, 1998.

3) Through the Electronic Doorway Library (EDL) Recognition Program, Library Development recognized 3,491 of the State's 6,300 libraries as basic EDL libraries, an increase of 35 percent since 1997. The percentage breakdown by type of library is: public libraries--99%; academic libraries--86%; school libraries--49%; and special libraries--25%.

4) The State Legislature provided a total of $4 million in funding for EDL purposes in two consecutive years. The current funding initiative for technology is the NOVEL (New York Online Virtual Electronic Library) component of the Regents New Century Libraries legislative proposal, which builds upon the EDL Services bill.

5) Library Development assisted 475 libraries and library systems to obtain over $56.9 million in E-rate discounts since 1997.

6) Library Development worked with the public library systems to assist 329 public libraries to secure $16.6 million in grants for equipment, software and training from the Gates Library Foundation.

7) Based on the New York State Board of Regents Commission on Library Services' vision in Meeting the Needs of All New Yorkers: Library Service in the New Century, the State Library worked with representatives of the library community to develop the NOVEL Implementation Plan in 2001. The Steering Committee for that initiative will begin its work in 2002. (See Appendix A for a complete copy of the Regents Commission report. See Appendix B for a complete copy of the NOVEL plan.)

b. To award grants to libraries and library systems under the LSTA technology category and under the State-aided Regional Bibliographic Databases (RBDB) Program and the EDL Initiative.

Results:

1) From 1997 to 2002, the State Library awarded $21 million to libraries and library systems for technology purposes. Of this total, $10 million was from the RBDB Program, $7 million from LSTA, and $4 million from the EDL Initiative. Funding from these sources has helped to provide the following:

  • Access to full-text electronic resources
  • Catalogs and/or other electronic links to resources
  • Z39.50-based virtual catalogs
  • ISO 10160 and 10161 protocol-compliant interlibrary loan software.
  • Digitization of selected resources
  • Retrospective conversion of library records
  • Technology training
  • Distance learning for training purposes (e.g., web sites, mobile laboratories, videoconferencing systems

c. To provide advisory services to library systems and libraries on the further development of their resources and services in a networked environment, and on the preservation of and access to endangered information resources.

Results:

1) Library Development staff worked closely with the reference and research library systems to develop their technology plans under the RBDB Program and to integrate those plans with the new system Plans of Service which were completed in 2001.

2) During this five-year period, Library Development staff approved two sets of three-year E-Rate technology plans for the public library systems and reference and research library resources systems. These systems, in turn, certified technology plans for many of their member libraries for these same periods.

3) Library Development staff advised the Gates Library Foundation on the networking status of public libraries and systems in the State as part of implementation of the $16.6 million Gates Statewide Partnership Grant project.

4) Library Development staff contracted with the Northeast Document Conservation Center (NEDCC) beginning in 1999 for the provision of expert advisory and training services on conservation/preservation. As a result, NEDCC conducted 9 workshops for 252 library staff on topics such as disaster planning, preservation of scrapbooks and photographs, preservation planning, and mold prevention and treatment.

d. To provide a comprehensive library data collection and dissemination program that uses current technologies to make information broadly accessible to the library community and the public.

Results:

Library Development collects data from all types of libraries and library systems to support local, State and Federal policy decisions. The bulk of that data is from public libraries, and since 1997 LD has been moving steadily to automate the collection of data from those libraries. In 1998 LD negotiated with Management Dynamics (now Informata, a subsidiary of Baker and Taylor) to customize their Bibliostat Collect software to use in collecting New York's public library annual report data and to disseminate that data for analysis and comparisons using their CD-ROM product, Bibliostat Connect. Beginning with the 2000 annual report, all 750 public libraries submitted their annual report data using a web-based version of Bibliostat Collect. A web-based version of Bibliostat Connect will be available in 2002 and will include all of New York's 1998, 1999, and 2000 data as well as annual report data from 32 other states.

2. STATEWIDE SERVICESRESEARCH LIBRARY

a. Improve access to government information and unique or unusual materials and collections for libraries and independent researchers through its online integrated catalog/information system, Excelsior, and other electronic information resources.

c. Improve access to government information through the Information Locator Service, a unique online Internet-based directory functioning as an electronic doorway for helping people determine where to access services and information resources available from State agencies, the State Legislature, and the State judicial system.

d. Provide a gateway to databases, like the GPO Access on the Web, which offers boolean searching and full-text access to selected Federal government databases for citizens and libraries in the State. (Activities a., c., and d. have been combined in the results section below.)

Results:

The New York State Government Information Locator Service (GILS) includes direct links to New York State government websites, a subject index to New York State government information, the New York State telephone directory, and links to the New York State Library/Archives online catalog.

The Research Library has responsibility for maintaining current links and collection statistics on GILS usage. In the past two years the GILS site has consistently been in the top ten most-used web pages on the Librarys website. More recently the GILS pages have been among the top five most frequently used. The chart below shows the hits from 1997 to 2001. The drop from 2000 to 2001 may be explained by the fact that the official State home page was launched in 2001.

Year

NYS Government Information Locator Service (GILS) Hits

1997

405,985

1998

2,042,793

1999

2,470,349

2000

2,199,119

2001

1,481,165

The State Library's website provides the general public 24-hour access to many services, some of which are noted below. In 2001, there were 6,631,031 web site hits on this popular service, an increase of 509 percent since 1997.

  • The public can search the catalog (Excelsior) of 1.6 million bibliographic records via the Web, telnet, dialup, or Z39.50 database access, and find sources of information on the topic of their choice.
  • Through the Historic Documents Inventory, the public can search catalog records for more than 25,000 archives and manuscripts collections housed locally at repositories throughout New York State (libraries, historical societies, and other organizations with established archives).
  • Reference requests can be made online.
  • Registered borrowers have 24-hour remote gateway access via proxy server to the Library's subscription research databases and e-books.
  • Registered borrowers can request materials, check the status of their requests and verify what items they have borrowed.
  • Interlibrary Loan requests can be made electronically.
  • New York State documents are currently being scanned from 1995 through the present, and are available online via WebCat and links to detailed information on many subject areas, including databases, genealogies, census, patents, laws and legislative histories.
  • In the New York State Documents section, the monthly and year-to-date cumulative Checklist of Official Publications of the State of New York is online, including a list of all libraries that participate in the New York State Document Depository Program, monthly Shipping Lists, and a list of popular documents.
  • The Federal Depository Library Program section includes a description of the program, information about the State Library's census holdings, a list of the New York State Federal Depository Libraries.
  • The U.S. Government Information page provides links to numerous Federal agency websites, including the GPO Access.

Other popular sections of the website include:

  • Interlibrary Loan pages provide program and contact information for libraries and the general public.
  • The New York Online Virtual Electronic Library (NOVEL) section includes published plans and fact sheets about the program, as well as information about the NOVEL forum held in May 2001.
  • The EmpireLink section provides background and news about the program, online registration forms for libraries, and usage statistics.
  • The Library Development site provides information about its services as well as all State and Federal aid programs including applications for the competitive grant programs.
  • The Virtual Research Library provides an extensive online Internet bibliography.
  • The New York State Government Information Locator Service includes links to New York State agency Web sites, a subject index, and links to agency contact and publications pages.
  • The New York State Newspaper Project section includes information about New York State Newspaper microfilm available at the State Library and around New York State.
  • Manuscripts and Special Collections pages include news and information valuable to researchers, including Finding Aids for specific collections and subject guides such as Selected Civil War Collections Held by the New York State Library, Selected Womens History Collections Held by the New York State Library, and an Annotated Bibliography of Selected New York State Maps: 1793-1900.
  • The Collections section contains a description of the Library's strengths and links to each of the subject areas, some of which are databases, genealogy, census, patents, law and legislative history.
  • The Library's website hosted the Regents Commission on Library Services (RCOLS), which published meeting minutes and testimony, and the final report of the Commission, Meeting the Needs of All New Yorkers: Library Service in the New Century.
  • The New Netherland Project provides information about Dutch history in New York State, project seminars and publications. The NNP can be accessed from a link on the Librarys website.

b. Provide interlibrary loan and reference and information service to improve access to unique or unusual collections onsite or, for remote clients, through e-mail and other modes of communication.

Results:

Interlibrary Loan Service

All New Yorkers have access to the resources of the New York State Library through their local libraries or library systems. These libraries submit requests to borrow materials or obtain photocopies, micro-reproductions or digital images of items in the State Librarys collections. The Research Library is unusual among lending libraries in allowing circulation of its microfilm or microfiche collections, notably newspapers and many genealogy titles. For some unique, valuable or fragile materials the Research Library scans the images and electronically transmits them to libraries.

Interlibrary loan surveys of customer satisfaction conducted in 1997 show a high level of satisfaction with the State Librarys interlibrary loan service. The general survey included responses from customers who use a variety of interlibrary methods, e.g. OCLC, RLIN, and mail. The following summarizes responses to questions about success in filling requests, timeliness of filled requests, and staff courtesy and cooperation as well as action taken following the survey to improve service.

New York State Library Research Library

Interlibrary Loan User Satisfaction Survey

N= 198 out of 465 Interlibrary Loan Clients
NA Responses Eliminated
 

Excellent/Good

Satisfactory

Fair

Success

91.1%

7.3%

1.6%

Timeliness

90.1%

7.1%

2.8%

Staff

93.8%

4.8%

1.4%

 

1997 Interlibrary Loan User Satisfaction bar chart

Selected Comments from Customers: "A high percentage of requests are filled; At times the delivery of resources has been quicker than local services; Even if a request cannot be filled, I know they will exhaust all the possibilities; No complaint, excellent service; Delivery has become extremely fast; Staff always friendly and helpful. Willing to go the extra mile."

A related survey on health information services from the Research Library indicated that 97% of requests were filled in a satisfactory or better manner.

Some changes were made based on issues raised in the two surveys. The State Library joined DOCLINE and updated its holdings in SERHOLD; implemented "Rush" Interlibrary Loan Service Policy; and promoted services in handouts and a newsletter.

Compared to other members of the Association of Research Libraries (ARL), the New York State Library consistently ranks favorably in total number of items loaned and in its net lender statusi.e., in relation to other ARL libraries, it lends a high number of materials and borrows a low number of materials, making it clearly a strong net lender. The following table and graph illustrate the comparison to ARL libraries:

New York State Library Research Library Interlibrary Loan Activity

 

Requests
Received

Requests
Filled

Borrowed

Net

ARL
Lend
Ranking

ARL
Borrow
Ranking

1996-97

94,020

67075

3332

63743

8

109

1997-98

86,838

65732

2622

63110

12

111

1998-99

76,920

53268

2580

50688

19

112

1999-00

72,314

50780

2406

48374

19

112

2000-01

64,314

50924

2329

48595

18

112

Ranking out of 112. Source: ARL Annual Reports

line chart depicting interlibrary loan

The decline in the number of requests received is most likely explained by the increase in point-to-point borrowing (as opposed to sending requests to the State Library) and by the increased availability of full-text online databases.

Reference Services

The New York State Library provides statewide reference service via email, telephone, letter, and onsite. The Library Reference Service responds to questions and refers requesters to other resources where appropriate. During the week of November 6-10, 2000 the Library conducted a survey of individuals using the Librarys services. Customers were asked about their use of the State Library and their information needs, as well as their satisfaction with the quality of the services. Of the people who answered the question on satisfaction, 89.9 percent indicated that they were extremely or very satisfied. A graphic summary of the survey follows:

Overall Customer Satisfaction bar chartTimeliness of Service; Receipt of Materials pie chart

Other surveys on satisfaction with facilities and use of the collections were conducted in the past two years. The results of these surveys have helped to identify collection needs and facilities improvements that are reflected in the revised Collection Development Policy and the planning for redesign of the 7th floor. The State Library plans a customer satisfaction survey at least once a year.

The number of reference questions answered at the State Library has declined; however, ARL statistics show that those numbers are declining for most research institutions. The availability of online catalogs, remote access to indexing and abstracting databases and electronic full text resources means fewer trips to the library. The number of email and electronic reference queries, however, is increasing significantly.

In 2001 the State Library implemented a gateway to electronic resources to all registered borrowers, including all staff in the State Education Department. This new service has been enthusiastically received and the usage of the electronic databases is growing rapidly. Overall reference activity has grown and is substantial as seen in the following table and graph. This is particularly significant because a review of only the data that was traditionally captured might suggest a decline in activity and less stress on the reference staff, when in actuality there is more demand for staff intervention. Only the format for delivering the service has changed.

New York State Library Reference Activity

 

Reference & Other Info Transactions

E-mail Reference Transactions

Use of Electronic
Resources: On line Databases & CD-Rom*

Total
Reference
Activity

1996-97

101,569

370

 

101,939

1997-98

117,547

1,264

 

118,811

1998-99

134,586

4,347

 

138,933

1999-2000

126,214

4,160

 

130,374

2000-2001

123,048

3,835

37,655

164,538

*Data gathered beginning 2000

line chart shows NYS Library reference activity

While the number of people using the Research Library onsite is leveling off, the number of registered borrowers including libraries (I) has increased by 36.2 percent and the number of registered borrowers excluding libraries (E) has more than doubled in the past four years.

Research Library

  Registered Borrowers (I) Increase Registered Borrowers (E) Increase
1997 Jan

11,900

 

4,860

 
1999 Jan

14,440

17.3%

6,274

22.5%

2000 Jan

15,141

4.6%

6,803

7.7%

2001 Jan

17,533

13.6%

9,061

24.9%

2001 Aug

18,666

6.0%

10,060

9.9%

5-yr increase  

36.2%

 

51.6%

Registered Borrowers Including Libraries; line chart Registered Borrowers Excluding Libraries; line chart

e. Work with other comprehensive research libraries in New York State to create a New York State digital library of materials relevant to New York State history, commerce, economic development, and other State-related topics that would be available electronically for residents and libraries in the State.

In 1997 and 1998 the eleven Comprehensive Research Libraries in New York State (NYCRL: Columbia University, Cornell University, The New York Public Library, The New York State Library, New York University, Syracuse University, University at Albany, University at Binghamton, University at Buffalo, University at Stony Brook, and University of Rochester) began a collaboration called the "Making of New York Digital Library Project." The purpose of the project was to capture and make accessible unique historical resources of these libraries through digitization. The lack of ongoing funding sources and the turnover of key leadership at each participating institution hindered the progress of the project. However, the Regents New Century Libraries budget and legislative initiative to develop the New York Online Virtual Electronic Library (NOVEL) includes a digitization component that, once funded, should provide some potential resources for this and other collaborative projects.

f. Strengthen partnerships with customer groups such as the Business Council, Chambers of Commerce, the Association of Mayors, and the Association of Towns to target client groups, identify key information needs, and work with them to provide access to the information, primarily through electronic means.

During the later 1990s the number of library consortia and partnerships expanded significantly. Collaborative efforts to leverage library buying power were spearheaded by the New York State Office of General Services (OGS). OGS developed contracts with major vendors of electronic resources and databases that could be used by all types of libraries statewide. Another organization that provided leadership was the New York Consortium of Library Consortia, composed of representatives from academic, school, public and other library consortia. These groups met with vendors and negotiated favorable pricing for various electronic products.

All of these activities helped to pave the way for the development of the EmpireLink project to provide access to full-text electronic information through libraries statewide. EmpireLink built on the partnerships and collaborative efforts by seeking advice on priority databases, negotiating better pricing and conveying information on the initiative to other libraries. The success of EmpireLink can be directly attributed to the connections with these other activities.

g. Work collaboratively with the Temporary State Commission on Geographic Information Systems (GIS), dataset owners, and customer groups to identify dataset needs of organizations looking for collaborators, assist users in locating and using geospatial data, and cooperate with dataset owners and others to ensure coordination and communication and economy of use of GIS information.

The New York State Library was among the first Association of Research Libraries to participate in the Geographic Information Systems (GIS) program and training. Once the service became established in the Research Library there were many requests for maps graphically depicting census, geographic, pictorial and other information. This graphic representation of data continues to serve researchers and library advocates. For a few years the Research Library housed the New York State Geographic Information Systems Clearinghouse that provided a repository for storing and sharing metadata developed by state and local government agencies. The Clearinghouse program became so successful that it was later transferred to the New York State Governors Office for Technology.

3. TECHNOLOGY GRANTS PROGRAM

During the period October 1, 1997September 30, 2001, the New York State Library administered four annual competitive grant programs. The main purposes of the Technology grant section were to:

  • Emphasize the development of electronic content, including both new content and linkages to other content;
  • Support linkages between library catalogs and provide access to a variety of information resources through regional and statewide gateways; and
  • Increase the availability and quality of technology training so that library staff will be able to effectively deliver electronic doorway library services, as well as train their customers in locating, evaluating and using electronic information.

The Technology section had two categoriesElectronic Content and Technology Training. The purpose of the Electronic Content category was to enable libraries to provide library resources in electronic format for local, regional, statewide and global access and to create value-added information products which package information available in libraries or link the user to other electronic sources. In addition, from FY 1999FY 2001, there was a special invitational incentive program under Electronic Content to help school library systems member libraries with retrospective conversion of cataloged records. The LSTA grant provided the incentive, and the school district provided the matching requirement of purchasing all hardware and software necessary to automate the library.

Over the four years, library systems and their member libraries developed a wide range of Electronic Content projects, from designing web pages, digitizing small and large collections, enhancing interlibrary loan through delivery of digital content, to developing virtual catalogs. The retrospective conversion projects administered by the school library systems provided school library users with access to more materials for resource sharing and made the school libraries stronger partners with other libraries in regional and statewide networks. Over the four years, 116 grants were awarded in this category for a total of $3.8 million.

In the Technology Training category, the purpose was to enable library staff to gain expert knowledge to access and evaluate electronic information resources; provide excellent instruction and support of electronic library services to library staff and users both onsite and outside the library; and develop expert knowledge of infrastructure and software to position libraries for full participation in the networked environment. Over the four years, library systems and their member libraries maintained a steady and high interest in using LSTA grants to develop their library staffs skills in searching the Internet, developing web sites and home pages, and in managing their technology infrastructure. Each year the number of applications in this category was significantly higher than those in other categories. During the four years, 100 grants were made for Training projects, for a total of $3.28 million. The following chart shows a summary of all Technology grants from FY 1998 through FY 2001.

Category

Fiscal Year

  FY 1998   FY 1999 FY 2000 FY 2001
 

# Proj.



$ Awarded

# Proj.



$ Awarded

# Proj.



$ Awarded

# Proj.



$ Awarded

Technology

 

 

 

 

 

 

 

 
- Electronic Content

23

$784,292

21

$703,096

18

$649,297

13

$561,674

- Retrospective Conversion

 

 

14

$446,227

14

$349,657

13

$307,955

Technology Training

25

$786,487

26

$818,033

24

$851,577

25

$820,747

Totals

48

$1,570,779

61

$1,967,356

56

$1,850,531

51

$1,690,376

Analysis of Data in Technology Training Grant Reports

This section provides an analysis of data in Technology Training grant reports as an example of the results obtained in grant projects in the Technology category. An intern working for the Division of Library Development summarized the information in the annual reports for the years 1998, 1999, and 2000. The data provided in LSTA annual grant reports was generally inconsistent in the reporting of both quantitative and qualitative results of projects. It is impossible to report with confidence actual totals for items such as project users. However, to totally ignore the data would be an injustice to the important work of the projects. Therefore, data in this analysis are reported with the acknowledgement that they are incomplete. They are provided because they show a picture of activity that supported the goals of the LSTA plan.

Training Projects: Training projects offered training in technology for such activities as Internet access and use, use of electronic resources, word processing, Web design, use of union catalogs, software applications, and use of specialized electronic information (e.g. health, consumer, etc. information). The grants supported such things as equipment, development of training materials (manuals, videos, etc.), training centers, mobile training units, distance learning, and teleconferencing.

Various users were targeted. Often staffs were trained to be trainers with the expectation that they would train other staff and library patrons. In those instances, the training was reported in the annual report but the real benefit took place in the following year when the training was exponential. The follow-up data are not known, but in some cases the report reflected the anticipated results. For example in one grant to train school library media specialists, 28 libraries participated in 8 training workshops and developed 14 web sites. With the trainers trained, they expected to pass that training on to 10,360 students and 725 educators. In other cases, library patrons received the training. For example, in a project to train in vernacular searching of the library catalog and databases, 225 Chinese-speaking, 112 Spanish-speaking; and 61 Korean-speaking patrons were trained.

Many of the projects developed multiple web sites, some reporting web hits. Again many of the web hits on the new sites developed would not be known in the first year of the project, but where the sites were developed in time for some data collection, some systems reported hits.

The following table is a summary of data for 1998, 1999, and 2000. Users are defined as staff or patrons depending on the project. The numbers are given in recognition that not all reported user numbers and that in the follow-up years the user numbers would certainly grow exponentially in the case of the "train the trainer" projects. Sessions are defined as workshops, teleconference programs, classes, lectures, and distance learning sessions. Again data are incomplete. For example, one project trained 1400 users. Each user took 9 classes in an unknown number of sessions so no data are reported as sessions for that system. Web hits are reported when given although many systems reported development of sites without data on hits. Others mounted sites too late for hit data to be collected.

Training Projects

Year

Users

Sessions

Web Site Hits

1998

9,848

1,184

1,670

1999

16,585

1,119

1,089,286

2000

13,193

1,545

57,409

The qualitative data provided in the annual reports included satisfaction surveys. Most of the projects conducted some form of participant evaluation. However most reported results in comment form such as "high satisfaction" or "favorable responses" or "requested more training."

The following is a sample of results when actual numbers were given.

97% workshop interesting
93% improved Internet skills
89% liked hands-on and individual attention
92% gave all 8 questions highest rating of 5
Pre-class survey 64% confused by Internet; Post-class survey 87% felt more comfortable
92% wanted similar workshops
86% recommend to others
98.7% instruction just right
97.7% workshops excellent/good

Many libraries reported anecdotal information about training projects. While such information cannot be quantified, it does give a strong sense of the value of LSTA grant programs. Samples of those anecdotes follow:

  • "The program was just what the doctor ordered. The 6 female patrons [who attended health information sessions] wanted information on breast-feeding, prenatal care, diabetes, and asthma. We were able to obtain information on these subjects using all the databases suggested in the slide show. The chosen databases were at the appropriate level for the layperson and were easy to use." Urban Public Library System
  • "Talking with a patron (home health aide) before class, revealed that she had her first introduction to the use of computers from a paralyzed 7 year old girl she was caring for. The amazing young girl had to use a pointer attached to her head to work the computer, but she was extremely knowledgeable, as was her 3 year old brother, and she taught the patron computer basics. This patron was now attending the e-mail class because she was presently caring for a quadriplegic who wanted her to set up and operate an e-mail account for him. She really didn't know what e-mail was and did not have her own computer. This class proved to be an ideal way for her to learn something that could be of value to her at work for her client and of great value, at the library, for her own e-mail. She was tremendously appreciative at the end of class for all that she had learned." Rural Public Library System
  • "A director from a small, rural library sent an EXCEL spreadsheet of her 2000 operating budget to me. She had this to say: The EXCEL workshop enabled me to set up this new financial report. Thanks for the training." Public Library System
  • "The workshops gave me several new insights on writing techniques and of the importance of incorporating both information and artistic quality in preparing a page. Polly showed us some sites that allowed you to create and manipulate graphics, as well as sites to go to for HTML tutorials. I have already been busy modifying our page to incorporate some of the techniques I learned. The information was very useful and the presentation was excellent. Finding good workshops for web authoring is difficult and it was a great service to provide for your member libraries." Reference and Research Library System
  • Links between local students and students in Beijing were enriched as a result of the workshops, according to one teacher. This teacher described feeling more inventive and less wary about Internet resources and connections. Another teacher described linking with other schools, as well, to see their projects and activities, "bringing the world closer." Rural BOCES School Library System
  • "Almost every time there was a session scheduled at a library, these older adults arrived significantly early for the appointed time. When asked if they had the time wrong, they would say, Oh, Im so excited about this I wanted to be sure to be on time. We cant recall anyone not showing up for a scheduled class in any of the libraries! That, too, is unbelievable. Senior citizens have been waiting for this project, and they really appreciate it." Suburban Public Library System

B. INFORMATION ACCESS THROUGH SPECIAL SERVICES

1. STATEWIDE SERVICESLIBRARY DEVELOPMENT

a. Work with libraries and library systems to help them provide high quality, cost-effective services that meet the needs of their communities.

e. Administer grants to library systems and libraries and provide advisory services to libraries and library systems on the further development of their resources and special services.

f. Provide advisory services to library systems and libraries on the further development of their resources and special services.

h. Collect and disseminate information in print and electronic formats which further the development of special services. (These activities have been combined in the Results below.)

Results:

1) The Division of Library Development librarians provided expert advisory and training services, including grant-writing workshops, to libraries and library systems offering special education and outreach library services. Examples of such services include library-based adult literacy programs and family literacy and youth services.

2) Library Development staff also led a partnership effort with representatives of the three types of library systems and member libraries to reinvent and streamline system strategic planning processes for the 21st century. All 74 library systems now have new five year plans of service. Library Development will soon begin work on reinventing and streamlining the annual application and reporting processes to link to the new plans of service.

3) Library Development staff also partnered with the 23 public library systems to assist the States 750 public libraries in meeting new minimum standards of public library service by 1999. The following charts illustrate progress of public libraries to date in meeting minimum standards.

I. Percentage of Libraries Meeting Minimum Standards for Access to Information

[bar chart showing Percentage of Libraries Meeting Minimum Standards for Access to Information]

1996

1997

1998

1999

2000

Telephone

99.9%

100.0%

100.0%

99.9%

100.0%

Photocopier

97.8%

100.0%

100.0%

99.9%

99.9%

Microcomputer or Terminal

89.1%

91.2%

93.8%

98.9%

99.3%

Printer

93.0%

93.7%

93.8%

96.9%

97.6%

Facsimile Capability

97.2%

97.8%

98.8%

99.7%

99.7%

Percentage of Libraries Meeting Minimum Standards

[bar chart showing Percentage of Libraries Meeting Minimum Standards]

 

1996

1997

1998

1999

2000

Administrative

99.9%

100.0%

100.0%

99.9%

100.0%

Evaluation

97.8%

100.0%

100.0%

99.9%

99.9%

Hours Open

89.1%

91.2%

93.8%

98.9%

99.3%

Facility

93.0%

93.7%

93.8%

96.9%

97.6%

Qualified Director

97.2%

97.8%

98.8%

99.7%

99.7%

4) The following chart illustrates a portion of the advisory services provided by Library Development Staff. The increases in grants (both State and Federal-supported) monitored, site visits made by staff, and events sponsored illustrate the growing involvement of Library Development staff in statewide leadership activities. The visits made represent on-site monitoring, and the attendance figure represents attendees at Library Development sponsored events. The staff logs collected by the evaluation consultant also show a significant workload of advisory services to the library community performed by telephone, e-mail, correspondence, and in meetings with advisory groups. 


Library Development Annual Transactions
Library Development Annual Transactions; bar chart

1997-98

 

Grants

688

Visits

88

Events

39

Attendance

1,166

1998-99

 

Grants

786

Visits

249

Events

74

Attendance

2,429

1999-00

 

Grants

967

Visits

155

Events

21

Attendance

1,312

2000-01

 

Grants

950

Visits

195

Events

297

Attendance

3,757

b. Work for public policy that acknowledges and supports the roles of libraries in a learning society.

The primary way Library Development worked to promote public policy in support of New Yorks libraries was its support for the Regents Commission on Library Services and the resulting New Century Libraries legislation.

Commission Charge

In June 1998, the Board of Regents charged the Commission on Library Services to take a visionary look at library services in New York State and to develop a comprehensive set of policy recommendations to improve library services. The Board asked the Commission to address:

  • equitable library service for all New Yorkers
  • organization and relationship of libraries and library systems
  • financial support for libraries and library systems
  • projections of service needs in the 21st century
  • role of the New York State Library in leading and supporting new directions for library services
  • visibility and public awareness of libraries and library systems as a source of lifelong learning and enrichment,

Throughout 1998 and 1999, the Commission conducted research and surveys, held regional public hearings and had lengthy discussions with stakeholders throughout the State about library services. One survey showed that New Yorkers are willing to spend more on libraries. Of those surveyed, 67% would double the average per capita spending on libraries from $32 to $64. In early 2000 a draft report was circulated and regional hearings were held for comments from the public. A final report, Meeting the Needs of All New Yorkers: Library Service in the New Century with ten policy recommendations to improve and transform library services in the State was adopted as policy by the Board of Regents in July 2000.

Resulting Legislation: New Century Libraries

Library leaders throughout the State collaborated with the New York State Library and the New York Library Association to formulate legislation that would invest an additional $105 million to begin implementing the Commission recommendations. New Century Libraries recognizes the critical role libraries play in education, the economy, and communities. Advocacy for New Century Libraries continues.

c. Support libraries and archives efforts to preserve and make available information of enduring significance for the use of present and future generations.

The Division of Library Development provided technical assistance on conservation and preservation by site visits, telephone, and e-mail to libraries, archives and other not-for-profit organizations. For these organizations $500,000 is available annually in State competitive grant funds. Grants funded preservation surveys, microfilming, reformatting of photographic collections and sound recordings, repair and rebinding of bound volumes, and conservation treatment of books, manuscripts, maps, and architectural drawings. The Division also provided education programs, publications and grant workshops each year.

Annually, $126,000 is granted to each of the eleven comprehensive research libraries for preservation of their collections. These same institutions also competed for additional grants totaling $350,000.

d. Work with SEDs Welfare Reform Workgroup, New York State Literacy Volunteers of America, Laubach Literacy International, the Center for Family Resources and others to promote the librarys role in improving literacy skills and helping to improve economic opportunities for all New Yorkers.

In 1998-99, staff from the Division of Library Development began initial discussions about a collaborative project between the State Library and the New York State Department of Labor. At that time, NYS Department of Labor was developing a new software program for job seekers and job changers called CareerZone. The State Library set aside LSTA grant funds to support a two-year demonstration project involving five public library systems. The systems chosen to participate in the grant were Brooklyn Public Library, North Country Library System (Watertown), Chautauqua-Cattaraugus Library System (Jamestown), Westchester Library System, and Mid-Hudson Library System (Poughkeepsie).

Professional librarian project directors were employed by each system to plan and provide for training sessions on the use of CareerZone, install the software in member libraries, and beta test the software. The librarians provided feedback to the Department of Labor on the software as it was being further developed and refined by the labor analysts. The project concluded with a joint presentation by the library system project directors and Department of Labor analysts and a staff member from the Division of Library Development at the New York Library Association annual conference in Albany in October 2001.

g. Work with SED, the Department of Labor and other organizations on planning for and implementing New Yorks plan for a re-engineered Workforce Development System.

Results:

In 1999, a Library Development staff member was appointed to the SED committee working with the Department of Labor on how the educational community could participate in the Workforce Development System. As a result, several public library systems became active in local Workforce Investment Boards, strengthening the delivery of employment information to jobseekers and career changers.

i. Work with the library community and others to plan and implement statewide and local events that encourage reading, including the Statewide Summer Reading Program and the Great New York ReadAloud.

Results:

LSTA funds supported two major statewide programs that encourage reading and promote literacy: the Statewide Summer Reading Program and the Great New York ReadAloud.

Statewide Summer Reading Program

This program is based on studies concluding that children who continue to read during the summer when school is not in session perform better in the fall when school resumes. The New York State Library partners with public library systems and a statewide committee of youth services librarians to develop programming materials organized around compelling themes that vary each year. Kits that include program manuals, attractive posters, and engaging educational activities are distributed to all 1,200 public library, branch library, and reading center locations in the State, making available a professional quality reading program to every community, no matter how small or remote its library may be. The programs are free of charge to all children from toddlers to teens, and parent participation is encouraged.

A corporate partner, Janway, works with an artist to produce a poster, related artwork and prints and distributes the materials to the libraries. They also produce t-shirts and other incentive items for libraries to purchase (if desired) at very economical prices.

The role of the local library is pivotal. Local librarians contribute ideas for the activities manual and creatively adapt and us the program to meet local needs and interests. The libraries contribute local funds and involve library staff, local businesses, and other organizations to implement the program. Local library staff help children select reading materials and provide such literacy-enhancing programs as storytelling, music, and puppet shows.

Data are available for Summer Reading Program activity for the years 1999 through 2001. The Division of Library Development set goals for the summer reading program through 2003. The following graph illustrates that the program exceeded the 2001 goal of 400,000 participants by 18 percent, and the number of participants from 1999 to 2001 increased by 335 percent.

The Division of Library Development has set a goal of 1 million participating children by Summer 2003. The following graph shows progress toward that goal.

Number of Particupants in Statewide Summer Reading Programs; line chart

The Great New York ReadAloud

This LSTA-supported Statewide Reading Program encourages reading celebrations, organized around a different theme each year during National Library Week. These one-day reading celebrations are designed to promote the love of reading, the benefit of reading aloud and the lifelong learning associated with reading. The ReadAloud programs focus activity on the joy of reading out loud.

18,000 posters in three languages -- Chinese, English and Spanish -- are donated annually by SIRS Publishing, Inc. of Boca Raton, Florida. These posters, accompanied by Idea Sheets for reading celebrations, are mailed to 6,000 school library media centers and public libraries early each year.  The Statewide ReadAloud Coordinator provides technical assistance in planning and implementing reading celebrations upon request by local libraries and library systems.  The ReadAloud celebrations for 1998, 1999, 2000 and 2001 reached over 750,000 persons in New York State.

This program also demonstrates the power of collaboration. In addition to the corporate sponsor, SIRS, other sponsoring organizations include the School Library System Association, the New York Alliance for Arts in Education, the National Education Association of New York, New York Library Association, the New York State Assembly Standing Committee on Libraries and Educational Technology, New York State Association of Library Boards, the New York State Congress of Parents and Teachers, the New York State Education Department, the New York State Library Assistants Association, the New York State Reading Association, the New York State Senate Subcommittee on Libraries, New York United Teachers, and the United University Professions.

The pre-event brochure, idea sheets and poster images are available on the Library Development web page at: http://www.nysl.nysed.gov/libdev/readaloud/racover.htm

2. STATEWIDE SERVICESRESEARCH LIBRARY

a. Build coalitions with outside customer groups to identify unmet information needs of persons in rural, underserved or inner-city areas.

Results:

As part of the transition from LSCA to LSTA the Research Library participated for the first time in the Library Development sponsored meetings with the library community across the state to discuss customers recommendations on the services and programs that should be funded through the LSTA program. Among the many recommendations was the need to support a program to offer statewide access to full-text electronic information similar to what other states were providing. Shortly after these meetings the Research Library began a strategic planning process that included customer focus group meetings. In these meetings as well the interest in having increased access to electronic information was a strong recommendation. The recommendations from these meetings and focus groups converged to form the basis of the State Librarys EmpireLink initiative which has been described on an earlier page of this report.

b. Form partnerships with other key information providers to improve access to needed information and publicize the Librarys collections and services.

Results:

The State Library contains many significant and unique collections related to history, law, standards, and medicine. These collections include manuscripts, rare books, microforms, and Federal and State documents. To provide increased awareness of these materials the Library made its catalog available from the website. As a result, the State Librarys collections and services are becoming more visible to New Yorkers and researchers in other states. While the State Library does not have the resources to provide reference services to everyone, it does offer reference service for its unique materials.

One of the many valuable documents that the Library owns is the draft copy of the Preliminary Emancipation Proclamation in Lincolns own hand. For many years the document has been stored in the Librarys vault with only occasional viewings. The State Library and the State Museum are working together to determine how to provide more opportunities for the public to see the document while still preserving its integrity.

The State Library hired a communications specialist to develop a public relations plan to increase the visibility of the Librarys collections, services and programs. The plan included enhancing the website, increasing press releases and programs and focusing on unique services and collections. The substantial and growing use of the Librarys website was reported in the Technology section of this report. The web link to the catalog (Excelsior) has provided easier access to the catalog and is receiving increasing numbers of hits on the site.

c. Collect and provide access electronically or through interlibrary loan to high-interest materials in such areas as law, consumer protection and health information.

Results:

1) In an effort to expand access to the collections of the New York State Research Library, the Library initiated a pilot project to allow retired New York State employees to borrow library materials directly. Currently there are 84 retired New York State employees who have signed up as registered borrowers under this program.

2) Another access improvement activity of the Research Library involved working with the New York State Department of Correctional Services to improve access to legal resources for prisoners. The Prisoner Services Project evolved as a result of federal court decisions requiring States to "protect the right of prisoners to access to the courts by providing them with law libraries or alternative sources of legal knowledge."

3) A third access improvement activity was to expand access to the Librarys 10,000 highest priority historical maps. To date, 975 historical maps have been cataloged and added to Excelsior, the Librarys online catalog that is accessible via WebCat on the Internet. The Library is currently making almost 600 additional maps per year accessible to customers.

4) In the past four years the Research Library conducted several surveys to gather data on collection usage or customer satisfaction with services, equipment or facilities. Two major issues identified in the surveysneed for renovation of space and increased hours--have not been addressed because of lack of funds. To address the growing need for information around the clock, the Research Library is making considerably more digital and electronic information available any time any place.

3. SPECIAL SERVICES GRANTS PROGRAM

The main purpose of the Special Services grants was to provide all New Yorkers with access to library resources and services that advance and enhance their lives through public library systems and public libraries. The Special Services section contained the two categories of Literacy, including both Adult Literacy and Family Literacy, and Economic Opportunity.

The Literacy categorys purpose was to enable public libraries to provide programs and services that promote the improvement of literacy skills for people of all ages. The Adult Literacy grants assisted public library systems and their member libraries to provide library-based adult literacy programs and services for basic reading and English as a second language. In the four years, 26 grants were made for Adult Literacy projects, for a total of $1.28 million.

The Family Literacy grants helped public library systems and their members to assist parents and caregivers to understand the importance of reading to children and to help them with their own literacy skills improvement so that they could easily read with their children. This section also supported an annual grant to a public library system to develop a theme and materials for a Statewide Summer Reading Program. Thirty-two Family Literacy grants, including the Statewide Summer Reading Program, over the four years totaled $1.12 million.

The purpose of the Economic Opportunity category was to enable public libraries to assist individuals to develop job-readiness skills and help small business owners and individuals starting their own find the business information resources they need to succeed. Forty-six projects over the four years totaled $1.84 million. In addition, in FY 2000 and FY 2001, a special invitational grant program was offered to public library systems to collaborate with the New York State Department of Labor (DOL) to train library staff to use specific DOL job-related products and to evaluate them for effectiveness with the public. Five public library systems received two grants each for this two-year program for a total of $364,439.

The following chart shows a summary of all Special Services grants from FY 1998 through FY 2001.

Category

Fiscal Year

 

FY 1998

 

FY 1999

 

FY 2000

 

FY 2001

  # Proj. $ Awarded # Proj. $ Awarded # Proj. $ Awarded # Proj. $ Awarded
Special Services

 

 

 

 

 

 

 

 
Literacy

 

 

 

 

 

 

 

 
Adult Literacy

6

$317,805

8

$333,927

6

$352,496

6

$273,335

Family Literacy

6

$294,525

9

$271,389

12

$429,140

5

$122,769

Economic Opportunity
Services to individuals

8

$430,809

9

$417,844

4

$150,491

5

$178,946

Services to businesses

6

$166,807

 7

$187,244

2

$102,661

5

$204,612

Collaboration with Dept. of Labor

 

 

 

 

5

$214,439

5

$150,000

Totals

26 

$1,209,946

33

$1,210,404

29

$1,249,227

26

$929,662

Analysis of Data in Family Literacy Grant Reports

This section provides an analysis of data in the Family Services grant reports as an example of the results obtained in grant projects in the Special Services section. An intern working for Division of Library Development summarized of the information in the annual reports for the years 1998, 1999, and 2000. The data provided in LSTA annual grant reports was generally inconsistent in the reporting of both quantitative and qualitative results of projects. It is impossible to report with confidence actual totals for items such as project users. However, to totally ignore the data would be an injustice to the important work of the projects. Therefore, data in this analysis are reported with the acknowledgement that they are incomplete. They are provided because they show a picture of activity that supported the goals of the LSTA plan.

Family Literacy: The Summer Reading Program included in a previous section of this report is part of the LSTA Family Literacy initiative, but the data reported here exclude the Summer Reading Program data. As with the Technology project reports, these data are also inconsistent from project to project. For example, many projects involved working with community agencies, but detailed information about the collaborations was not provided. Nonetheless, there is a substantial amount of information provided in the reports that gives a picture of the first year activity of the projects. Many of the projects completed organizational activities in the year of the project and made substantial contacts, but often the bulk of activity for the effort expended occurred beyond the terms of the grant and after the annual report was filed.

Some projects targeted actual users, i.e., parents and children in at-risk families, parents with newborns, families in need of English as a second language instruction, and preschoolers. Others targeted childcare providers and community agencies including some that used a "train the trainer" approach. Staff training was a component of several projects.

The following chart records data that, except for the collaboration and circulation items, represents about 70 percent of the reporting libraries. Only four libraries reported numbers of collaborations and only one library reported circulations.

Family Literacy Projects

1998

1999

2000

Participating libraries

46

70

92

Users

28,405

14,784

35,649

Workshops presented

133

337

322

Staff trained

519

1,008

1,849

New cards issued

193

494

Programs for children

380

951

55

Web sites created

1

2

2

Materials acquired

5,458

6,498

17,042

Publications prepared

20,940

14,328

13,009

Information packets distributed

1,642

7,087

8,393

Circulations

8,142

Collaborating agencies

71

22

158

The following are samples of anecdotes from the Family Literacy projects:

  • "An amazing number of households I visit have no books in the homes. To some a book is a luxury, not a priority, and a free book may be the only one a child has. The gift bags get the parents to go to the library to get a library card and a free book, which results in the parents discovering the excellent sections for kids and parents. This is a great project." Rural Public Library System
  • "Several days after doing a workshop at the Richmond Hill facility, the principal of a local school stopped the Hospitals Public Affairs Associate on the street saying he could not thank Medisys enough for the workshop. As the Public Affairs Associate reported to us, parents talked about it in school and were very excited about sharing books and learning games with their children. Many had not been using the library or reading with their children and had really become encouraged to do so by the program. The principal was so excited by their reactions that he wanted to have another workshop presented in his school so that other parents who had only heard about the program could attend." Urban Public Library System
  • "Mrs. R has 4 children all under the age of six. At the first workshop, she admitted to the group that she herself is not a reader and was not read to as a child. Therefore, she finds it very difficult to choose books for her children though she understands how important it is for them. With the encouragement she received through Bookstart activities and the shared experiences of the other parents in the group, she came to enjoy reading to her family. She was especially proud that her children loved the book she made them during one of our workshops. She took extra materials home with her so that she could help them each make a book of their own." Suburban Public Library System
  • "Math Nights held at the other libraries engaged families in cooperative learning experiences. It was not a program designed to entertain children but an event that encouraged interaction between parents and children. It also promoted the library as a learning environment presenting an image of the library not frequently held by many in the community including teachers. It reinforced the learning partnerships that exist between parent and child, school and home, and school and library." Rural Public Library System
  • "Less than two weeks after taking the Red Cross Infant/Child-saver course, a participant was family camping. While out in the woods, she was able to save her three-year-old daughter from choking on a piece of raw carrot, using techniques shed learned in that class. Perhaps a grant project, no matter how complex, needs no more significant outcomes than this!" Public Library System

III. Results of In-Depth Evaluations

This section of the evaluation report provides an overview of the in-depth evaluation conducted by an outside evaluation firm. The complete in-depth report can be found in Appendix C. Some of the tables are quoted exactly from the complete report, and some have been created from the data.

The State Library in November, 1999 established an Evaluation Committee including members of the LSTA Advisory Council, several State Library staff members, and an evaluation facilitator. This group drafted an RFP that resulted in an outside evaluation consultant, CDA Corp, being hired in November, 2000. To work more closely with the outside evaluator, the State Library formed an internal team of staff members, including a graduate student intern and the evaluation facilitator.

Working with the internal team, the evaluation consultant reviewed the LSTA Five-Year Plan and developed a program logic model with four major long-term outcomes that were selected for in-depth evaluation:

4. All New Yorkers will have statewide electronic access to information resources.
5. Libraries and library systems will deliver programs that meet and anticipate constantly changing needs for library services.
6. New York State libraries, library systems, other library organizations and the New York State Library will strengthen public policy support for libraries.
7. All New Yorkers will have access to library resources and services that advance and enhance their lives as workers, citizens, family members and lifelong learners.

Each of these outcomes had implementation objectives, activities, and outputs, and there were significant crossover relationships among all of these elements. For example, the implementation objective to "expand the capacity of local libraries to provide electronic services," relates to long-term outcomes 1 and 2. As the consultant gathered new data, she looked for evidence that contributed to more than one outcome.

The consultant used six key informant groups to gather data for the in-depth evaluation. They were public library system directors, reference and research library resources systems directors, school library system directors, central and co-central library directors, LSTA grant project directors and members of the staff of the New York State Library. The methodology included interviews, surveys, and focus forums.

In the Executive Summary of the final in-depth evaluation report the consultant indicates that "positive impacts have been felt" as a result of LSTA funding in New Yorks libraries. She states,

"Survey results and focus forum participant responses indicated that LSTA funds facilitated and supported the objectives of the New York State Library Services and Technology Act Five Year Plan October 1, 1997 September 30, 2002. A majority of grant project directors agreed that their system had learned a great deal through the LSTA supported opportunity to innovate. Both the LSTA Grants Program and the development of the state virtual library network in the EmpireLink, and more recently, NOVEL strategies were easily recognized by most key informants as important to the future quality of the library services in New York State. Reporting on the impact of Statewide Services, focus forum participants reported that the impact is on the ability of their system (Reference and Research Library Resources System, Public Library System, or School Library System) to help their system member libraries to accommodate the dynamic needs for library services. They also noted that it is their use of Statewide Services that helps the system directors to support strategies which will stimulate change and transform services in their regions." (Evaluation of the LSTA Library Services and Technology Act Program in New York State, p. i. All subsequent page references are to this document.)

A. Technology

The goals studied in-depth that most related to the technology category were:

  • All New Yorkers will have statewide electronic access to information resources.
  • Libraries and library systems will deliver programs that meet and anticipate constantly changing needs for library services.

Technology -- EmpireLink

The forum "participants indicated that among the most important things done by the State Library using LSTA funds has been the development of EmpireLink." (p.8) They offered reasons for their strongly positive attitudes (e.g. "EmpireLink is one of the best things to come out of LSTA"):

  • "equal access afforded to any library of a free full-text database such as EmpireLink actually functions as a quality equalizer which helps all libraries meet a minimum standard of service.
  • In the case of large systems and libraries, the money saved by using a state supported database allows them to allocate their limited resources to support other services and programs.
  • Without the state supported database small libraries would not be able to afford any database at all." (pp. 8-9)

These findings through the focus forums are consonant with the data provided by the State Library regarding EmpireLink. The reported positive impacts of these programs are supported by the heavy usage data collected on EmpireLink.

Technology --Other Statewide Technology Services

The consultant surveyed directors of all library types about other statewide services that support the goals of the in-depth study. The survey findings were reported as follows (p. 11):

Table 2: Use of Statewide Services by System Director Type

Statewide Service

Reference and Research Library Resource

Public Library Systems

School Library Systems

Central and Co-Central Directors

Reference Related Services
Electronic Inter-Library Loan

77.8%

12.5%

78.9%

42.9%

Reference Information

0

37.5%

52.6%

28.6%

Obtaining cost free access to full-text electronic databases

77.8%

87.5%

89.5%

21.4%

Funding Related Services
Technical Assistance with State aid

11%

81.3%

84.2%

57.1%

Technical Assistance with competitive grants

66.7%

50%

68.4%

42.8%

General Services
Information about NYS libraries and library services

55.6%

75%

73.7%

42.9%

Information for planning and advocacy

66.7%

62.5%

89.5%

35.7%

Technical Assistance with system member library concerns

55.6%

87.5%

89.5%

35.7%

These survey results confirm the usage data provided by Library Development and the Research Library in previous sections of this report about substantial usage of statewide services by New Yorks libraries. The consultant asked the deeper question about impact of the statewide services and reported results as follows (p.12):

Table 3: Impact of Statewide Services on System Member Libraries


In general, since 1997:
Reference & Research Resource Public Library Systems School Library Systems
Support through statewide services has helped the libraries in our system to accommodate the changing and shifting needs for library services.

2

25%

6

37.5%

12

63.2%

Support through statewide services has helped to support system-wide strategies which will stimulate change and/or transform services in our region.

1

12.5%

6

37.5%

11

57.9%

Statewide services have not really been useful in either of these areas.

5

62.5%

6

37.5%

2

10.5%

While usage of services was evident both from usage reports and from the survey, the survey raised questions about impact that were pursued in the focus forums and clarified as follows:

"Focus forum participants reported that from the perspective of the library system directors the impact of these Statewide Services is on the ability of their respective systems (Reference and Research Library Resource Systems, Public Library Systems or School Library Systems) to help their system member libraries to accommodate the dynamic needs for library services. And, focus forum participants noted, it is their use of Statewide Services that helps the system directors to support strategies which will stimulate change and transform services in their regions. The Statewide Services were seen as mediating the ability of the systems to support their member libraries and not as directly affecting member libraries as the survey question asked."

"This interpretation of the data is supported by the Activity Logs completed by members of the New York State Library staff during May 2001, and by the information collected by the evaluators during interviews of the New York State Library staff in April 2001. The consultants report indicates that 67% of the log entries reported interaction with a system director or a library system staff person. The effect of the Statewide Services is to support the work of the library systems in New York as they work closely with their member libraries to meet the needs of the residents of the state." (pp. 12-13)

Technology --Grants --Electronic Content

Electronic content was one of the two major technology categories supported by LSTA grants. The consultants survey elicited information about progress toward the goal of "a statewide automation of libraries in New York with seamless interoperability that utilizes both collections resident in libraries across the state and resources external to the present New York State system." Findings were that "Library Services and Technology Act funds have been used by New York State to further both the digitization of bibliographic and of unique and historical records and to encourage broad access to information. The number of New Yorkers with electronic access to information resources has grown at a strong pace over the past five years." (pp.16-17).

The survey conducted for the in-depth study showed that bibliographic records of significant portions of the collections of New Yorks libraries have been converted to electronic format as illustrated in the following table:

Average Percent of Libraries Converted (As Much of Collection As Necessary)

Reference and Research Library Resources Systems 73.5%
Public Library Systems 78.1%
Central and Co-Central Libraries 98.2%
School Library Systems 62.1%

"Discussion of the conversion of bibliographic records during focus forums indicated that these conversions are considered important and will continue until completed." (p.17)

The value of the conversion of bibliographic records to electronic format comes to fruition when New Yorkers have access to those records. The goals of the State Librarys new plan for statewide access, NOVEL, includes the sharing of those records as the name implies in the "New York Online Virtual Electronic Library." The in-depth study also surveyed the extent to which the records have been linked and are virtually accessible with the following response:

Table 4: Number and Percent Library Systems Linked to State, Regional and/or System Catalog by System Type

System Type

Linked to State System

Linked to Regional Catalog

Linked to System Catalog

Reference and Research Library Resource Systems (9)

1

11%

6

67%

1

11%

Public Library Systems (16)

1

6%

4

25%

14

88%

School Library Systems (19)

1

5%

18

95%

15

79%

"Respondents to the LSTA Library System Director Survey in June 2001 indicated that the converted records in their system are linked to regional catalogs or system catalogs. These converted records in both reference and research library resource systems and school library systems are more often linked to regional catalogs. In public library systems, the converted records are more often linked to system catalogs." (p.18)

The survey respondents indicated there is ongoing need for electronic content support. The consultant reported that, "Library systems that have completed conversion are now turning their attention to other tasks related to the electronic medium, for example, the purchase of electronic full-text journals and reference materials, the digitization of historical documents and exploring the issues surrounding e-books." (p. 20)

Technology Grants - Technology Training

The technology training grant category was created in recognition that technology training of professional staff in New Yorks libraries is essential to the delivery of high quality technology services by those libraries. Focus forum participants in the in-depth study indicated that one of the "two most important things that they had done with LSTA Grant funds had been to provide technology training to member libraries." (p.8)

Use of LSTA grants for technology training was reported as follows:

Reference and Research Library Resources Systems

100% (9)

Public Library Systems

75% (12)

School Library Systems

37% (7)

The consultant noted that, "the rate among school library system directors is smaller, thirty-seven percent. This is probably due to the fact that school library systems have only recently been allowed to apply for LSTA funds and have been using this source of support for extensive retrospective conversion of their bibliographic records." (p.23)

The survey conducted for the in-depth study elicited information about how training grants supported professional development and about the on going need for training with the following results (p.25):

Table 6: Areas of Professional Development Supported by LSTA Grant Funds
 

Reference and Research Resource Library System

Public Library System

School Library System

Central & Co-Central Library

 

Use reported in the past, and required in the future:

Past Future Past Future Past Future Past Future
Professional development in how to access information on the Internet. 89% 78% 100% 88% 47% 89% 0 64%
Professional development on how to help the public to access information on the Internet. 33% 67% 92% 88% 33% 68% 29% 93%
Professional development on how to use office based software. 67% 100% 75% 88% 20% 84% 14% 86%
Professional development on how to use electronic means to track electronic usage. 11% See below 58% See below 13% See below 14%  
Training in Internet use, digitization and new and diverse technologies. (Past report only included.) 89%   86%   14%   60%  

(Note: data recorded in past column for central and co-central libraries was eliminated because they did not receive LSTA funds for training; any reported training in that column would have occurred with other means of support. The future column remains because it shows a degree of interest in technology training).

It is clear that the systems conducted a substantial amount of technology training. This confirms the data reported from the annual reports on the grants. It is also clear that there is a need for ongoing and increased technology training. The reasons cited are:

  • "The change in library services due to the growing technological infrastructure has increased the need for system member librarians to be able to use technology applications and to help the public to use them, and
  • The high rate of turnover in library personnel, requiring basic instruction in these critical areas to be repeated regularly." (p.24)

Technology: Tracking Public Use of Electronic Resources

The in-depth study yielded insight into an area of need that has grown out of the technology activities undertaken by New Yorks libraries with various means of financial support including use of LSTA funds. When reporting on the reference activity of the State Library in a previous section of this report, the issue was very clear. The onsite reference activity of the State Library has declined and could lead to a belief that the need for professional support had changed. However, tracking of electronic reference requests that call for professional reference intervention and that sometimes take longer than onsite requests showed that overall reference activity had substantially increased. Tracking of public use of electronic resources whether electronic reference, web activity, or database use is a critical issue because it is important to capture the value to the public and because a substantial amount of professional work goes into provision of the services.

The in-depth report described the issue and reported the input from the professional library community as follows:

"In the opinion of many of the interviewees, this has become an issue for libraries because the traditional methods of measuring the magnitude of library services to their constituencies, i.e., tracking circulation of resident collections, logging of inter-library loan requests, counting of reference searches, and similar measures, no longer capture the true magnitude and extent of modern library services. Quality and magnitude of services delivered by any level of the library services system in the State have to be measured in ways which are considered fair, rigorous and representative.

"Failure to attend to the issues surrounding the measurement of the new and emerging profile of library service delivery in each of the three library system types and in the member libraries of each of these systems would result in a serious undermining of the system as a whole. This is so for two reasons. First, key informants to this evaluation regularly pointed out the local basis of support, both fiscal and ideological, for library services. Failure to report on quality and magnitude of services in ways that make sense to local constituencies and their policy structures, they reported, would have serious consequences. Second, the operation of services within a complex and intertwined system of systems, as is the case in New York, makes services difficult to report on in ways which make sense to the state constituencies and their policy structures. The first step in the development of an acceptable accountability system for use in this new and emerging context was reported to the evaluators to be the development of methods for tracking public use of electronic library resources. The Statewide Library System Directors Survey, therefore, included a question about present progress and perceived future need for technical support in this area. Results of that question are presented with the rate of response reported in the columns of Table 7 below. What can be seen in this table is that the majority of the system directors believe that this is an important area for future attention and that they need help in the development of these tracking methods." (pp. 26-27)

Table 7: Progress on Development of Tracking System of Public Use of Electronic Library Resources by System Type
Do you think it will be important to develop methods for tracking public use of electronic library resources in the future?

Yes, and we have developed them.

Yes, and we have started (and need help)

No, existing systems are fine.

Reference and research library resource Systems (8)

0

5

(63%)

3

(38%)

Public Library Systems (15)

2

(13%)

13

(87%)

0

School Library Systems (15)

1

(7%)

12

(80%)

2

(13%)

Technology (In-depth study): Telecommunications Issues

Another area of concern that grew out of the in-depth study is telecommunications support for library technology. The extent of concern among system directors is as follows:

Table 8: Resolution of Telecommunications Issues by Library System Type
Is the resolution of telecommunications development issues important to future technology development in your context?

Very Important

Somewhat Important

Not Very Important

Reference and Research Library Resource Systems (8)

4

4

1

Public Library Systems (15)

14

1

0

School Library Systems (18)

15

2

1

The study reported, "Focus forum participants noted two things here. First, that it is this area in which they need the most technical assistance, and it is this area in which expert help is unlikely to be resident at the New York State Library, and where consultant experts are extremely expensive. Second, that the details of the telecommunications issues differ according to the stage in technology development of the library system in question, and so an approach to this concern will have to be flexible."

"The increasing use of web-based applications by libraries was noted by focus forum participants as the reason why telecommunication has become an issue. Indeed, one central library director interviewed during the pre-survey development stage of this evaluation reported that the cost of connectivity in his system was immense, thus creating a real barrier to service delivery to his client population." (p. 28)

Technology: State Library Leadership

The in-depth study looked at leadership provided to the systems by the State Library, including both Library Development and the Research Library. Results are reported here under the technology category because so much of the planning and development was technology related. The findings speak to the questions of electronic access posed by the in-depth study as well as to a third study goal, "New York State Libraries, library systems, and other library organizations and the New York State Library will strengthen public policy support for libraries." The latter includes a planning and advocacy role assumed by the State Library.

Two key questions asked of the library community were the extent to which there was statewide coordination of strategic planning and how much the systems use statewide services and information for planning and advocacy. The answers follow (p. 30):

Support Given to Planning by the New York State Library Statewide Services

 

Reference and Research Library System Directors

Public Library System Directors

School Library System Directors

Coordinating strategic planning at the statewide level

33%

63%

79%

Information for planning and advocacy

67%

63%

95%

Organizing responses to public policy issues

22%

19%

21%

Providing information for advocates on library and library system related issues

33%

31%

58%

Communicating information about proposed legislation of interest to libraries and library systems

44%

56%

68%

"The New York State Library is seen as the source of policy level information for the library service delivery system in the State. In addition, the need for information regarding public policy issues and proposed legislation both to managers at the three types of library system in the State and to advocates of the library services system was indicated as growing both through survey responses and in focus forum discussions. Open-ended responses on the survey by both public library system directors and school library system directors indicated a future need for support in the areas of library advocacy, and the development of sources of financial support for library services." (p. 47)

The responses from the library community corroborate information provided by Library Development in a previous section of this report about Library Development planning and advocacy activities and about advisory services provided by the State Library. It also is consonant with the level of advisory activity evident in the sampling of State Library staff logs. Given the complexity of the state system it is not surprising that there are varying levels of dependence on the statewide leadership services, and it is evident that substantial sectors perceive the presence of the leadership provided.

Responses from the library community particularly responded positively to the leadership of the State Library in the provision of EmpireLink services and the statewide plans to evolve those services into the more comprehensive NOVEL (New York Online Virtual Electronic Library). Focus forum comments confirmed the following learned in the survey of system directors (p. 31):

System Directors Contact State Library For Help
  Reference and Research Library System Directors Public Library System Directors School Library System Directors Central Library Directors
Obtaining Cost-free Access to Full-text Electronic Databases

78%

88%

90%

21%

B. Special Services

The goals studied in-depth that most related to special services were:

  • Libraries and library systems will deliver programs that meet and anticipate constantly changing needs for library services.
  • All New Yorkers will have access to library resources and services that advance and enhance their lives as workers, citizens, family members and lifelong learners.

In the overview, the report quotes from the current New York State LSTA Five-Year Plan, "Libraries empower people so that library services must be dynamic and responsive to the changing needs of people of all ages and all abilities. A diverse state such as New York, which has both large numbers of people to be served in densely populated urban areas, as well as large geographic areas with smaller numbers of people, requires a complex network of library services to assist all citizens of the State to locate and use information and services that will help to advance their lives in many ways." (p. 33)

Focus forum participants in the in-depth study indicated that one of the "two most important things that they had done with LSTA Grant funds had been to expand services using outreach projects and new initiatives which involve member libraries and foster collaborations." The other of the two, technology training, was discussed under technology. (p. 8)

"LSTA Grant Project Directors were asked if their system had used LSTA funds to enhance or expand services for individuals. Forty-two of the sixty-one respondents (69%) indicated that their system had used LSTA funds to enhance or expand these services. The grant project directors were then asked to indicate specific program areas where LSTA grant support had been used. Respondents were asked to indicate all of the areas in which their system had used LSTA funds in services for individuals, with the result reported here as Table 9." (p. 34)

Table 9: Project Directors Reported Use of LSTA to Support Program Areas

Number using LSTA to support this service: (percent of all indicating use of support)

 

Enhancing or expanding:

25 (60%)

Services to job seekers and career changers.

16 (38%)

Services to entrepreneurs.

19 (45%)

Adult literacy services.

22 (52%)

Family literacy programs.

Public Library System directors were asked the same question on their statewide survey. Fourteen of the sixteen survey respondents, or 88%, indicated that they had used LSTA funds to enhance or expand services for individuals in these categories." (p. 34)

Table 10: Public Library System Directors Reported Use of LSTA to Support Program Areas

Number using LSTA to support this service: (percent of all indicating use of support)

Enhancing or expanding:

11 (79%)

Services to job seekers and career changers.

10 (71%)

Services to entrepreneurs.

12 (86%)

Adult literacy services.

10 (71%)

Family literacy programs.

"As can be seen from the data reported in Tables 9 and 10, the use of LSTA grant funds to support special services is widespread among those that answered the surveys sent out as part of this evaluation, as reported by both grant project directors and public library system directors."

"Choices of library development areas that will need support over the next five years differed for each type of library system. Three of the reference and research library resource system directors indicated adult literacy services, as well as services to new populations, making both of those areas the third most chosen by this group of directors. Eleven of the public library system directors indicated family literacy services as among those that will be the most important library development areas over the next five years. These directors also chose services to new populations as their third most frequently indicated area that will need support." (p. 35)

Special Services: Services to New Populations

"The seventy-six focus forum participants were clear in their reporting that their library systems and system member libraries provide extensive services to new populations. Public library system members and central libraries, especially those in large cities (New York, Buffalo, Rochester, Syracuse and Yonkers) and small cities (e.g., Utica, Binghamton, Troy, Schenectady), typically provide both adult literacy and family literacy programs. Adult Basic Educators have long considered libraries as the ideal place to meet with adult learners who will not be embarrassed if seen entering a public place that provides voluntary services to adults. This legacy has meant that libraries are increasingly perceived by other community agencies as a neutral environment in which information for job seekers, and training in work-readiness skills can be delivered without undue stress to the information seekers."

"Some of the new populations referred to by the reference and research library resource system directors and the public library system directors on the Statewide Survey were the same: new Americans and non-Native English speakers were noted in general, along with underserved areas and US Army personnel. Specific reference was also made to Hispanic and Chinese ESL populations."

In general, each library system type defined new populations somewhat differently. Reference and research library resource systems define new populations as new types of special libraries to whom they can offer services, and as existing systems of the other two types in their region to whom they can offer regional services. Public library system directors define new populations as groups of the public who have not traditionally used library services. These new groups can be delineated by geographic location (thus requiring the expansion of existing services to under-served locations) or by population characteristic (thus requiring the expansion of services to include programs and services targeted to specific client needs, such as English as a Second Language instruction, job finding and/or career changing resource development, etc.). School library system directors identified new populations as special needs clients, meaning students who are mainstreamed from Special Education programs."

"Open-ended responses on the Statewide Survey indicated that some of the library system directors believe that outreach to new populations can be enhanced through networking and collaboration with other community agencies." (pp. 35-37)

Special Services: State Library Leadership

State Library leadership was discussed earlier in the context of technology. The advocacy role of the State Library was fully discussed in the technology section and is not repeated here. This section covers how the study looked at state leadership in the specific context of "encouraging information empowerment through special services." In the RFP used to hire a consultant for the study, the LSTA Advisory Group asked that the study look at some specific outcomes that were desired through the use of grant support. These outcomes were:

  • Library collaboration with other members of the library community
  • Library coalition building outside the library community and
  • Changes to professional practice.

Collaboration

The study showed that "library systems report high levels of collaboration with other libraries both in and outside of their regions. Collaboration reported as both indicates collaboration with the indicated system type both in and outside their region. The following "table is read from the left column across. Same library system types have been shaded for ease of reading."

Table 11: Extent of Participation in Library Collaborations by System Directors
We collaborate
with ______
>

Reference and Research Resource

Public Library Systems

School Library Systems

___> (of) this region. In Out Both In Out Both In Out Both
Reference and Research Library Resource
Systems >

NA

9

0

8

4

3

9

4

0

Public Library Systems >

10

0

4

4

2

10

7

0

7

School Library Systems >

15

0

2

15

0

3

6

0

13

 

"Table 11 indicates that there is a great deal of collaboration in New York State both within library system types and across library system types. In addition, collaboration is not confined to geographic regions. All of the respondents indicated that they collaborate with same type library systems both in their own region and outside of their region, that is, nine of the nine reference and research library resource system directors, sixteen of the sixteen public Library system directors and nineteen of the nineteen school library system directors. In addition, all of the reference and research resource library system directors report working with public library systems and school library systems both in and outside of their regions. Fourteen of the sixteen (88%) of the public library system directors report collaborating with school library systems, seven in their region only and seven both in and outside of their region. Only two of the school library system directors did not indicate that they collaborate with reference and research resource library systems, and only one indicated that they do not collaborate with any public library systems. " (pp. 40-41)

Coalition Building

The in-depth study found that "libraries have become members of community coalitions in order to play an important role in the provision of seamless service delivery at the community level in New York State. Increasingly, library system directors told the evaluators, this participation in the development and provision of integrated human services in their communities has meant that libraries and library systems have modified their own role in the community. Libraries have discovered that they are effective partners in this system-level integration of services for a number of reasons, not least of which because their role is not colored by an agenda that includes traditional agency level territorial issues. Directors of all three types of library systems do agree that collaboration is a powerful tool in bringing libraries to the table at all levels of policymaking. The majority of these respondents said that they participate in these collaborations because networking is important and that it offers an opportunity to further the librarys mission." (pp. 41-42)

Level of Participation in Coalitions Outside the Library Community

Reference and Research Library System Directors

Public Library System Directors

School Library System Directors

Central Library Directors

78%

88%

90%

21%

"In addition, system directors were asked if they believed that it was important to link libraries in their system to educational, social or information services outside of the library community. Their responses also indicated the extent to which these links have already been constructed. As can be seen from the following table, a majority of both the reference and research library resource system directors and the public library system directors indicated that this is an important thing to do. A lower proportion of school library system directors answered that this is important for them to attend to. Those school library system directors that did indicate that it is important to link their system to other service delivery agency systems noted the career planning and job search area and higher education information systems as of primary concern." (p. 42)

Table 12: Linking of Library System Member Libraries to Information Services Outside of the Library Community

Is it important to link libraries in your system to educational, social or information services outside of the library community?

Total Yes

If Yes, Level of Progress To Date

All are linked

Some are linked

Only a few are linked

Reference and Research Library Resource System

5

(56%)

1

2

2

Public Library System

11

(79%)

5

5

1

School Library System

9

(47%)

2

6

1

Changes to Professional Practice

"Both in the pre-survey interviews and the focus forums carried out as part of this evaluation, library system directors and the directors of member libraries indicated that there had been a significant change in services for users over the five year period covered by the present LSTA Five Year Plan. Participants in the focus forums carried out as part of this evaluation indicated that these changes in practice are not always comfortable, and that they need to gain a comfort level with the new changes. Some work has already been done in the area of providing the support in the development of this comfort level with the new changes. This has been done using professional development targeted specifically at the issues of changes in library practice and what that means to professional librarians."

"The Library Services and Technology Act grants program in New York has been used extensively in the area of professional development during the last five years. Reports on the use of professional development to increase technology based skills among librarians in the state were provided elsewhere in this report. One question on the Statewide Survey addressed the use of professional development specifically to address issues of changes in library practice. The answers to that question established a high magnitude of need for this type of professional development (reported below as % some need) and whether directors felt that the call for this type of professional development is high, moderate or low." (pp. 44-45)

Table 13: System Directors Report of Need for Professional Development Targeting Changes in Library Practice
 

 
Reference and Research Library Resource Systems (9) Public Library Systems (16) School Library Systems (19)
Our system has the following level of need to provide professional development which targets changes in library practice. 78% some need.

High 5

Moderate 1

Low 1
100% some need.

High 9

Moderate 6

Low 1
100% some need.

High 9

Moderate 9

Low - 1

"The Statewide Survey also collected some open-ended information which indicated an increased awareness among library system directors regarding the need for the development of an infrastructure which will support the changes which are now taking place in professional practices and the role of libraries. As part of that infrastructure, public library system directors indicated a need for professional development in change management and organizational development. School library system directors listed developing the infrastructure for library change. These open-ended comments led to the inclusion of questions regarding these issues at the focus forums. Participants there noted that, while the field would like to be part of the policy level discussions and debate that they know are important " we (library systems in the State) don't have the infrastructure to add a diverse voice to policy." The infrastructure they referred to is the structure for systematically eliciting information across stakeholder groups in the state and the process for synthesizing that input. Development of that infrastructure was cited by some focus forum participants as a role for the New York State Library." (p. 45)

IV. Lessons Learned

Lessons Learned Impact of LSTA: Previous sections of this report have concentrated on quantitative and qualitative analysis of the activities and outcomes of activities supported by LSTA funding from 1997 to 2002. There is no doubt that LSTA has had a significant favorable impact on the ability of New Yorks libraries:

  • To develop technology that has improved New Yorkers electronic access to information resources. The in-depth study reported that, "All New Yorkers do not have electronic access to information resources. However, the number of New Yorkers with access has increased over the five-year period of the present LSTA Five Year Plan, as reported to the evaluators by system directors. LSTA grants have helped these system directors to increase and improve access for their member libraries and the clients of those libraries. However, the rapidly changing nature of the technology applications which are the substance of this provision means that professional development and other grant provided services will have to continue into the next five year plans period. The contribution by the Statewide Services was more difficult to establish. The evaluation findings clearly indicate that the development of NOVEL is thought to be an important and valuable contribution to the delivery of services in the state. The impacts of the Statewide Services focused on facilitation and support of change and innovation were more difficult to measure. This is due in part to the lack of a clear articulation by the New York State Library of how all of the various components of this system development fit together. The contribution of leadership in results based planning was finally identified as the most important Statewide Service to this outcome."

    The evaluators recommended that, "The New York State Library should continue to provide and if possible expand the human resource development support that has been provided through this grant program. In addition, consider the technology infrastructure issues which were raised during this evaluation (telecommunications issues, purchase of commercial databases). (p. 74)

  • To deliver programs that meet and that anticipate constantly changing needs for library services. The in-depth study reported that, "The data collected within this evaluation would indicate that the restrictions on use of LSTA funds and on the type of library which can use certain types of program funds, seriously restricts the systems ability to use LSTA funding to anticipate dynamic needs for library services. In addition, more success in this outcome area is evidenced in the Technology Grants than is evident in the Special Services grants. In most cases, the Special Services grants were reported as delivering needed services to special populations, but not as innovations per se. Innovation in the Special Services programs were more often in the mode of program delivery (using computer based instruction in an adult basic education program, for example, or helping job-seekers to search help wanted databases) rather than in the content of the programs. The findings regarding the Statewide Services were more promising. These services seem to provide necessary information to the systems in the State for planning, advocacy and merging of support. This therefore allows for the systems and libraries to meet and anticipate dynamic needs at the local and regional levels." (pp. 75-76)

    The evaluators noted the difficulty of evaluating this outcome. As the report stated, "establishing that professional development has changed librarian practicedoes not establish that the change in practice improved client services." The evaluators recommended "further data collection, using the focus forum techniqueto seek this information from system directors and member library directors." (p. 76)

  • To advocate for strengthened public policy support for libraries. In earlier sections of this report it was apparent that the State Library has assumed a leadership role in coordinating advocacy efforts to strengthen public support. The State Library has organized responses to public policy issues, provided information for advocates on library and library system related issues, and communicated information about proposed legislation of interest to libraries and library systems. Despite these activities and for a complex set of reasons the efforts have not yielded the proposed legislation. The in-depth study reported that, "The perception of the system directors and library directors who attended the focus forums is certainly that public policy support for libraries has not strengthened. The defeat of various library support options in the New York State budget over the years has led to an overall feeling among the members of the field that their work is not valued, their position in the human service delivery system is not understood, and the need to fund libraries is not seen as an important issue to the State."

    The evaluators recommended that "More work needs to be done to articulate the role of the services which libraries and librarians can deliver. The appropriate group to do this is the system directors, a role in which they expressed an interest that solidified during the focus forums." (pp.77)

  • To provide resources and services that advance and enhance the lives of New Yorkers as workers, citizens, family members, and lifelong learners. The in-depth study reported that, "All New Yorkers do not have access to library resources and services that advance and enhance their lives. However, more have access than had access five years ago, and the quality of that access has improved along with the magnitude of the access provided. Informants to this evaluation reported that they have increased the amount of access in their areas, by using EmpireLink, for example, and they have increased the quality of the services they provide, for example, by collaborating with other libraries and/or service delivery agencies in their regions. Increased literacy in the adult population, through both adult basic education and English as a second or foreign language instruction, can be measured as the removal of a barrier to employment. And, for programs that are family oriented, such as family literacy programs, the program content can include elements such as information regarding human services available to New Yorkers, family wellness and parenting support. In addition, increased literacy means increased use of libraries by adults and potentially, by their children. What is new and increasing is that, especially in urban areas, greater responsibility for the provision of the adult and family literacy programming is falling to the libraries. Interviewees and focus forum participants in this evaluation indicated that there is an increased need for training of library staff in the management of this type of programming, for example, not in how to deliver this type of programming. This change has been subtle, moving the library (typically the public library) from the site of service to the source of service, but it brings with it, increased stress on the library service delivery system in the state." (p. 78)

    "The findings of this evaluation are that many new populations are being served in New Yorks library service delivery system, with new services and innovative approaches to delivering those services. The new populations include residents new to the United States, traditionally underserved populations using the library, and professional support services being provided to new audiences. The nature of the librarys delivery of information has changed. Librarians are finding they have more time for more complex tasks as their clients can do more routine information seeking on their own, etc. As noted elsewhere in this report, the scope of library responsibility has increased in recent years, which focuses the need to address changes in scope of services as they impact the library services delivery system as a whole. (p. 79)

This evaluation study has confirmed the value of LSTA funding to New Yorkers. It has also pointed out important lessons about need for change and improvement and future directions. The following section of the report concentrates on those needs.

Lessons Learned -- Limited Grant Categories as Barriers to Participation: "Grant project directors were asked if they believe that the LSTA grant support is worth the effort which has to be expended to apply for and report on the funding. Ninety-eight percent (fifty-six of the fifty-seven completing the survey) of them agreed that the grant support is worth the effort. However, the directors identified limited grant categories as "barriers to participation" (p. 49)

"In 1997 the New York State Library in consultation with the LSTA Advisory Council made the decision to limit the number of grant categories fundable under LSTA local grants. This decision was made in order to maximize the funding for fewer categories so that the available funding could be more effectively targeted. Findings from this evaluation would indicate that early implementers have found the restrictions on the grants categories to have somewhat reduced their ability to further their established pattern of innovative programming." The following chart shows the response to this question by system directors and project directors responding to the survey. (p. 49)

Table 15: Percent System and Project Directors Reporting Limited Grant Categories a Barrier to Participation


Limited grant categories presents a problem.
System Directors

Project Directors

All Types

Reference & Research Resource Public Library Systems School Library Systems

4

44%

9

56%

9

47%

28

46%

Lessons Learned -- Best Practices and Replication: "The evaluation also addressed the management issue of identifying the best practices which are funded by the program. Ways to identify best practices from LSTA grant funded programs were addressed through the focus forums. The point was often made at the forums that there has to be a clear and comprehensive vision for the State in order for the field to be able to align their grant funded programs with the States outcomes. The alignment of any practice with these outcomes is necessary for that practice to be considered to be best. The vision is also related to the use of best practices information. "There needs to be a very clear vision, or else we need to get information out about good programs and how to replicate them."

The feeling among some of the focus forum participants was that there is not enough of a replication thrust. They expressed the belief that it would be advantageous for replication to be encouraged, and to do that "the field needs more information and research on good quality programming". Indeed, one forums group felt that it might be advantageous to have grantees submit a replication handbook as part of the product of their grant funded project. These materials should be made available electronically via a web-portal maintained by the State Library or a contracted service provider.

Participants also noted that they could use better information about the quality and results of past grant funded programs. They expressed the belief that if the State is to move forward in the innovative environment the program developers at library system and member library level have to receive clearer information regarding what innovations have been tried and tested and with what result. Indeed, forum participants at all six of the focus forums held as part of this evaluation, expressed the opinion that this basic information would allow them to make informed choices regarding the programs that they proposed under LSTA. In addition, they saw this as a means of developing further understanding of the scope and achievements of the program." (pp. 51-52)

The in-depth study recommended a strategy "whereby innovative programs funded under the grants program would be piloted in one funding cycle, replicated in a controlled design in the following funding cycle, and that the replication site would produce as a final product a handbook on the program. Finally, these handbooks and other pertinent information on innovative programs would be available in an electronic format for consideration for use by other sites. While this may not be appropriate for all types of grant funded programs, the New York State Library should consider adopting this approach where appropriate. (p. 82)

Lessons Learned -- Need for Improved Communications: "Communication within a system as complex as the library services delivery system in New York is important to the success of innovation and change at the system level. Communication in general was discussed at the focus forums because of open-ended responses on the Statewide Surveys that addressed issues of communication. In particular, focus forum participants discussed the quality of the communication of grant requirements and funded program information from the New York State Library to them, and then broadened their discussion to include the within system communication networks that do and do not exist in our State."

"As can be seen from the responses reported by Grant Project Directors on Table 16, the actual management logistics of the grant program are considered to be clear and easy to follow by a majority of the respondents. Seventy-seven percent agree that the application rules and procedures are clear, and eighty-eight percent agree that the management procedures for the grants once awarded are clearly communicated by the New York State Library." (pp. 52-53)

Table 16: Grant Project Directors Report of LSTA Grant Management Paperwork
 

Strongly Agree

Agree

Disagree

Strongly Disagree

LSTA grant application rules and procedures are clear and easy to follow.

13

23%

30

54%

13

23%

0

LSTA grant management procedures are clearly communicated by the New York State Library.

16

29%

33

59%

6

11%

1

1%

While the communication of grant application rules and grant management procedures is applauded, the evaluation pointed to possible areas for improvement including:

  • Need for communication of grant program successes and failures. It was recognized that the information is currently collected but requires summarizing and disseminating, perhaps electronically.
  • Need for more efficient means to disseminate information than a ListServ.
  • Need for a more tightly coordinated change process for the Five-Year Plan and the Grant Program Guidelines and the regular communication of that process as well as any changes in emphasis or objective.
  • "The use of existing communication mechanisms that are good as the basis of any new system of communication is strongly recommended. Informants to the evaluation expressed their wish to continue one on one communication with the New York State Library staff, for example, and expressed a wish to have the opportunity to meet with one another in person at least annually. While this is provided by some of the professional organizations to which professional librarians already belong, the key stakeholders to this evaluation noted that those meetings are already full of information and material. What was requested are meetings attended by key stakeholders where key informants would discuss issues specific to their library system type or to their region. The evaluators recommend that every effort be made to accommodate this request."
  • Need for exploration of new technologies to allow for both asynchronous and synchronous communications about critical issues. (pp. 53-56; 70, 81)

Lessons Learned Barriers to Institutionalization of Grant Funded Innovations: "The question of institutionalizing innovations supported by the LSTA Grants programs was a critical question addressed by this evaluation." (p. 56) Some of the barriers identified were:

  • Difficulty working cooperatively with agencies with only one year funding;
  • Time is needed to get a program up and running and work out problems;
  • One-year is too little time for follow-up to ensure institutionalization;
  • Difficulty getting impacts during timeframe and collecting meaningful outcome data.
  • Inability to use grant funds to pay existing staff to work on grant funded activities. Aspects of this issue include:
    • Negative impact on quality of work being done;
    • Impact on ability to institutionalize innovative programs.
    • Inability of the systems to pay for administrative costs out of the LSTA grants present a challenge to the efficiency of grant administration. (pp. .56-58)

Despite the reported barriers to institutionalization, the study showed that 96 percent of the Grant Project Directors reported that work begun with LSTA grant funding tended to be continued. Leveraging of grant funds to obtain other sources of funding was also limited: 20% school library systems; 33% public library systems; and 44% reference and research library resources systems. These two findings seem contradictory to the expressed concerns. The study sought explanations in the focus forums and reported the following:

"What focus forum participants told the evaluators was that LSTA funds are often a small part of the overall funding for one of these projects. If one considers the fact that existing staff cannot be paid out of the LSTA grant, including administrative staff, the opportunity cost of running one of these programs can be quite high. The point was made at more than one forum that the funds used to implement the programs supported by LSTA grants is from local budgets. This means that funds are leveraged prior to the implementation of the program being funded, and will continue to be expended to support the core program components as long as the library systems believe the objectives of the program are important. The commitment of the systems to the programs that they propose is evidenced by this willingness to support the programs with their own funds, as one participant put it: "carved painfully from our resources describes it all". (pp. 56-59)

It is revealing that LSTA funds are only a small part of producing innovative projects. LSTA funds cannot make things happen independent of multiple sources of funding. This fact certainly advocates for increases in LSTA funding.

Lessons Learned Need to Strengthen LSTA Program Evaluation: The evaluator who conducted the in-depth study, the evaluation facilitator who analyzed grant reports, library development data, and research library data, and the State Library staff all reported on the shortcomings of LSTA program evaluation. The following issues and recommendations were offered:

  • Current data collection is insufficient for measurement of the impact of LSTA on the library services of the State;
  • Current data focuses on activities of programs and scope of services rather than on user outcomes;
  • Project managers and system directors seek information that is not collected;
  • Some programs have large amounts of quantitative data that is never summarized or analyzed for change decisions;
  • LSTA programs require an ongoing evaluation strategy rather than a periodic evaluation effort;
  • Performance indicators and evaluation criteria are needed for all grant categories and for all statewide services funded by LSTA;
  • Evaluation of LSTA should yield impacts on users, evidence of change, and best practices.
  • Evaluation of LSTA programs needs to be communicated widely and regularly and used for decision-making.
  • Need for professional development in evaluation should be studied.
  • Periodic feedback from the field regarding the effects of LSTA funded programs and services on the library systems and their member libraries should be collected.
  • "Review of the LSTA Five Year Plan biannually in an open and interactive environment with key informant groups can enhance opportunities for innovation."
  • "Collection of follow-up data from grantees six and twelve months following the grant funded program, specifically reporting on the institutionalization of components of that grant funded program (including funding) would be helpful to the grant program managers and to possible replication sites."
  • A formal monitoring of the characteristics of and impacts resulting from collaboration should be developed and implemented.
  • Further data collection, using the focus forum technique would be effective to seek information from key informants about how professional development has changed library practice.

(Evaluation of the LSTA Library Services and Technology Act Program in New York State, p.64-76; feedback from Library Development staff, Research Library staff, and evaluation facilitator)

V. Brief Review of Evaluation Process

A. Who was involved?

Part of the evaluation was conducted by two outside consultants, part by State Library staff. Responsibilities were as follows:

Dr. Kate Toms, President of CDA Corp. and Evaluation Consultant:

  • Took lead responsibility for the design and implementation of the evaluation
  • Designed surveys or interview questions
  • Conducted focus groups and interviews
  • Provided a report on the in-depth evaluation study

Dr. Eleanor Carter, evaluation facilitator:

  • Assisted in development of RFP for an evaluation consultant
  • Participated in the design of the evaluation
  • Advised Evaluation Committee and internal team as needed
  • Worked with Research Library and Library Development staff to analyze usage statistics
  • Reviewed overall assessment by internal evaluation team
  • Coordinated final report

Marisa Gitto, graduate student intern:

  • Read all LSTA project reports from FY 1998--FY 2000 and summarized significant information as agreed upon with State Library staff
  • Made follow-up calls to secure missing data

Sara McCain, LSTA Coordinator:

  • Coordinated evaluation process, working with Evaluation Committee, internal evaluation team, evaluation consultant, evaluation facilitator, and library community

LSTA Advisory Council Evaluation Committee:

  • Advised on RFP for evaluation consultant
  • Advised on implementation of evaluation

Internal Evaluation Team

  • Advised on RFP for evaluation consultant
  • Assisted in the design of the evaluation
  • Participated in overall assessment of Five-Year Plan activities

B. How was the evaluation conducted?

The State Library in November 1999 established an Evaluation Committee from the members of the LSTA Advisory Council, several State Library staff members, and an evaluation facilitator. This group drafted a RFP that resulted in the hiring of an evaluation consultant, CDA Corp, in November 2000. The Evaluation Committee was involved at various stages in implementing the evaluation. The State Library also formed an internal team of staff members, a graduate student intern, and the evaluation facilitator to work with the outside evaluator.

The internal evaluation team had major responsibility for the overview of the current Plan's goals and objectives, noting significant accomplishments in both Grants and Statewide Services and describing what went well and what could be improved.

1. A graduate student intern analyzed all LSTA subgrants awarded during three annual grant cycles.

FY 1998--74 grants

FY 1999--94 grants

FY 2000--85 grants

2. The internal team as well as the evaluation facilitator and the evaluation consultant reviewed the data from this analysis.

3. State Library staff reviewed the Statewide Services' objectives, noting accomplishments and improvement areas. The evaluation facilitator reviewed their work integrating it with existing data and the in-depth findings.

4. The consultant, CDA Corp., concentrated on the four long-term outcomes that were selected for in-depth evaluation. She conducted surveys and focus groups and submitted a report that is cited in the in-depth evaluation of this report and included in full in Appendix C.

Of particular interest to the State Library during the in-depth evaluation was information on the following topics:

  • Ways in which services to users have been transformed as a result of the LSTA programs;
  • How users--both library staff and those libraries' users--have been affected by the LSTA program;
  • Evidence that new populations were served;
  • Demonstration that programs funded by LSTA are ongoing beyond the terms of the initial LSTA support;
  • Instances of leveraging of LSTA funds to obtain other sources of financial support and to enhance services begun with LSTA;
  • Whether LSTA funds have been used to create meaningful and enduring partnerships or collaborations;
  • Key successes and failures of LSTA funded programs;
  • How effectively results were shared with other libraries or service providers.

Methodology

Evaluation methodologies included the quantitative and qualitative analysis of:

  • Existing reports, plans, and statistics
  • State Library staff logs
  • Web tracking data
  • Surveys, focus groups, and interviews carried out by the consultant

Specific techniques for gathering information were assigned as needed. For example, State Library staff was assigned topics for narrative reports and responsibility for summarizing particular data. A program logic model developed by CDA Corp and the internal evaluation team was used as an overall guide to the evaluation process.

Funding

The State Library allocated $75,000 for the outside evaluation consultant for this project. Funds for the evaluation facilitator and the graduate student intern totaled $15,000.

Timeline
November 1999
  • Evaluation Committee began work with evaluation facilitator
February-April 2000
  • Evaluation Committee and internal committee developed RFP
May 2000
  • RFP sent out to bid
June 2000
  • Graduate student intern began analysis of reports
July 2000
  • Committee reviewed bids and made selection
August-September 2000
  • State Library staff worked with SED's Contract Office
November 2000
  • Contract approved; internal evaluation committee met with evaluator
December 2000
  • Committee and evaluator worked on program logic model
January 2001
  • Evaluation design work began
February 2001
  • Evaluation design reviewed by external Evaluation Committee
March-September 2001
  • Evaluator conducted interviews, focus groups, etc.
December 2001
  • Evaluator provided draft report to Evaluation Committee
February 2002
  • Final report from evaluator on in-depth study
Summer 2001-March 2002
  • Evaluation facilitator worked with State Library staff on final overall report
March 2002
  • LSTA Advisory Council reviewed in-depth report and draft of new Five-Year Plan using results from evaluation
April 2002
  • LSTA Advisory Council and Regents Advisory Council on Libraries reviewed overall evaluation report
  • Evaluation report submitted to IMLS



Go to Library Development home page

Last Updated: May 26, 2021