Final Report of the New York State Federated Search Portal Pilot Project

September 2007

This report summarizes the results of a two-year pilot testing the application of WebFeat federated search software for the online collections of the New York State Library and the collections of the NOVELNY pilot project.

Upon the recommendation of the NOVELNY Steering Committee, the New York State Library implemented a pilot program in 2005 to test federated search software. Federated search software allows users to simultaneously search across multiple online resources including databases and web sites. The pilot program used the software developed by WebFeat to test the application of federated search to the NOVELNY collection of resources. Fourteen libraries and library systems participated in Phase 1 of the pilot, and four members of the Phase 1 group continued in Phase 2 to further test integration of their locally subscribed online resources with NOVELNY databases in a single search. The Portal Pilot Project ended on August 31, 2007.

The NOVELNY Steering Committee reviewed evaluation reports from the two phases of the Portal Pilot in August and concluded that the Portal Pilot was a successful demonstration of the application of federated search technology. Federated search will continue at the New York State Library and on the NOVELNY web site. While cost considerations may limit its widespread adoption, the findings of the pilot program may be of interest to other libraries using, or considering implementation of, federated search.

For more information and background on the report or the pilot project contact Liza Duncan, Principal Librarian, Technical Services and Systems.

Background

The implementation plan for NOVEL (New York Online Virtual Electronic Library), titled Libraries Expanding Information Access for New Yorkers in the New Century, includes five major initiatives. Initiative V states, “Develop a NOVEL User Interface (or Portal) that Integrates the Services and Resources Brought Together Under NOVEL.”

Since the plan was issued in 2001, the NOVELNY (formerly NOVEL) Steering Committee has been working toward implementation of Initiative V as well as the other initiatives. During the course of the implementation, the New York State Library has investigated federated search products that could provide a single point of access to its electronic resources, including the NOVELNY collection of resources.

In 2004, a task force consisting of State Library staff and members of the NOVELNY Steering Committee formed to find and test a product that would both meet State Library needs and help implement NOVELNY Initiative V. The task force evaluated vendors, selected the libraries that would participate in testing, and oversaw the implementation and evaluation of the Federated Search Portal Pilot Project.

In 2005, the task force selected WebFeat as the vendor for the pilot project. WebFeat users comprise more than 5,000 public, academic, government, and Global 1000 libraries, including more than half of the largest 20 U.S. public libraries, 12 state libraries or statewide agencies, and one out of every 10 Association of Research Libraries (ARL) institutions.

Phase 1

The following New York libraries and library systems were participants in phase 1 of the pilot project:

Albany Public Library
Dominican College
Dutchess County BOCES School Library System
Hobart College
Liverpool Public Library
Mid-York Library System
Monroe County Library System
Morrisville State College
New Paltz High School
Onondaga-Cortland-Madison BOCES School Library System
Orange County Community College
Wallkill Senior High School
White Plains Public Library
Yonkers School Library System

The pilot provided a single federated search (available at http://www.novelnewyork.org) for the NOVELNY databases, with user authentication by means of New York State driver license number.

In the initial phase of the pilot evaluation, the participating libraries indicated a need to do the following:

Provide an alternative to the driver license authentication scheme (which may be available at the local library level).
Allow the federated search to access local OPACs and local databases as well as NOVELNY databases.

As a result, the State Library initiated phase 2 of the pilot project in 2006.

Phase 2

Phase 2 allowed libraries to test alternative authentication schemes and the ability of the federated search to include NOVELNY databases, locally subscribed resources, and local OPACS simultaneously.

Library participation in phase 2 was based on participation in phase 1, as well as local expertise, technical infrastructure, and representation of different library types.

The following libraries participated in phase 2:

Albany Public Library, representing public libraries
Monroe County Library System, representing library systems
Orange County Community College Learning Resource Center, representing academic libraries
Wallkill Senior High School, representing school libraries

WebFeat provided the four participating pilot libraries with their own WebFeat search pages, custom branded and configured to include the NOVELNY databases and each library’s local resources, including the library catalog. Participating libraries received the Express version of the WebFeat Administrative Console, which enabled them to configure their own resources, including database menus and subject categories. Pilot libraries were able to choose one of three standard interfaces from the WebFeat Express interface module, and could change interfaces at any time during the pilot.

WebFeat hosted the individual systems (the hardware and software are owned and maintained by WebFeat at its site), maintained all translators, and managed translator updates for each system. Each participating pilot library also received WebFeat’s SMART usage tracker to track database usage at the library.

Evaluation

A survey measured how much library staff time was involved in setting up and maintaining the federated search capability, what problems were encountered, what user reaction was, and whether or not the library would continue the capability if they had to pay for it.

The survey was conducted in May 2007 after eight months of use (four months for one library). All four participating libraries responded to the survey.

Overall, the participants liked the federated search. Survey responses showed that implementation went well, although one library took a longer time because of patron authentication testing. One library spent more time than the others in preparing for the implementation, which helped during the actual implementation. The majority reported that the pilot project required approximately one percent of their time for the entire project. All agreed that the system was easy to customize, and maintenance is minimal.

The reactions of users varied. Here are some examples of the survey comments:

"Patrons have not expressed a reaction either way…. Staff have been very happy with the service."
"Students and staff embraced it."
"The service is probably underutilized."
"Reference Department librarians have said they don’t use the Federated Search system often, that their searches tend to be more focused, using specific databases…."

One of the participating libraries tracked usage by using WebFeat’s statistical package, SMART. The others did not formally track usage.

When asked if the library would continue with the federated search if it had to purchase the service itself, two libraries responded that cost would prevent them from making a purchase. The other two libraries were not sure and felt they would need to track usage to justify it.

The participating libraries offered the following advice for other libraries just starting to investigate federated search:

"It would be important to benchmark those libraries that have implemented federated searching."
"Go for it—check out all possible vendors, ask for a demo from each one, and if fiscally feasible, get federated searching!"
"Think carefully about how the service will be integrated."
"Do a lot of promotion, get staff excited about using it and find useful ways to use it. Highlight it in computer training classes along with other database searching options. From a technical standpoint, make sure you understand the basics of what is needed for authentication for WebFeat before you take on the project."

Phase 2 continued through August 2007, providing participating libraries with an opportunity to expose more librarians and patrons to federated searching. Although the implementation went well and generated some positive feedback for using a federated search tool, it is still not clear if demand and usage justify the application of the pilot program resources in this area for a large number of libraries. In addition, the business model for a federated search tool may still be prohibitive for many smaller public and school libraries.

Conclusion

Phase 2 of the pilot project allowed libraries to test a customized federated search product that simultaneously searched their locally subscribed resources and local OPACS in addition to the NOVELNY databases. The pilot libraries were also able to use a local authentication scheme (e.g., a library card number) instead of the driver license authentication used at www.novelnewyork.org. The overall implementation was successful.

The NOVELNY Steering Committee reviewed the findings of the pilot project and concluded that the principal objectives of the Federated Search Pilot were met. The novelnewyork.org site will continue to use federated search.

New York State Library staff thank the participating libraries for donating their valuable time and expertise to this project. Thanks go as well to the NOVELNY Steering Committee for their support and recommendations.