Friday December 15, 2006

Open library data as a platform for research and development

I'm not sure how big the audience is for this topic, but there's an excellent recorded discussion on, among other things, the challenges and benefits of making library bibliographic data freely available.  The conversation included three Talis employees, Ross Singer and Tim Spaulding of LibraryThing.  A motivation for this discussion was the announcement that the winner of a recent Mellon Award for innovative library software would use the prize money to purchase bibliographic data from the Library of Congress and make it freely available.

There are many issues raised by the prospect of making large amounts of bibliographic data freely available for unrestricted use.  Plenty of legal, economic, technological and social issues to go around.  What I am most interested in is what can be done with large piles of freely available bibliographic data.  How could freeing this data improve the services libraries provide to their users?  If folks (other than library geeks) could envision tangible benefits of open library data, there would be more support for these efforts.  Library conference programs of late demonstrate a healthy interest in making library catalogs better so I think there would be a receptive audience.

Perhaps what is needed is some structure to move this agenda forward.  I wonder whether the library community would be receptive to a TREC style approach to spur research and development in improving the library catalog.  In this arrangement, a major institution (LoC? OCLC? Amazon.com?) would make a large dataset of bibliographic records available in some useful format and challenge folks to solve one or two specific problems of general interest to the library community.  Perhaps some prize money could help generate interest, though press releases highlighting winners don't hurt.

Here are three catalogs-specific problem areas I can think of:

• Applications of the FRBR model in catalog discovery interfaces


• Relevance ranking models for catalog search


• Recommendation systems


• Subject clustering