The Horseless Library

I remember reading about this probably seven or eight years ago: holographic storage. It seemed too Star Trek to be real. Now, apparently, it's about to go on the market, and libraries might be one of the first customers. See this Slashdot article for a little more.

OK, time to try to breathe some new life into this blog

Yesterday I read about LibraryThing's tagged data being included in the Danbury, CT Public Library's catalog. First, bravo to the Danbury Library for being gutsy enough to go first and experiment with this.

From what I've read of the commentary on NGC4Lib and elsewhere, the results aren't perfect, but they are pretty good. And hey, the results weren't perfect before from just the LC headings, so I believe that mostly this will help. I sometimes wonder, when people point out counterexamples for how searching fails in some particular case, did they actually expect it to work well always?

The emails tend to point out some weirdness in recommendations based on LT's social data, but that's exactly the crux of social data. Odd connections and preferences emerge, especially if the title has a small set of tag data - whether they are in fact helpful is harder to tell because helpfulness in the this context is bound to be so personal.

Years ago I worked part time at a public library as an assistant and one of the most challenging questions I was regularly asked was, "can you recommend a good book?" I could tell you several, but the odds are unless you read and are exactly interested in very similar things as me, you may not like my recommendations. Or maybe you will, who knows? Being able to do that well was an art and it took time to really begin to know authors, genres, reading habits, etc and not focus on my own personal tastes. It wasn't easy and all this was before any social networking tags let alone Amazon reviews.

I looked at the Danbury catalog yesterday and put in the title I'm currently reading, Our Mutual Friend, by Charles Dickens. I guess LT's data worked well enough; several other Victorian novels recommended; though only one by Dickens weirdly enough. And the tags themselves made sense. But Dickens has a lot of tags - he's one of the top 20 LT authors by copies and even though OMF is not one of the most famous of Dickens' works, LT still has data on 687 copies (I salute all of those people for taking the time to tag this good book). When I actually clicked on one of the tags and went into the tag browser, I got a really nice reading list of 19th century fiction, all of which were at that library. Same thing for the tag, "Dickens." And for "London." Nice lists, not random. It was easy to use as a browsing tool and something I think I would enjoy as a patron.

A lot of the power of LT and tags comes when there are enough of them for any given title or author to statistically overwhelm the odd tags and entries that are too idiosyncratic to be of much use to anyone except the tagger herself. I'm not sure how helpful this would be for a really obscure title that LT has little data on - so perhaps I'm wondering just how good tagging is at reasonably producing Long Tail recommendations. Tagging is something that, like Amazon reviews for a given title, while perhaps possessing a Long Tail distribution, I believe only really displays its power in the much smaller set (of titles, of reviews that are tagged as helpful) that accounts for most of the data.

My conclusion after that last run-on Dickensian sentence is that adding LT's huge tag dataset could be a very interesting addition for most public libraries, especially given the price which is supposed to be very low. What's to lose?

Just now I had the extremely weird experience of looking something up with Google and getting as the very first result, ta-da! my own blog. It was a bit like dialing a random telephone number and getting my own voice mail.

Here's what happened. Someone wrote me an e-mail about a Petrarchan sonnet and used the word "octet" to refer to the first eight lines. I was pretty sure that this was inaccurate, and that the correct term is "octave," which people mix up with the term for the last six lines, "sestet."

But I wasn't quite sure, so I figured I'd check. I thought I remembered that Karen Ciccone told me that Google now allows truncation with asterisks, so I entered this search string:

petrarchan sonnet sestet oct*

I've now remembered that what Karen actually said was that you can use asterisks within quotation marks to search Google, so that, for instance, you can search "the * is too much with us" and find Wordsworth's famous sonnet. But you know how it is when you're Googling, especially if you're a fast typist -- it's easier to Google than it is to think.

So Google didn't recognize my asterisk as a truncation, and simply searched for "oct." Which, of course, is a well-known abbreviation for "October," so you can see why a blog that uses abbreviated months in its datestamps would rank high. And that helps explain why the first result I got for that search was the blog I created last semester for a graduate course in Victorian Poetry. I also wonder whether Google KNOWS IT'S ME (I was signed in at the time) and is giving me a top result from a site it has reason to believe I'd be interested in.

But it's still weird to me, just how findable the course blogs that I created last semester are, and how high their hit counts continue to be. My course blogs for ENG 560 and ENG 669 continue to make the "hot blogs" list on the front page of WolfBlogs, getting anywhere from 5 to 30 hits per day, usually. The Victorian Poetry blog, in particular, continues to keep its profile high even though no one's visiting the blog anymore. The course is over, so no one's posting or commenting, and no other site has linked to the blog.

So here's my question: Why are those basically defunct blogs still so "hot"? I've looked at the referrer logs, and while they do show lots of hits from Google and other search engines, they also list lots of "direct" (which can mean "unidentified") hits. Sometimes the "direct" hits are double the hits from search engines. Any explanations?

I did write once to Scott Warren, "I'd bet that the reason my course blogs continue to rank high in WolfBlogs is that both the assignments and especially the comments contain a whole lot of really specific, concrete, familiar terms, such as names of authors, poems, and books. Those kinds of terms are heavily searched, as Jakob Nielsen points out." If that's true, it would confirm the real importance of particular kinds of web writing in good web ranking. Blogs do, of course, rank high in Google results generally, but that doesn't explain why my old, unvisited blogs are often getting more hits than (say) Horseless Library.

Wouldn't it be cool if we could make a library website that appeared in the top results whenever information-illiterate students Googled their research topics? Imagine what would happen if all those dynamically-generated catalog pages were crawled.

By the way: That Google search didn't really do it for me, so I turned around in my chair, reached up on the shelf, pulled down my copy of The New Princeton Encyclopedia of Poetry and Poetics, and discovered that I was right. The correct term is "octave."

Peter Brantley shares some provocative thoughts on the future of libraries.

He quotes a friend: "discovery has moved to the network layer and libraries should stop allocating their time and money trying to build better end-user UI, and concentrate instead on delivery".

He goes on say as "discovery services move to the network there is less reason why libraries should maintain duplicative local data caches."

Please read the full post to put these quotes in context.

What I find interesting is that the library technology community now seems precisely focused on "trying to build better end-user UI" and trying to develop "local data caches" in response to the limitations of licensed access. I don't hear much about delivery and fulfillment services at digital library or library technology conferences. Thoughts?

Take a minute to read this blog post from a landscape architecture researcher about how google books is changing academic history.

Some choice quotes in there such as "[in six months Google] Book Search has accomplished enough to transform the academic profession." Also: "Research in my world is very often a personal matter of haggling for more time with the particular librarian in question. They're used to us, and I figure they need a good struggle to keep them alert. But thanks to Google Book Search, these days of scavenger-hunt and tug-of-war are drawing to an end."

Whoa.

Interesting that I found this post through a search industry analyst blog and not through the usual library channels.

Also worth checking out is Content Statistics from O'Reilly books.

An interesting article appeared in the NYT business section on Sunday, http://www.nytimes.com/2007/02/25/business/yourmoney/25slip.html?_r=1&oref=slogin
about a company called Blinkx. Blinkx is trying to make digital video searchable and the article discusses their methodology and why it appears to be more successful than Google Video. GV searches metadata associated with video, whereas Blinkx is trying to automatically transcribe speech in videos (an extremely nontrivial problem) and then search that speech rendered into text.

I believe this merits digital library attention because when we discuss institutional repositories, there is always a focus on papers or data, but video doesn't seem as prominent. Yet video may be an important part of what IRs or future university archives end up holding. For instance, last Friday night Blanton Godfrey, the Dean of the College of Textiles, appeared on Watch North Carolina People with Bill Friday. His conversation was wide ranging. I'm trying to add a DVD/video of his appearance to the Textiles Library media collection. But if someone wanted to know if he spoke about a particular subject in his roughly half hour interview, there's no easy way to find out, without searching a transcript or watching the entire show.

This type of content may become more common in the future, especially for prominent individuals. A few weeks ago a number of us had an ongoing discussion about a concept called 'lifelogging' where a person tries to keep track of their entire life digitally (or entire digital life; the two are as yet distinguishable, the second being easier and more common). If video becomes something we collect in earnest, both for repositories and for regular collections, then being able to search that content in a meaningful way (i.e. not GV) becomes important.

It is, however, worth noting the limitations of what Blinkx is trying to do. The last paragraph in the article emphasizes that Blinkx is searching the sounds (or rather transcribed text) of a video and cannot do anything about the images in a video. Because so many videos (unlike the example I mentioned above) may not have much speech to go by, there is still no good way to search them (Blinkx for all intents and purposes is just creating much more textual metadata for a video). Imagine, for instance, trying to search Koyaanisqatsi to find out if there are depictions of certain places/jobs/landmarks, natural events, etc.

Try Blinkx out at http://blinkx.com/

Tim Spaulding has written a very interesting summary of an experimental comparison he performed examining some parameters of tagging at LibraryThing and Amazon. It can - and should - be read by librarians at http://www.librarything.com/thingology/2007/02/when-tags-works-and-when-they-dont.php

Tagging has been endlessly promoted for 2.0 catalogs, but as Tim concludes in his report, getting anyone to tag stuff that isn't theirs seems to be a real problem (heck, we even have to pay people (catalogers) to do it :) ). And avoiding spurious political or opinion tagging would be quite a challenge in any academic environment. Technology is easy; human behavior much less so.

An interesting follow-up to Scott's last two questions ("what are the ramifications for us as librarians in creating learning
spaces, both virtual and real, that will enable focused attention for
learning, not merely disperse it further; and 2) how does one gain
attention for collections in an increasingly (and wonder of wonders
this is not a contradiction) diasporic and dense information space?").

I read on a 'gaming in libraries' listserve this afternoon that McMaster University is now hiring for an "Immersive Learning Environments Librarian" who will, among other things, "steward the creative development, implementation, and evaluation of immersive learning environments for the campus community." One of the goals of this position is to increase awareness of the library's learning resources. That would be one approach to gaining attention for collections. Its not clear to me whether 'Reference and Instruction Librarians' are some of the other learning resources this new position will try to introduce to faculty and students, though.

Scott, I wonder if immersion in a virtual world makes all these streams of information seem somehow more manageable?

Below I've copied a post by Gary Frost in response to an IF: Book blog entry termed, Future of the Filter. The metaphor Frost employs in his second paragraph is, to me, wonderful and I think he raises a very good point about the increasing pressure to filter relentless streams of new content (or what filters the filters?). There is only so much to which anyone can pay mind to before a deficit of meaningful and sustained attention, much less thoughtful response, occurs. These discussions are part of a larger school of thought termed the attention economy. Other articles that I've recently perused on the subject are a First Monday article and a cogent interview with Richard Lanham, author of The Economics of Attention: Style and Substance in the Age of Information (thanks to Keith Morgan for that one).

When I was growing up there was only one broadcast TV channel in
Chicago. I believe it was only on three hours each evening. After the
broadcast we would sit in the dark watching the test pattern. Now there
are many more channels but the capacity of our visual attention to a
single screen has not multiplied as well.

One way to visualize the multiplicity of content now presented is to
imagine a circular kaleidoscopic view with content continuously
advancing inward from the perimeter. In this metaphor content
dissipates as it migrates to the center until only a single channel is
apparent at the very center. This metaphor represents the experience
and the attention of the bionic reader. Given such visualization it is
curious that the choices are not limitless but only two; either persist
focus on the resolved channel or supplant it with another.

I'm not sure how big the audience is for this topic, but there's an excellent recorded discussion on, among other things, the challenges and benefits of making library bibliographic data freely available. The conversation included three Talis employees, Ross Singer and Tim Spaulding of LibraryThing. A motivation for this discussion was the announcement that the winner of a recent Mellon Award for innovative library software would use the prize money to purchase bibliographic data from the Library of Congress and make it freely available.

There are many issues raised by the prospect of making large amounts of bibliographic data freely available for unrestricted use. Plenty of legal, economic, technological and social issues to go around. What I am most interested in is what can be done with large piles of freely available bibliographic data. How could freeing this data improve the services libraries provide to their users? If folks (other than library geeks) could envision tangible benefits of open library data, there would be more support for these efforts. Library conference programs of late demonstrate a healthy interest in making library catalogs better so I think there would be a receptive audience.

Perhaps what is needed is some structure to move this agenda forward. I wonder whether the library community would be receptive to a TREC style approach to spur research and development in improving the library catalog. In this arrangement, a major institution (LoC? OCLC? Amazon.com?) would make a large dataset of bibliographic records available in some useful format and challenge folks to solve one or two specific problems of general interest to the library community. Perhaps some prize money could help generate interest, though press releases highlighting winners don't hurt.

Here are three catalogs-specific problem areas I can think of:

Here's a year-end roundup of the latest trends in search. Is there anything here that could be directly applied to improving digital library search?

Over at LibraryThing, they announced that they had passed 8 million books today. Now I like and admire LibraryThing and the people behind it quite a lot, but I found the descriptions of those 8 million books a bit disingenious. LibraryThing says, "LibraryThing has between eight and fifty times as many books as the World's Biggest Bookstore, in Toronto, Ontario, if laid end-to-end, LibraryThing's books would stretch 1,075 miles, LibraryThing hold 58,580 books by J. K. Rowling, LibraryThing has 80 times as many books as you have hairs on your head," etc.

The problem is, LibraryThing itself actually does not have a single book. The place where they are correct is when they say, "If LibraryThing were a library, it would be the 14th larget library in the United States." That conditional is important. I may be splitting hairs, but this language question brings up an interesting question - just what is a library? If you have descriptions of books, but no actual books, are you a library? But what if your constituent members have personal libraries? Is the aggregate description of the collected holdings of physically extant libraries, personal or institutional, a library itself?

I think not. Nor are you a bookstore, since in neither case can the institution known as LibraryThing actually provide a book to anyone in and of itself . And that ability to provide resources is the key. Require payment for
providing the end product, you're a store; don't require payment, but seek subsidies for
providing resources to some defined group of people, you are a library. Don't require payment from anybody, seek subsidies or define users, and you are some sort of barter system likely based on personal, rather than institutional, collections.

LibraryThing doesn't
fall nicely into any of those categories, though to be fair it nicely enables the buying of books from several vendors (now including 2 local stores), the location of local libraries holding copies via WorldCat, and looking at popular swap sites like BookMooch. It plays some sort of very interesting middle role that greases the discovery and serendipitous browsing of titles, especially fiction, before any of those other transactions begin and its ability to play that role well grows more potent with every new entry.

What I think I'm driving at, however, is that having book data itself, while really really interesting and very potentially useful for stores, libraries, and swaps, doesn't itself constitute a library. Libraries (book libraries anyway) and book stores, by my definitions, are not just about the collections of physical books they have, though that is the first principle. They also have an economic dimension that revolves around the transactions involved in both the procurement and providing of book resources. And I don't think you can get away from that; not easily anyway.

This isn't to suggest that because LibraryThing is not a library or a store there is a problem. Far from it. Tim Spaulding et al. have done something really unusual in creating something novel. LibraryThing seems to be to be exactly what its name suggests, something for which we don't quite have a word, or at least a good one. If I were to describe it, I would call it at its smallest a personalized version of OCLC (which is based on institutional holdings), but I believe that doesn't do a good job of conveying it to nonlibrarians nor does it fully encompass the scope of what is happening. Perhaps in Geman we could string together a series of words to make a new one that perfectly described LibraryThing, but I'm at a loss in English. Anyone have a good idea for the class of objects that LibraryThing (and OCLC I believe) belongs to? Other than the best resource around to study personalized book reading (not book buying, that's Amazon) - which makes me wonder just when will the first thesis based on LibraryThing data arrive? A one word name. Neologisms welcome.

if:book has an interesting post on inventive ways to present public-domain texts. http://www.futureofthebook.org/blog/archives/2006/12/how_would_you_design_the_iraq.html

One example is Vivisimo, a search engine company, who did the following:

"The search engine crawled the PDF file of the final report issued by
the Iraq Study Group and indexed it by paragraph rather than as a
single document. By breaking the 142 page file into paragraphs, readers
of the report can now search for specific aspects that are of interest
to them instead of having to read through the entire document or
perform a tedious keyword search within the document using the Acrobat
application. When a search query is entered, the search engine returns
the relevant paragraphs in the search results. Additionally, the Velocity Clustering Engine
is used to cluster the search results into related topic areas. The
clusters allow the reader to easily browse related information and
uncover relationships between topics within the report. This demo was
placed online by Vivísimo within minutes of the publication of the
final report, showing the speed with which the Vivísimo Velocity Search Platform can be deployed."

I don't intuitively see how they've selected the topics that "cluster" under my search terms, but it's an interesting set of views of this text.

I wonder what we could learn from this. Is it (clustered/faceted views of search results) something we could do with our own digital content and Endeca? I'd like to consider useful ways to offer access to content we create, and would be glad to hear from those with a grasp of the technical challenges.

The folks at programmableweb are maintaining a list of programming contests. Under previous contests they list the OCLC Research Software Contest and the Talis: Mashing Up the Library Competition. I would love to see more library related contests on the list.

The source for the table below is a report from the UK-based Research Information Network called Researchers and discovery services: behaviors, perceptions and needs. They collected their data by phone interviewing 400 researchers and 50 librarians in the UK. The executive summary is worth a review, though I want to call attention to the section on the comparison between researcher survey results and librarian survey results.

One of the areas of high divergence that caught my eye is the librarian perception that researchers only undertake simple searches, whereas a significant percentage of researchers self-report as undertaking a variety of search strategies (simple to sophisticated) depending on the task.