David Cloutman pointed to Craiglist’s job ads as an indicator of programming language popularity. Here’s the hit counts for “web design jobs” and “internet engineering jobs” in the Bay Area:

Cloutman has a few ideas for what the numbers mean, but I’m just entertained by the data. (Note: he corrected his original numbers.)

A chat with Ryan Eby, also an Edward Tufte fan, elicited this line about another reason we continue to struggle with the design of our catalogs:

data isn’t usable by itself
if it was then the OPAC would just be marc displays

And yesterday I was speaking with Corey Seeman about how to measure and use “popularity” information about catalog items. It got me thinking about Flickr’s interestingness metric, which seems to combine the number of times a photo has been “favorited,” viewed, and commented. In a related fashion, I’ve been looking at ways to track the terms people use to find catalog items and use those to help improve search results. A basic form of this is in the OPAC prototype I demonstrated yesterday.

And all of this has me looking forward to Aaron Krowne’s Quality Metrics presentation at code4lib.

library, libraries, opac 2.0, library catalog, popularity, search ranking, data visualization, ryan eby, corey seeman, metrics, search rank, opac, library 2.0

Way back in April 1997, Jakob Nielsen tried to educate us on Zipf Distributions and the power law, and their relationship to the web. This is where discussions of the Chris Anderson’s Long Tail start, but the emphasis is on the whole picture, not just the many economic opportunities at the end of the tail.

Here’s how it works with hits to websites:

• a few sites become popular and form the “big head” at the left
• a few more sites form the slope
• a huge number of websites score very low and form the “long tail”

Nielsen adds these examples:

• a language has a few words (“the”, “and”, etc.) that are used extremely often, and a library has a few books that everybody wants to borrow (current bestsellers)
• a language has quite a lot of words (“dog”, “house”, etc.) that are used relatively much, and a library has a good number of books that many people want to borrow (crime novels and such)
• a language has an abundance of words (“Zipf”, “double-logarithmic”, etc.) that are almost never used, and a library has piles and piles of books that are only checked out every few years (reference manuals for Apple II word processors, etc.)

But the point here is about Google (or Yahoo, etc.) search results ranking, which puts enormous value in the number of incoming links to a page. It turns out that these links also follow a power-law distribution and it not uncommon to find complaints that Google’s Page Rank recognizes popularity over other factors.

So it’s worth wondering: is popularity bad? Are popularity and quality mutually exclusive? Do search rankings represent some sort of global group-think?

Now put this in an academic library context and consider a student Googling for background for a research paper (think University freshmen the night before it’s due). Is it possible that linking patterns work like Wikipedia and tend to favor quality, or do they simply represent lowest common denominator popularity. Do search results reflect the sum of our altruistic linking intentions or our base crudity?

More about search ranking and libraries:

• Findability, The Google Economy, and Libraries
• The Google Economy Vs. Libraries
• Changing Modes Of Communication
• The Google Economy Will Beat You With A Stick

tags: academia, academic library, google, google economy, googling, group think, jakob nielsen, libraries, library, lowest common denominator, networked information, popularity, quality, research, search engines, search rankings, search result rankings, search results, wikipedia