Archive for the ‘works’ Category

Thursday, March 15th, 2007

thingISBN data in one file

thingISBN is a simple API for discovering related editions. Give it an ISBN and it returns a list of other ISBNs—different formats, translations, etc. We offer the API free for non-commercial use. Today we’re releasing thingISBN in one giant feed, under the same conditions.*

thingISBN is based on LibraryThing’s first-of-its-kind “work” system, by which regular people—LibraryThing members, mostly—combine and separate editions. Members run over 2,000 work-combination actions per day. Although some do it for pure altruism, combining editions helps LibraryThing users by improving the quality of their connections.

LibraryThing’s results compare very favorably with its competition, OCLC’s xISBN service (also free for non-commercial use). xISBN’s coverage is better, but where LibraryThing is built on the collective judgment of humans, xISBN is just a computer algorithm. As the fella says, xISBN is “based on a world which is built on rules and because of that, [it] will never be as strong or as fast as [thingISBN] can be.”**

APIs, while nifty, can be a pain. Both thingISBN and xISBN have a 1,000-per-day limit. So, starting today, thingISBN is also available in feed format—one giant XML file with all the data from over two million unique ISBNs.

Here’s a sample file with just 1000 ISBNs:

As you can see, the format is not ISBN-to-ISBNs. This would involve too much repetition—the full XML file is already 96MB! Instead, it goes work by work, listing the ISBNs inside them:

<work workcode="183">
<isbn uncertain="true">999107371X</isbn>

This format should go into a database well, e.g.,

CREATE TABLE isbn_to_work (
itw_workcode mediumint(8) unsigned NOT NULL,
itw_isbn char(13) NOT NULL,
itw_uncertain tinyint(4) NOT NULL default '0',
PRIMARY KEY (itw_workcode,itw_isbn)

As you can see, some ISBNs are listed as “uncertain.” This happens when an ISBN crosses works. In a perfect world, these works would be combined, but LibraryThing doesn’t do it automatically. There are a couple ways that can go wrong. For example “great books” sets often sport a single ISBN across volumes. It wouldn’t do to combine “Pride and Prejudice” with “Moby Dick” just because their publisher wouldn’t pony up for two ISBNs.

So, you can use the “uncertains” if you are willing to accept more errors. Otherwise, ignore them.

The feed itself is in and is called “thingISBN.xml.gz”. It is 16MB compressed.

We’d love to hear what people are doing with the data.

*Commercial use requires our permission. See
**Okay, the comparison in inexact, but OCLC does have a “Matrix” feel to it.

Labels: apis, frbr, oclc, thingisbn, works, xisbn

Wednesday, June 14th, 2006

Introducing thingISBN

UPDATE: thingISBN is now also availabe in feed format.

Many of you are familiar with OCLC’s xISBN service. Give it an ISBN and it returns a list of “associated” ISBNs from WorldCat. So—xISBN’s canonical example goes—give it an ISBN for one edition of Dune, and it will return a list of ISBNs of other editions, in XML format. This is red meat for mashups. (Speaking of which, did you know about Talis’ Mashing up the Library competition?)

Today I’m releasing “thingISBN,” LibraryThing’s “answer” to xISBN. Under the hood, xISBN is a test of FRBR, a highly-developed, well thought-out way for librarians to model bibliographic relationships. By contrast, thingISBN is based on LibraryThing’s “everyone a librarian” idea of bibliographic modeling. Users “combine” works as they see fit. If they make a mistake, other users can “separate” them. It’s a less nuanced and more chaotic way of doing things, but can yield some useful results.

To use thingISBN, point your browser at a URL like this, replacing the ISBN as appropriate:

To compare xISBN and thingISBN add &compare=1

thingISBN vs. xISBN.
UPDATE: OCLC has disallowed comparison.
I’ve done some preliminary comparisons between the two services. The results are pretty interesting. For starters, OCLC has much broader ISBN coverage. The dataset is orders larger, and “regular people” just don’t own certain books. Where the data sets overlap, however, LibraryThing can contribute a lot, particularly when it comes to paperbacks and non-US editions.


  • 031228884 (Elizabeth Cook, Achilles). Recently-published novel. OCLC and LibraryThing know about two ISBNs. LibraryThing adds two others, a UK hardback and a UK paperback.
  • 0553212583 (Wuthering Heights). OCLC and LibraryThing share 60 editions. OCLC alone knows 266. LibraryThing alone knows 32.
  • 0520071654 (Peter Green, Alexander of Macedon…). OCLC and LibraryThing both know this hardcover ISBN. LibraryThing knows the paperback, but OCLC includes the 1974 first-edition.*
  • 0310241448 (Lee Strobel, The Case for a Creator). OCLC and LibraryThing know of one hardcover edition. OCLC knows of no other editions. LibraryThing knows of seven others. Wow.**
  • 0393049841 (Jason Epstein, The Book Business). OCLC and LibraryThing share two ISBNs. OCLC knows one by itself. LibraryThing also knows one by itself, but it’s to Simple Pineapple Crochet. Yes, you read that right. I’m not sure where the error is from, but it’s either a pitfall of the “everyone is a librarian” system, or of LibraryThing’s occasionally ratty data.

Mashups? I brought out thingISBN in part to provide more grist for Talis’ Mashing up the Library competition. I was careful to make thingISBN’s output follow the conventions of xISBN, so that existing xISBN code could be reused. I’m looking forward to see if anyone does anything with it. (One obvious application would be as an addition to LibX, an open-source Firefox extension that leverages xISBN to help you find things in your library. Here’s an excellent screen cast of it at work.)

As usual, comments, criticisms, bug reports and feature requests are asked for and gratefully received.

The fine print. By using thingISBN you agree to the following terms and conditions:

  • thingISBN is available for non-commercial use only.
  • You cannot hit thingISBN more than once per second.
  • If you’re going to hit thingISBN more than 1,000 times/day, you must notify LibraryThing (we’d love to hear what you’re doing). This is the current policy. If thingISBN turns out to be a success I’ll optimize the code more, put it on my second server and allow it to be hit as hard as people want to hit it.
  • ThingISBN is provided “as is,” without any promises or guarantees. LibraryThing is not responsible for any errors in the data, damages resulting from its use, your teenager’s attitude or the state of the world generally.
  • We reserve the right to change these terms and generally make things up as we go.

*Stratch that. LibraryThing knows it now too. A user had it, but it wasn’t combined; I went ahead and combined it. Actually, Green changed a lot between editions, but they still qualify as one “work.” (This edition, with another ISBN, may also be the same work, but I’m not sure, so I left it.)
**I started look around to see if this disparity was true in general of religious books. I think it isn’t, or at least the effect isn’t as striking.

Labels: apis, frbr, thingisbn, works, xisbn