OUseful Info

Dirty Name Data in the OU Open Repository Online

Now I'm not one to criticise the Library (?!;-), but over the weekend I've been playing with one of the Library's newest collections, the Open Repository Online, with a view to visualising who's co-authored papers in the collection with whom. And looking through the author information provided in the results listing for a search on a particular author name, I have to take issue with one or two things...

Like the considerable amount of ambiguity in both the naming convention and the stylistic presentation used for name references.

For example, the following names all appear in the author information for papers returned from an ORO name search on the author "pillinger":

• Pillinger, C. T.


• Pillinger, C.T.


• Pillinger, Colin T.


• Pillinger, Colin


• Pillinger, C.T, (or maybe you could read this as "Pillinger, C.T")

(I'm also guessing that Semjonova, L. F. and Semenova, L.F. are the same person (apologies to both parties if not)?)

If I was to construct a table of author names, using direct/literal pattern matching of text strings to match individuals across different article references, all of the above Pillinger variants would appear as different authors, because the text strings are different and simple string matching is dumb.

(This is just one of the issues that was mentioned at the CRiG barcamp a month or so ago (CRIG DRY (Don't Repeat Yourself) Metadata Barcamp), in particular by the MIMAS Names project, which "is going to scope the requirements of UK institutional and subject repositories for a service that will reliably and uniquely identify individuals and institutions" in order to "provide important information about the future usefulness of a name authority service for institutional and subject-based repositories, and other applications beyond the repository sector." One of that things that was discussed then related to reconciling repository name references with staff records, but for data end users, if different presentation modes (as above) are used to reference the same individual, then simple pattern matching algorithms used in mining the data are seen to be brittle, unless the data user seeks to normalise the data in some way. Which may be advisable..? That is, always assume the data is messy/noisy...).

Where the information is the source of it's own metadata, or where appropriate web page markup - such as microformats* and cool URIs - provide implicit metadata that the adventuous can build hacks around, it's quite handy if the literal textual representation of the data is coherent/self-consistent; and in the case of ORO, it doesn't appear to be...

*though bear in mind this recent second look at microformats from an accessibility point of view from the BBC: Removing Microformats from bbc.co.uk/programmes!

So this is where I start to wonder where the Library fits in to the workflow of online university research repositories if they are to be useful to data users? For example, what do we conclude if the Library can't guarantee the quality of data in their own, most recent, collection (which was born de novo only a couple of years ago)? It seems to me that "reference gardening" (which must one the lousiest jobs imaginable!), is one of the areas where they could really add value, although ideally the information would be collected/submitted in a way that removed ambiguity and ideally guaranteed conformity with some standard, authoritative naming scheme.... But that will never happen...

So whilst I appreciate that no matter how many checks and balances you put in place, there is always going to be mess. And maybe the Library can help by finding ways to help users cope with the inevitable mess... Entropy always wins out...

Ideally, of course, the data would be clean/correct, and if it wasn't that ideally any errors, once found, would be corrected. But maybe there's also scope for the Library to accept that the data is a mess, and help data patrons find heuristics for cleaning it and removing some of the noise. For example, as well as helping users write advanced (and robust) search queries (like how to use advanced search operators and set up saved search alerts), I wonder whether they should also maintain a set of 'reference functions' that users of (messy) "data as metadata" can apply as heuristics to try and tidy up textual data.

So for example, the fiollow javascript function will (I think? Err...) replace a spelled out forename in the above name referencing style with an initial:

function tidyName2(s){return s.replace(/(,\s*[A-Z])[a-z]*/g, "$1.");}

e.g. Pillinger, Colin T. will become Pillinger, C. T.

And this one will remove the punctuation:

function tidyName3(s){return s.replace(/[\s ,\.]*/g,"");}

e.g. Pillinger, C. T. will become PillingerCT

By applying several of these heuristics, we can produce a name format that removes some of the typographical ambiguity, though losing an initial may be going too far? e.g. going from PillingerCT to PillingerC may be the wrong thing to do, even though it would set up an identity between these two original forms: Pillinger, Colin and Pillinger, C. T.

That said, how many library staff in general even know what a regular expression is, let alone how to construct one to help a 'search a replace' patron tidy a data set in a text editor or even in Word (Add power to Word searches with regular expressions)?

Tags: library, oro, CRIG,

Posted by ajh59 at July 6, 2008 10:33 PM