Monday, 14 September 2015

Getting hold of citation data

Where do you get citation data from? This is a problem that I've found surprisingly difficult to answer. Below are a few possible sources and the problems I've encountered. My biggest problem has been time resolution. I want to know what date the citing and cited paper were published, but the downloads I have got my hands on only give the year in almost all cases. Pretty basic, public, but inaccessible information.

Perhaps the most obvious answer, especially if you are looking at impact factor, is Thomson Reuters' Web of Science. There's a few problems with this. Firstly, you need to be privileged enough to have access, which is not a huge barrier if you are somewhere in or near academia. You can do a cited article search, but the number of articles you can look at is limited to 500, meaning that only moderately large journals of work can be analysed. There is also the date resolution problem I mentioned above.

Crossref is a database contributed to by publishers. At least in this case access is free. There is a simple GUI, but I couldn't work out how to query articles that cite. You can use the API, but an expert tells me that the results are also limited to year in time resolution. In addition, the data is not complete: not all publishers submit data to Crossref and often with months worth of delays. It is not reliable for an up-to-the-minute, accurate picture.

You could try approaching publishers directly, but how many separate messages is that, and I simply don't have the time! Also, publishers treat the bibliography as part of the article which means that if the article is paywalled, so is the bibliography so you need to pay or get persmission, unless there is some kind of data-mining policy in place. This ends up in a minefield of permissions.

Google scholar do citations as well, but as far as I know there is no systematic way to download the data, and I have heard rumours that they are restricted on how they use the data, so probably couldn't release it even if they wanted to (see comments about publishers).

Is it too much to ask to have a comphrehensive database of citations? For better or worse, this is the basis for how  most researchers are assessed, so accurate data would seem to be a priority.

Just a final thought, what does and doesn't count as a citation is a much more complex question that it might at first seem. Google scholar, Crossref and Web of Science all give different numbers for citations. This is based on the selection of journals they use to source citations and probably also the quality of data - I suspect Thomson Reuters put a lot more work into verifying data than the others mentioned above, although I'm sure can't fix all mistakes.

The solution? To be honest, I have no idea beyond encouraging publishers to take Crossref deposition seriously and allowing users to access the publication date. Crossref data goes into Ithenticate, the standard software for detecting plagiarism, so a good motivation would be to persuade publishers that they can reduce misconduct and retractions by depositing quickly. Accurate citation data would be a nice side-product and of great benefit to those whose careers depend on it.