Getting hold of citation data

Where do you get citation data from? This is a problem that I've found surprisingly difficult to answer. Below are a few possible sources and the problems I've encountered. My biggest problem has been time resolution. I want to know what date the citing and cited paper were published, but the downloads I have got my hands on only give the year in almost all cases. Pretty basic, public, but inaccessible information.

Perhaps the most obvious answer, especially if you are looking at impact factor, is Thomson Reuters' Web of Science. There's a few problems with this. Firstly, you need to be privileged enough to have access, which is not a huge barrier if you are somewhere in or near academia. You can do a cited article search, but the number of articles you can look at is limited to 500, meaning that only moderately large journals of work can be analysed. There is also the date resolution problem I mentioned above.

Crossref is a database contributed to by publishers. At least in this case access is free. There is a simple GUI, but I couldn't work out how to query articles that cite. You can use the API, but an expert tells me that the results are also limited to year in time resolution. In addition, the data is not complete: not all publishers submit data to Crossref and often with months worth of delays. It is not reliable for an up-to-the-minute, accurate picture.

You could try approaching publishers directly, but how many separate messages is that, and I simply don't have the time! Also, publishers treat the bibliography as part of the article which means that if the article is paywalled, so is the bibliography so you need to pay or get persmission, unless there is some kind of data-mining policy in place. This ends up in a minefield of permissions.

Google scholar do citations as well, but as far as I know there is no systematic way to download the data, and I have heard rumours that they are restricted on how they use the data, so probably couldn't release it even if they wanted to (see comments about publishers).

Is it too much to ask to have a comphrehensive database of citations? For better or worse, this is the basis for how  most researchers are assessed, so accurate data would seem to be a priority.

Just a final thought, what does and doesn't count as a citation is a much more complex question that it might at first seem. Google scholar, Crossref and Web of Science all give different numbers for citations. This is based on the selection of journals they use to source citations and probably also the quality of data - I suspect Thomson Reuters put a lot more work into verifying data than the others mentioned above, although I'm sure can't fix all mistakes.

The solution? To be honest, I have no idea beyond encouraging publishers to take Crossref deposition seriously and allowing users to access the publication date. Crossref data goes into Ithenticate, the standard software for detecting plagiarism, so a good motivation would be to persuade publishers that they can reduce misconduct and retractions by depositing quickly. Accurate citation data would be a nice side-product and of great benefit to those whose careers depend on it.

Maybrain update

It's time for an update on the brain  simulation project, Maybrain. The aim is to combine a few Python modules to be able to map and manipulate the connectome of a brain (i.e. neuronal nodes and their connections) and show in conjunction with isosurfaces of the skull and other data.

There's been a few important changes over the past couple of weeks. We implemented algorithms for degeneration, either by node spatial proximity or along edges connecting nodes. There's also a whole host of analysis and input/output functions that will no doubt continue to expand over time.

On a programming level, you can now call Maybrain as a Python module and there are maybrain.write and maybrain.analysis functions to neatly separate out the writing and analytical functions from those that actually change the connectome, which are still in maybrain.brainObj. All this is explained on the wiki quickstart guide, although other wiki changes are lagging behind the development a little at the moment!

Here's a taster of the degeneration functions on a 10 by 10 grid where each node is connected to its four nearest neighbours.

Openscience.org

To explore how to carry out science in an open way, the best way to proceed has to be to find out who is already doing it and what lessons can be learned from their approach.

Today's post is based on my thoughts after looking at openscience.org. It's a website that hosts open source software for scientists, with  the majority of programs within maths, computational chemistry and life sciences. It also hosts a blog written by Daniel Getzler which gives pointers to good uses of open source within scientific research.

The concept of open source software is one that has existed for a long time and provides probably the closest working model of what my ideal research environment might look like. In fact, many of those writing open source software are researchers, who have a vested interest in as many people using their software as possible and little reason to restrict use or charge for their code.

What the open source software model tells us is that open collaboration can work and produce some really quite stunning results. Linux springs to mind - an entire computer operating system for free, mostly built by people in their spare time and given away for the benefit of many thousands around the world. Delving deeper, the structures used for software development can also be instructive. For example, the numbering of software versions shows a clear development and gives a staging point for other pieces of software to reference and establish compatibility. Bug tracking gives a nice method for community feedback, which might be feature suggestions or code deficiencies.

Where I see the most obvious application of these kinds of methods to experimental science is in protocols and standard operating procedures. If you are not familiar with these concepts, just think of them as the recipe for an experiment. Now imagine coming up with a new procedure for converting lead into gold. Under the current system, when you publish there could be all sorts of questions as to whether the protocol was robust. I have heard anecdotal evidence that in some fields important steps are deliberately missed out in papers to create difficulty for anyone trying to replicate the experiment. Now imagine that the protocol was published as a version 1.0 on a website similar to gihub or googlecode, or maybe something like these crossed with Wikipedia. On the download page there is soon a comment from Dr Rumpelstiltskin that his lab has successfully reproduced the results. This is confirmation that it works - under the current system no such feedback is possible without another publication that no-one is interested in reading. Later Dr Al-Kumis adds an improvement that increases the yield by 10%. With your permission the protocol is moved on to version 2.0. And so on and so on.

Ultimately your protocol can be used by many people, but you can still maintain ownership of it if you wish, or hand it on to others (i.e. keeping or sharing adminitrator rights). I think the move away from traditional publishing also allows for the addition of comments like 'this only worked with my left hand' which might be necessary for the protocol to work, but cause embarrassment in a published paper. I see no reason why this cannot be combined with peer review, the thing that many researchers value about publishing above all else.

And if you have any suggestions for an improvement on this method for publishing protocols, please let me know.

An open science work flow

I'm convinced that research should be done in a more open manner. The reasons for this are discussed in numerous places by people who have taken much more time thinking it over than me (I'm also including networked science in this). What I have found less information on is how you go about it. Even the definition can be a bit vague. To me, open research means that everything you do in the project can be scrutinized and commented on by others who can also become involved in the project.

An open project requires a well defined structure so that potential collaborators can come in at any point. It also helps keep the project on track and stop it it either heading off in directions it was never intended to (ok, sometimes this is good but you need to know it's happening!), or get stuck in a rut with people spending all their time on an insignificant part.

As an aside, I was recently in a room of people discussing some data. The data was good, in fact very good. The problem was that they had not though in advance how they were going to use it! It seemed to me that the project had lost sight of being useful in practice and even a proof of principle problem had not really been defined. Just asking what the end product of the project was would have been quite useful, and should have been set out much earlier on.

Below is the work flow of a typical research project. Ok, I know it doesn't work out like that in a lot of cases, but if you think about it either it should have been like that, or it was and you just couldn't see it at the time.

An open project needs collaborative tools for all of these stages. To me what is exciting is that you can then taken any stage in isolation and work on just that. What does that mean? Imagine a scenario where you have a great idea for a project but just don't have the time to do it. You post it to a message board for open questions and someone else picks up on it, expands the problem and go and solves it. You come back at the analysis stage when the results are put on a blog (linked to your original question) and realise it has results that you can feed back into your other current research.

Even more importantly, it divorces the necessity for the person collecting the data to be the one who analyses it. Very few people have the capability to do both of these well. As long as the aims of the project are well expressed and understood, you can have experts working at every stage to really make the most of the project.

Where do we go from here? Well, you need the tools for each stage and that is something I would like to discuss in future posts.

Maybrain

This will be brief, more details to follow! I've just established a project on googlecode called MayBrain. Ok, so the name is a bit lame, but it is trying to get at the project being the application of Mayavi, a powerful 3D visualisation program in Python, to look at brain data.

Very little of the plotting code is original, it's just patching together modules for loading, analysing and visualising the data. The hope is that it will be useful for those working in neurology to quickly visualise data, overlay several different kinds of data and create videos to share.

If you want to get involved, please do! I'm indebted to my brother, Tim, for kicking off the project and writing a lot of the code before I became involved.

Over the next few days and weeks, the plan is to add a few extra features, finish off the GUI and write  a wiki so that it can actually be useful to someone some day.

Resurrecting the blog

This blog has been out of commission for quite some time. There are various reasons for that, but now I find I have time and motivation to get it up and going again. However, it will look a bit different, at least that is the plan.

I'd like to blog about things I'm enthusiastic about. What does that mean? Well, as far as research goes, it's Open Access, but more than that open research (aka networked research). Most published science is behind closed doors, but so also is most active science. Why? This is a question I have been thinking about for some time now and here are the principal reasons that I see:

1. Inertia in the system. The changing nature of information availability has not been matched by a change in attitude to how information is shared. Scientists are very innovative in how they think about their work, but on the whole much less innovative in how they approach a project. In addition, those currently at the top are those who have thrived in the current system - why change?

2. Credit. Every culture has its currency. In research, it is the publication and the impact of that publication. This means that each piece of research is inexorably tied to a few people and most principally the first author on the resulting paper. This means two things. Firstly, the first author must do most of the work as they are the ones to get the most credit. Secondly, collaborations where two or more people contribute to a significant degree are not in anyone's interest because only one will get the credit.

3. Funding. What drives the above two is, of course, where the money comes from. Projects are funded, not people, and the basis of allocating the money is based on publications (ok, a few other things, but that's the main factor at play). Publish more, get more money.

Not all research should be out in the open. Some of it is commercially sensitive, but most is not. In fact, sharing resources and pooling skills should lead to much more rapid progress.

One of the aims of this rejuvenated blog will be to find out who is doing open science and see how successful are being with it. I also have one or two projects myself that I will put updates of on here. Let's see if the future can be a lot more open than the past.

Bruker nanostar

Lurking in the basement of Reading's chemistry department is  a shiny new Bruker Nanostar instrument. In the dim, distant past when I wasn't indoctrinated with science, I remember the main association with x-rays was ray-gunning people and looking at your skeleton. Well, these days I sagely realise there's a lot more that can be done with them. In particular, the Nanostar instrument will measure what is known in the trade as small angle x-ray scattering (SAXS). I won't go through the details, but essentially the principle is that you can measure really small things with really small wavelengths. This means potentially structures of protein crystals and, in my case, structures of water channels a few nanometres thick. 

So, how have I got on with the instrument? My assessment is basically: hardware good, software bad. We have a detector of a type which will take a full x-ray beam without being burned to smithereens, unlike the old style of detectors. I hear it's not quite as good a resolution as some of the other state of the art detectors, but it's easily good enough for the applications we've put it to so far. Everything seems pretty sturdy. Shortening the sample-detector distance is time-consuming I'll grant, but not difficult. We're probably exceptional in needing that fairly routinely and I've got it down to about 45 minutes to do the swap-over. 

My issues come when you start to use the software. It simply isn't intuitive. A simple example: to make the simplest kind of measurement requires going into the menu 'collect', selecting the submenu 'scan' and then choosing one of the following options: 'add', 'single run' or 'multi-run' (among other options). Can you tell me why any of those menu names would be the most obvious choice? No, thought not. Also, reading this I'm sure you have no idea what any of those three options do. It transpires that the easiest way to run several samples in different positions is 'single run'. I still haven't worked out what 'multi-run' does. 

There is a scripting language, with which you can put in commands (garnered from a log file or 'Help - Errata', obviously) and get the instrument to do more or less what you want. The whole thing smacks of software written by programmers who don't understand the capability of your average bench-top chemist. The prospect of writing a program is a pretty daunting one for most of our users. In my case, using grazing incidence, it is the only option since there is no standard measurement routine that controls sample angle. 

At the end of the day, we are happy with the results we have (lots of pretty spots and so on) but for an instrument with the capability and cost of the Nanostar, I am very disappointed with the standard of the software. I don't buy the argument that extra capability makes the software necessarily difficult to handle. There's already two pieces of software for simple and advanced control to stop stupid people messing things up. The simple one just isn't simple enough and feels like it was written in a rush by someone who has never run an x-ray scattering pattern.