Thursday, 4 June 2015

What could be done to change the impact factor?

How we love and love to hate the impact factor. Scholars have an undwindling attraction to its simplicty, while at the same time bemoaning its oversimplicity and crude application. Its strength has also been its downfall, you could say. Except that no downfall seems to be iminent, despite the hopes of many.

So, what to do? There's no prospect of the revolutionaries who would ditch this all-pervasive metric succeeding, and nothing to replace it that comes close to holding the same kind of sway over grant bodies and promotion committees. Here's an option that I don't think has been given a serious airing as yet: let's try improving the impact factor. Revolution is not working, so let's try an evolutionary approach.

I have a couple of suggestions of my own, but I'll keep them under my hat for now because the aim of this post is to make you think. What would a 'better' impact factor look like? Here's a few criteria I think it has to follow:

1. It has to be intuitively simple. The beauty and attraction of the impact factor are that anyone gets a grasp of what it's trying to measure: if you publish in this journal, on average you will receive this many citations. Arguments aside about how well it achieves such a feat, you can't beat it for conciseness.

2. It must be a citation metric. Wonderful and interesting as alternative metrics are (e.g. Facebook likes, mentions on Twitter and the like), scholars rate success on how many other scholars mention your of work when they publish themselves. It's a high-brow popularity contest. Other metrics simply don't correlate well with citations.

3. It has to solve at least one of the flaws of the impact factor. Hopefully it's a given that any new proposal must be better than what is currently on offer.

4. It should give a fair reflection of a body of work. The impact factor is not an article level metric, and there are occasions where that is useful - usually where you have a great deal of data to analyse and need a brief summary. 

That's my criteria. Suggestions, additions, variations and comments are welcome.

No comments:

Post a Comment