Tales from the arXiv: Impact factor edition

Here is an abstract, just posted on the arXiv, that attempts to analyze the inflation in journal impact factors over the years:

arXiv:0804.3116
Date: Sat, 19 Apr 2008 00:17:22 GMT (995kb,D)

Title: Differences in Impact Factor Across Fields and Over Time
Authors: Benjamin M. Althouse, Jevin D. West, Theodore Bergstrom, Carl T.
Bergstrom
Categories: physics.soc-ph
Comments: 9 pages, 3 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
\\
The bibliometric measure impact factor is a leading indicator of journal
influence, and impact factors are routinely used in making decisions ranging
from selecting journal subscriptions to allocating research funding to deciding
tenure cases. Yet journal impact factors have increased gradually over time,
and moreover impact factors vary widely across academic disciplines. Here we
quantify inflation over time and differences across fields in impact factor
scores and determine the sources of these differences. We find that the average
number of citations in reference lists has increased gradually, and this is the
predominant factor responsible for the inflation of impact factor scores over
time. Field-specific variation in the fraction of citations to literature
indexed by Thomson Scientific's Journal Citation Reports is the single greatest
contributor to differences among the impact factors of journals in different
fields. The growth rate of the scientific literature as a whole, and
cross-field differences in net size and growth rate of individual fields, have
had very little influence on impact factor inflation or on cross-field
differences in impact factor.
\\ ( http://arxiv.org/abs/0804.3116 , 995kb)


If I am interpreting this abstract correctly, the authors of this paper seem to be concluding that the differences one sees between fields is predominantly the result of which journals reporting venues (in this case, Thomson Scientific's Journal Citation Reports) choose to index. In other words, they conclude that the dominant factor is basically noise at best and corruptible choices at worst. That's just lovely. (I haven't read the paper, so I am offering no opinion as to whether I agree with the authors' conclusion. I'm just saying that this is an extremely disturbing conclusion.)