How does one judge the health of a publication such as the SPE
Journal? To the publisher, subscriptions, whether print or electronic, and
subscription revenue traditionally are the major indicators of health. The
current environment changes this answer dramatically. Large databases of
manuscripts and search engines that scan multiple databases have made it easier
to locate and obtain content. Maintaining a subscription to particular journals
is no longer necessary to remain up to date on what is appearing in the pages
of a journal. For the SPE Journal, our articles are fully integrated
within the OnePetro library of
technical documents from the moment of publication and often before because our
technical content draws heavily from the SPE technical proceedings. A reader
effectively has all of the benefits of subscription without needing to purchase
a subscription explicitly. SPE Journal does not receive a credit every
time one of our papers is downloaded, thereby complicating the publisher’s
assessment of health. The continued evolution of electronic publishing and
dissemination are likely to make this traditional assessment even more
difficult in the future.
Obvious answers to the question of the health of a journal within the
research community appear to be that people read and cite the articles that the
journal publishes and that authors send what they perceive to be their best
work to be considered for its pages. A broad base of contributors, editors, and
reviewers, embodying geographical and intellectual diversity, is also a good
qualitative predictor of continued publication of important research in the
coming years. Hence, researchers likely focus on the "intellectual health" of
the publication and the pipeline of publications that represents
sustainability. Intellectual health is the topic that I wrestle with for the
remainder of this summary.
To make comparisons and quantitative assessments of journals, we have turned
to indices such as the impact factor (number of citations to articles from the
previous two years within a journal vs. the number of citable items for those
years.) A 5-year impact factor is also computed frequently in recognition of
the longer gestation period for articles within some disciplines. Some
institutions assess their faculty and researchers on the basis of metrics such
as the impact factor of journals where they publish, thereby giving weight to
one or more indices. Among publications in the petroleum engineering science
and geosciences community, the SPE Journal fares well using
the impact factor. Our impact factor and 5-year impact factor through 2009 are
1.089 and 1.455, respectively1. In comparison, the scores for the
SPE Reservoir Evaluation & Engineering journal are 0.538 and 0.691,
The impact factor is not without its shortcomings, however. Consider the
case of Acta Crystallographica Section A (Dimitrov et al. 2010).
Historically, that journal's impact factor was roughly 2. A paper in 2008
appeared that was a historical summary of a software program used to refine the
description of crystal structures. The software is downloadable freely for
nonprofit use, and the article is rightly referenced as a description of the
software. Since the appearance of this single article, the impact factor of
Acta Crystallographica Section A has soared to 40.93 in 2009. This is an
interesting example of how the impact factor for a journal can be swayed by
particular publications because it does not permit any inference about or
removal of outliers. Additionally, the impact factor of that journal will drop
again after the two-year period of the index passes.
Additional metrics are needed for a more complete and meaningful assessment.
Three proposals follow.
- The median number of citations generated per article without limit of time
published speaks to the relevance, importance, and longevity of information
within the papers published. The median clearly deals with the problem of
outliers. This quantity is also easier to explain than impact factor,
eigenfactor, or h index.
- Ideally, we would like to know how many people actually read any given
article because a true measure of impact is the number of times an article is
read. Online article databases, such as OnePetro, are well
positioned to collect meaningful proxies for the number of times a paper is
read. For any article, full statistics of the viewing of its abstract and the
downloading of the entire manuscript are collectable and reportable. The median
number of times an article is downloaded likely gives a different glimpse of
the impact and health of a journal in the era of online dissemination of
- Finally, it is curious that SPE conducts surveys of journal reader
satisfaction yet does not assess the author's experience with publishing.
Completion of a short survey by the corresponding author of every paper
published likely would provide us with important insight. Except for almost
purely anecdotal information, we do not know author satisfaction regarding the
time needed for review, decision making, revision where appropriate, and
publication when a manuscript is expected. We also do not know authors'
opinions about whether revisions requested as a part of peer review actually
improved their manuscript. Importantly, I would like to know if authors
believed that they are submitting their best work for evaluation. Just as an
acid test determines if a metal is real gold, this serves as one important test
of journal health.
1 Clearly, the number of significant figures reported is
problematic as SPE Journal, for instance, published 74 articles in 2009.
The division operation suggests that only 2 figures of the impact factor for
SPE Journal are significant.
Dimitrov, J.D., Kaveri, S.V., and Bayry, J. 2010. Metrics: Journal's Impact Factor
Skewed by a Single Paper. Nature 466 (8 July 2010):
179. doi: 10.1038/466179b.