Kovscek

How does one judge the health of a publication such as the SPE Journal? To the publisher, subscriptions, whether print or electronic, and subscription revenue traditionally are the major indicators of health. The current environment changes this answer dramatically. Large databases of manuscripts and search engines that scan multiple databases have made it easier to locate and obtain content. Maintaining a subscription to particular journals is no longer necessary to remain up to date on what is appearing in the pages of a journal. For the SPE Journal, our articles are fully integrated within the OnePetro library of technical documents from the moment of publication and often before because our technical content draws heavily from the SPE technical proceedings. A reader effectively has all of the benefits of subscription without needing to purchase a subscription explicitly. SPE Journal does not receive a credit every time one of our papers is downloaded, thereby complicating the publisher’s assessment of health. The continued evolution of electronic publishing and dissemination are likely to make this traditional assessment even more difficult in the future.

Obvious answers to the question of the health of a journal within the research community appear to be that people read and cite the articles that the journal publishes and that authors send what they perceive to be their best work to be considered for its pages. A broad base of contributors, editors, and reviewers, embodying geographical and intellectual diversity, is also a good qualitative predictor of continued publication of important research in the coming years. Hence, researchers likely focus on the "intellectual health" of the publication and the pipeline of publications that represents sustainability. Intellectual health is the topic that I wrestle with for the remainder of this summary.

To make comparisons and quantitative assessments of journals, we have turned to indices such as the impact factor (number of citations to articles from the previous two years within a journal vs. the number of citable items for those years.) A 5-year impact factor is also computed frequently in recognition of the longer gestation period for articles within some disciplines. Some institutions assess their faculty and researchers on the basis of metrics such as the impact factor of journals where they publish, thereby giving weight to one or more indices. Among publications in the petroleum engineering science and geosciences community, the SPE Journal   fares well using the impact factor. Our impact factor and 5-year impact factor through 2009 are 1.089 and 1.455, respectively¹. In comparison, the scores for the SPE Reservoir Evaluation & Engineering journal are 0.538 and 0.691, respectively.

The impact factor is not without its shortcomings, however. Consider the case of Acta Crystallographica Section A (Dimitrov et al. 2010). Historically, that journal's impact factor was roughly 2. A paper in 2008 appeared that was a historical summary of a software program used to refine the description of crystal structures. The software is downloadable freely for nonprofit use, and the article is rightly referenced as a description of the software. Since the appearance of this single article, the impact factor of Acta Crystallographica Section A has soared to 40.93 in 2009. This is an interesting example of how the impact factor for a journal can be swayed by particular publications because it does not permit any inference about or removal of outliers. Additionally, the impact factor of that journal will drop again after the two-year period of the index passes.

Additional metrics are needed for a more complete and meaningful assessment. Three proposals follow.

• The median number of citations generated per article without limit of time published speaks to the relevance, importance, and longevity of information within the papers published. The median clearly deals with the problem of outliers. This quantity is also easier to explain than impact factor, eigenfactor, or h index.
• Ideally, we would like to know how many people actually read any given article because a true measure of impact is the number of times an article is read. Online article databases, such as OnePetro, are well positioned to collect meaningful proxies for the number of times a paper is read. For any article, full statistics of the viewing of its abstract and the downloading of the entire manuscript are collectable and reportable. The median number of times an article is downloaded likely gives a different glimpse of the impact and health of a journal in the era of online dissemination of technical information.
• Finally, it is curious that SPE conducts surveys of journal reader satisfaction yet does not assess the author's experience with publishing. Completion of a short survey by the corresponding author of every paper published likely would provide us with important insight. Except for almost purely anecdotal information, we do not know author satisfaction regarding the time needed for review, decision making, revision where appropriate, and publication when a manuscript is expected. We also do not know authors' opinions about whether revisions requested as a part of peer review actually improved their manuscript. Importantly, I would like to know if authors believed that they are submitting their best work for evaluation. Just as an acid test determines if a metal is real gold, this serves as one important test of journal health.

¹ Clearly, the number of significant figures reported is problematic as SPE Journal, for instance, published 74 articles in 2009. The division operation suggests that only 2 figures of the impact factor for SPE Journal are significant.

Reference

Dimitrov, J.D., Kaveri, S.V., and Bayry, J. 2010. Metrics: Journal's Impact Factor Skewed by a Single Paper. Nature  466 (8 July 2010): 179. doi: 10.1038/466179b.