Thirty Years of Impact Factor and the Journal

Tobin, Martin J

Impact factor has become the most widely used metric for assessing the overall quality of a biomedical journal (1, 2). The 2003 impact factor for AJRCCM has increased to 8.876. The increase of 2.9309 units over the preceding year is the single largest increase ever achieved by the Journal.

Impact factor is calculated as a ratio (1, 2). The numerator of impact factor for AJRCCM in 2003 is the number of citations in all journals in 2003 to articles that had been published in AJRCCM in 2001 and 2002 (9,151). The denominator is the number of articles published in AJRCCM in 2001 and 2002 (585 + 446 = 1,031).

The time periods included in the formula mean that impact factor is slow to reflect improvements made to a journal. The impact factor for 2003 reflects a period starting in January 2001; that is, 3.5 years before the release date. Another year or so can be added to account for the time between initial submission of a manuscript and its final print publication (consequent to the time taken for peer review, manuscript revision, copy-editing, and other delays). Thus, a new rating of impact factor is influenced by how a journal handled a manuscript some 4.5 years before the release date.

The formula for the calculation also makes it possible to predict impact factor for an upcoming year with reasonable accuracy. The denominator for the ratio is known more than a year in advance. Moreover, the numerator rarely falls much from one year to the next-indeed, most journals experience an increase. When AJRCCM’s impact factor for 2004 is released in June 2005, the denominator will be 826 (446 articles published in 2002 plus 380 articles published in 2003). If the numerator for 2004 is the same as for 2003, 9,151 citations, then next year’s impact factor will be 11.079. Moreover, it is reasonable to anticipate a further increase in AJRCCM’s impact factor for 2005: the denominator is likely to be lower than 700, and, if citations do not decrease, impact factor should exceed 13.00.

The varying time frames involved in the calculations can cause confusion when using impact factor to assess the performance of an editorial team that serves a 5-year term-as has been the case for AJRCCM since 1980 (a term shorter than that used by the majority of medical journals [3]). The current editorial team began in September 1999. The first impact factor to reflect the team’s performance was the rating for 2002 (released in June 2003), although that value reflected in part the performance of the preceding editorial team. Only three impact factors provide an unalloyed reflection of the current editorial team: the ratings for 2003, 2004, and 2005 (released in June 2004, 2005, and 2006).

Figure 1 shows every value of impact factor for the American Review of Respiratory Disease and its derivatives, the American Journal of Respiratory Cell and Molecular Biology and AJRCCM, since the index was introduced in 1974 (4). Compared with the impact factor of the leading journal for each internal medicine subspecialty in 2002, AJRCCM (6.567) was close to the bottom; an exception was nephrology, with an impact factor of 5.02 for Kidney International (2). With release of the 2003 impact factor, AJRCCM has moved ahead of journals such as Diabetes (8.290) and Arthritis and Rheumatism (7.190), and is now positioned among middle ranked subspecialty journals. When the 2004 impact factor is released in June 2005, AJRCCM will supersede journals such as Blood (10.120) and approach Circulation (11.164). And when the 2005 impact factor is released in June 2006, AJRCCM will rank with the highest rated subspecialty journals, Journal of the National Cancer Institute (13.844) and Gastroenterology (12.718).

Impact factor has far-reaching ramifications for the future of a medical subspecialty. Outside the United States, impact factor is widely used in deciding how much money is awarded to a research laboratory. Granting agencies decide funding using a formula that incorporates the impact factor of the journal in which each article by an investigator was published during the preceding year (5-7). All journals in both critical care medicine and pulmonary medicine have impact factors that are several units below journals of most other medical subspecialties (2). Consequently, funding for pulmonary and critical care research is likely to fall behind that of other subspecialties. Because more than 70% of manuscripts submitted to AJRCCM come from countries outside the United States (8), a decrease in funding for our discipline adversely affects the quality of manuscripts submitted to the Journal. The strong link between funding and impact factor explains why researchers outside of the United States are focused on this metric almost to the exclusion of the ten other measures of journal performance that I recently discussed (9). The more than doubling of AJRCCM’s impact factor between 2002 and 2005 (Figure 1) will increase funding for pulmonary and critical care research worldwide and hopefully enhance the standing of our discipline among other subspecialties.

Two major reasons explain the recent increases in AJRCCM’s impact factor. Reviewers have become more selective in the manuscripts they recommend for publication; the expectations and demands of scientists who serve as reviewers is the major determinant of the standing of a journal (10, 11). And authors are submitting higher quality manuscripts. These two variables become interlocked in a positive feedback loop (10). Authors submit their best manuscripts to the most prestigious journal they think might accept them. Higher quality manuscripts in turn lead to further increases in a journal’s impact factor. To capitalize on a system regulated by a positive feedback loop, an abrupt increase in momentum is the secret-as this leads to substantial increases in the output variable (in this case, impact factor); this accelerating action is referred to as “runaway” in studies of control of breathing (12). Most journals with an impact factor of 4.00 or below remain on a plateau (or miniscule slope) year after year; it is extremely difficult to force a sharp upward bend in the trajectory and acquire a steep slope. AJRCCM has now acquired that momentum.

Conflict of Interest Statement: MJT is editor of AjRCCM. He receives a fixed stipend from the American Thoracic Society. He does not receive financial support for research from pharmaceutical, biotechnology, or medical device companies. He does not serve as a consultant to or on the advisory board of any company. He receives royalties for two books on critical care published by McGraw Hill, Inc.

References

1. Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? J Med Libr Assoc 2003;91:42-46.

2. Tobin MJ. Impact factor and the Journal. Am J Respir Crit Care Med 2003;168:621-622.

3. Davis RM, Mullner M. Editorial independence at medical journals owned by professional associations: a survey of editors. Sci Eng Ethics 2002; 8:513-528.

4. Garfield E. Significant journals of science. Nature 1976;264:609-615.

5. Adam D. The counting house. Nature 2002;415:726-729.

6. Jimenez-Contreras E, Delgado Lopez-Cozar E, Ruiz-Perez R, Fernandez VM. Impact-factor rewards affect Spanish research. Nature 2002;417: 898.

7. Taubes G. Measure for measure in science. Science 1993;260:884-886.

8. Tobin MJ. American in name, international in scope. Am J Respir Crit Care Med 2004;169:1083-1085.

9. Tobin MJ. Assessing the performance of a medical journal. Am J Respir Crit Care Med 2004;169:1268-1272.

10. Tobin MJ. Rigor of peer review and the standing of a journal. Am J Respir Crit Care Med 2002;166:1013-1014.

11. Hoppin FG Jr. How I review an original scientific article. Am J Respir Crit Care Med 2002;166:1019-1023.

12. Younes M. Proportional assist ventilation. In: Tobin MJ, editor. Principles and practice of mechanical ventilation. McGraw-Hill, Inc. New York, 1994, p. 349-369.

DOI: 10.1164/rccm.2406005

MARTIN J. TOBIN

Editor

Are We Closer to Understanding the How?

Energy foods for active athletes