Impact of the Journal of Reading Behavior on reading scholarship, The

impact of the Journal of Reading Behavior on reading scholarship, The

McKenna, Michael C

The Social Science Citation Index was used to gather data concerning the frequency with which articles published in volumes 1 to 25 of the Journal of Reading Behavior have been cited in professional journals. The nature and range of those citations were also examined. These data were used to track the impact of the journal over the first 25 years of its history and to identify the 30 most frequently cited articles. These were analyzed by content, and the top lo were examined in greater detail. A summary and a recommendation are offered.

JUDGING THE IMPACT Of a professional journal is problematic under the best of circumstances and invites skewed and prejudicial thinking under the worst. In this article, we have taken an approach that we believe is as objective and bias-free as possible, though we acknowledge that some readers may not endorse our methodology. Our approach embodies a combination of quantitative and qualitative means, beginning with the frequency with which articles appearing in the Journal of Reading Behavior (JRB) have been cited and proceeding to a qualitative analysis of their topics and to subsequent quantitative analyses as well. Our goals were to (a) measure the frequency of citations of every featured article appearing in volumes 1 to 25, (b) track the total number of IRS citations by volume, (c) list the most frequently cited articles in rank order, (d) characterize the topics these articles addressed, and (e) examine the highest ranking articles in terms of the nature and range of the citations they have elicited.

Method

We began by compiling the tables of contents of the goo issues constituting volumes 1 to 25. We then entered bibliographic information concerning each featured article into the Social Science Citation Index (SSCI) in order to determine the number of citations as of September,1998. The sscI fully indexes 1,700 journals in 50 disciplines “as well as covering individually selected, relevant items from over 3,300 of the world’s leading scientific and technical journals” (SSCI,1998). At the time of our search, the database contained references to 2.8 million articles. Because sscI provides journal citations only, it underestimates the actual total. However, it does so in a consistent and systematic manner that applies to all of the J R B articles in the same way. Comparisons using ssci totals are therefore defensible, although it is important to remember that they systematically exclude some sources, such as books and paper presentations.

Three versions of SSCI are presently available. One is the traditional print version, which necessitates cross-referencing three ssci volumes simultaneously in order to locate citations. The second is a collection of compact disks which enable searches to be conducted electronically However, each disk is limited to a specific time frame so that citations before and after those stored on a given disk are not accessed. We chose to use the third and newest version of SSCI, which involves access through a web site available by subscription. The web site includes the entire database so that searches are comprehensive.

Because the on-line database begins in 1974, it is likely that some of the citations to articles in the earliest volumes of JRB were omitted, as volume 1 of the journal appeared in 1969. On the other hand, the lag between the appearance of an article and the publication of a subsequent article in which the first is cited can be considerable. For this reason, we concluded that the use of SSCI did not seriously disadvantage the earliest contributors to J R B. Similar reasoning led to our decision to limit the search to the first 25 volumes in that the number of citations of more recent articles may not reflect their real impact, because other scholars may have had insufficient time to reference them.

Finally, we believed that the feature articles of JRB were the proper target of an investigation of the impact of the journal. We noted that through its first 25 years the journal published contributions in a variety of other formats, labeled Sourdough, Readers’ Dialogue, Critical Perspectives in Reading, Brief Research Reports, Guest Editorials, Critical Issues, Book Reviews, From the Editor, and Research Methodology. Our view was that the feature articles were superior to these as an indicator of influence within the community of reading scholars, and that few if any of the publications appearing under these headings were likely to compare numerically with the most frequently cited feature articles.

Results and Discussion

The output of the ssci searches led to the number of journal citations for each JRB article and also to bibliographical information for the sources containing the citations. We looked first at overall journal trends and second at results concerning individual articles.

Long-Term Trends

Figure 1 graphically depicts how the total number of citations has varied over the first 25 years of the journal. After a 5-year start-up period involving few citations, an 11-year interval (volumes 6-16 ) characterized by relatively high citation frequencies can be observed. This period was followed by a 9-year span (volumes 17-25) during which the level of citations was more modest.

It is tempting to dismiss the results for the first 5 volumes as underestimates,because the SSCI electronic database does not contain entries prior to 1974 and several years may have been required in order to establish the reputability of J R B. These two possibilities are not, of course, mutually exclusive. We did note that of 130 feature articles appearing in volume 1 to 5, 112 (86%) were not cited at all from 1974 to the present. By contrast, the most influential articles from JRB have a much longer span during which they are cited, a phenomenon to which we later return. It is with volume 6 that IRS seemed to have realized its potential as a forum for articles with substantive impact on reading scholarship. The dramatic increase in the number of citations between volumes 5 and 6 cannot be attributed to one or two outstanding articles. Rather, of the 33 articles featured in volume 6, 25 (76%) have been subsequently cited. The other striking increase during this intermediate period of JRB is between volumes 10 and 11. Here, much of the increase is attributable to a single article, the most frequently cited in the first 25 volumes: Pearson, Hansen, and Gordon (1979).

We wondered whether article length might differentiate those appearing in the first 5 volumes from those of volumes 6 through 16. We therefore compared the length of articles in volume 1 with those appearing in volume ii, the most frequently cited volume of the first 25. Surprisingly, we found that for volume i the mean length of the 22 feature articles was 14.2 pages (SD = 5.4), whereas for volume m the mean length of its 27 articles was 10.7 pages (SD =2.9), a significant decline in article length (t =2.7, p = .011). In consequence, a possible trend toward publishing lengthier (and presumably more substantive) articles not only cannot account for the higher frequency of citation during the middle period of JRB, but such a trend did not occur at all.

The relatively low plateau that commences with volume 17 leads to another tempting conclusion – namely that the recency of these later volumes prevents the total number of citations from being a dependable indicator of the eventual influence of many of these articles. Such an argument is impossible to refute. The beginning of this era, however, predated our investigation by 14 years. It therefore seems unlikely that a recency effect can adequately account for the relatively low number of citations during a period when the size of the reading research community (as evidenced, for example, by National Reading Conference membership) was greater than during the first 16 years of the journal. It is possible that differences in frequency of citation can be accounted for by a change in the number of feature articles published. For volumes 7 to 16, JRB averaged 24 articles per year, whereas for volumes 17 to 25, only 18.8 articles,per year appeared. However, the average number of citations per article dropped significantly between these two periods. For volumes 7 to 16, articles averaged 8.0 citations each (SD 12.7), compared with 5.5 citations (SD = 6.0) for volumes 17 to 25 (t= 2.6,p = .009). These results suggest a real decline in the impact of JRB on the profession between the two eras. This judgment must be tempered by the possibility that future citations may appreciably alter the situation, though we find this possibility rather remote given the lapse of time.

We wondered whether the growing number of qualitative research articles appearing in JRB might have occasioned fewer citations, as their increasing presence represented a major shift in the later volumes. We tested this possibility by categorizing the 19 feature articles of volume 25 according to research methodology We found that io articles used quantitative methods, 5 used qualitative approaches, and 4 used mixed designs. We eliminated those that employed mixed approaches. The quantitative articles averaged 2.8 citations per article (SD =2.2), whereas the qualitative articles averaged 4.o citations per article (SD = 1.0). The difference was not significant (t = 1.1, p = .27), and the notion that qualitative JRS studies may be cited less frequently than quantitative research reports was not supported for this sample.

The Most-Cited Articles

From a total of 572 feature articles appearing in volumes i to 25, the 30 most frequently cited appear in rank order in Table 1. The top two articles (Pearson et al.,1979, and Paris & Myers, 1981) are literally without peer in the number of citations. Although these two articles have been cited in professional journals i28 and 87 times, respectively, the remaining 28 of the top 30 range from 20 to 49 citations. Clearly, these two articles merit

special distinction.

In order to identify the topics addressed by frequently cited articles, we conducted a content analysis of the titles. The categories that emerged from this procedure also appear in Table 1. It is evident that those topics closely related to comprehension characterize the majority of the top 30 articles. Aspects of comprehension such as prior knowledge effects, self monitoring, text structure, and vocabulary are especially evident. This is not surprising considering the fact that most of these articles were published in the late1970s and early 1980s, an era during which comprehension was a principal focal point of research activity (Stahl, in press). Among these articles, there is comparatively little attention to issues that have characterized reading research in more recent years, such as social dimensions of reading acquisition (though see Allington, 1984; Johnston & Winograd, 1985). There are, on the other hand, several precursors of the present-day interest in decoding (see Biemiller,1979; Ehri,1987; Guthrie & Tyler,1976; Masonheimer, Drum, & Ehri,1984; Schreiber, 1980; Stanovich, Cunningham, & West, 1981).

In an effort to get beyond the sheer number of citations as an indicator of impact, we examined the citation history of the top ix articles in more detail. Specifically, we examined all of the citing articles in terms of their publication dates.and also the variety of journals in which they appeared. The results of this analysis appear in Table 2.

We noted an enduring impact for all of these articles, evidenced by their having been cited within the last year. This was true regardless of publication date, and two of the articles have now been in print for over zo years (Klare,1976; Meyer,1975). It is probable that some of these articles have attained “classic” status, leading to nearly routine citation in subsequent articles on related topics. In fact, Klare’s article was specifically identified in a reprinted version as a “citation classic.” We also suspected that the interval over which an article is referenced by other scholars is an important characteristic of impact. For the top 10 JRB articles of volum 1 to 25, this span of citations ranged from 8 to 21 years, though the 8-year span for Ehri (1987) is perhaps an outlier due to recency of publication. For all ix articles, citations first occurred soon after their appearance and have continued to the present.

Another indicator of impact is the variety of journals in which citations occur. We reasoned that because journals do not have perfectly overlapping readerships, the number of different journals in which an article is cited is one reflection of an article’s breadth of influence. An inspection of Table 2 reveals an obvious correlation between the total number of citations and the number of journals in which they appear. This relationship is rather strong (r = .83, p = .003 ), though it is less than perfect. Paris and Myers’ (1981) article, for example, has been cited in more journals than that of Pearson et al. (1979) even though it is a distant second in terms of total citations. Klare’s (1976 ) article has been referenced in a surprisingly wide variety of sources, and the ratio of citations to journals for this article is only i.z. This finding may be due to its topic (readability), which has had applications in many areas. Overall, the io most frequently cited articles have been referenced in an average of 27.9 different journals (SD = 10.7).

A Caveat

A trend that roughly parallels the first 25 volumes is the proliferation of journals in education. For example, the first volume of the Current Index to Journals in Education (CIJE, 1969), the publication of which coincided with volume 1 of J R B, listed 300 education and eucation-related journals as sources. Volumes 25 of CIJE, in contrast, listed 780 journals. The emergence of more forums in which to cite and be cited complicates any attempt to gauge the impact of a journal or its individual articles in terms of citations. On the one hand, more journals mean more articles with the potential of citing a JRB piece. Thus, overall citation rates should rise. On the other hand, more journals also mean that a given author may face a multitude of possible citations on a particular topic and would be compelled to cite selectively. Thus, the overall citation rates should fall.

A further complication is that these possibilities are not mutually exclusive. That is, the greater the variety of sources in which an article might be cited may offset, in part, the need for authors to be increasingly selective and thus to be less likely to cite it. In our view, these possible trends do not diminish the validity of using citation totals as an indicator of impact for individual articles and for the journal as a whole. However, they do suggest some important qualifications to any conclusions drawn from this approach.

Summary and a Recommendation

Journal citations of the feature articles in the first 25 volumes of TRS reveal rather distinct periods of influence on reading research and informed opinion. The first 5 volumes do not appear to have had much lasting influence, whereas the next 11 have contained a large number of widely referenced articles. The decline in citations, beginning with volume 17 and continuing through volume 25, is difficult to explain on the basis of recency or the number of feature articles published. The most frequently cited articles across the 25 volumes investigated here tend to concern comprehension– related topics, although word recognition is another important focus. In its first 25 volumes, JRB has published a number of enduring articles to which present-day scholars in a variety of disciplines continue to refer.

Retrospective analyses of this kind offer, in our view, benefits that extend well beyond satisfying our historical curiosity. They can serve a metacognitive function as well. We respectfully suggest that the leadership of the National Reading Conference use SSCI on a periodic basis as a means of monitoring the impact of articles appearing in JRB/JLR in an effort to identify and substantiate trends, guide editorial policy, and acknowledge authors and sources that have made a difference.

Author Note

We wish to thank Steven A. Stahl of the University of Georgia for his suggestions in planning this investigation. We also wish to express our appreciation to the Reading Department of that institution for granting on-line access to the Social Science Citation Index.

Volume 15 of JRB was not Paginated consecutively, and issue numbers are required for citations.

REFERENCES

Allington, R.L. (1984). Content coverage and contextual reading in reading groups. Journal of Reading Behavior, l6, 85-96.

Baker, L. (1979). Comprehension monitoring: Identifying and coping with text confusions. Journal of Reading Behavior, m, 365-374

Biemiller, A. (1979 ). Changes in the use of graphic and contextual information as functions of passage difficulty and reading achievement level. Journal of Reading Behavior, ii, 307-318.

Burger, N.S., & Perfetti, C.A. (1977). Reading skill and memory for spoken and written discourse. Journal of Reading Behavior, g, 7-16.

Ehri, L.C. (1987). Learning to read and spell words. Journal of Reading Behavior, 19, 5-32.

Fisher, D.E, & Frankfurter, A. (1977). Normal and disabled readers can locate and identify letters: Where’s the perceptual deficit? Journal of Reading Behavior, g, 31-44.

Fitzgerald, J., & Speigel, D.L. (1983). Enhancing children’s reading comprehension through instruction in narrative structure. Journal of Reading Behavior, 15 (z), 1-18.

Gambrell, L.B., & Heathington, B.S. (1981). Adult disabled readers’ metacognitive awareness about reading tasks and strategies. Journal of Reading Behavior, 13, 215-222.

Garner, R. (1980). Monitoring of understanding: An investigation of good and

poor readers’ awareness of induced miscomprehension of text. Journal of Reading Behavior, 12, 55-64.

Garner, R. (1982), Verbal-report data on reading strategies. Journal of Reading Behavior, 14, 159-168.

Guthrie, J.T., & Tyler, S.J. (1976). Psycholinguistic processing in reading and listening among good and poor readers. Journal of Reading Behavior, 8, 415-426. Hiebert, E.H., Englert, C.S., & Brennan, S. (1983). Awareness of text structure in

recognition and production of expository discourse. Journal of Reading Behavior, 15 (4),63-80.

Jackson, N.E., & Butterfield, E.C. (1989 ). Reading-level-match designs: Myths and realities. Journal of Reading Behavior, ai, 387-412.

Johnston, P H., & Winograd, EN. (1985 ). Passive failure in reading. Journal of Reading Behavior, 17, 279-302.

Klare, G.R. (1976 ). A second look at the validity of readability formulas. Journal of Reading Behavior, 8, 129-152.

Lenin, J.R., & Divine-Hawkins, P (1974). Visual imagery as a prose-learning process. journal of Reading Behavior, 6, 23-30.

Lipson, M.Y (1982). Learning new information from text: The role of prior knowledge and reading ability journal of Reading Behavior, 14, 243-262. Masonheimer, EE., Drum, PA., & Ehri, L.C. (1984). Does environmental print

identification lead children into word reading? Journal of Reading Behavior, ib, 257-272.

McKeown, M.G., Beck, LL., Omanson, R.C., & Perfetti, C.A. (1983). The effects of long-term vocabulary instruction on reading comprehension: A replication. Journal of Reading Behavior, i5 (i), 3-18.

Meyer, B.EJ. (1975). Identification of the structure of prose and its implications for the study of reading and memory Journal of Reading Behavior, 7, 7-48. Paris, S.G., & Myers, M. (ig8i). Comprehension monitoring, memory and study

strategies of good and poor readers. Journal of Reading Behavior, 13, 5-22. Pearson, ED., Hansen, J., & Gordon, C. (1979). The effect of background knowledge on young children’s comprehension of explicit and implicit information.

Journal of Reading Behavior, ii, 201-210.

Purcell-Gates, V> & Dahl, K.L. (1991). Low-sas children’s success and failure at early literacy learning in skills-based classrooms. Journal of Reading Behavior, 23, 1-34.

Raphael, TE., & McKinney J. 0983). An examination of fifth- and eighth-grade children’s question-answering behavior. Journal of Reading Behavior, is (3), 6786.

Royer, J.M., Hastings, C.N., & Hook, C. (1979). A sentence verification technique for measuring reading comprehension. Journal of Reading Behavior, 11, 355364

Schreiber, EA. (i98o). On the acquisition of reading fluency. Journal of Reading Behavior, iz, 177-186.

Social Science Citation Index. (1998). The web of science. Institute for Scientific Information. [Path is by subscription only ]

Stahl, S. (1983 ). Differential word knowledge and reading comprehension. Journal of Reading Behavior, 15, 33-50.

Stahl, S.A. (in press). Understanding shifts in reading and its instruction. Peabody Journal of Education.

Stanovich, K.E., Cunningham, A.E, & West, R.E (1981). A longitudinal study of the development of automatic recognition skills in first graders. Journal of Reading Behavior, 13, 57-74.

Wade, S.E., & Adams, R.B. (i99o). Effects of importance and interest on recall of biographical text. Journal of Reading Behavior, 22, 331-354.

Winograd, E, & Johnston, P (1982). Comprehension monitoring and the error detection paradigm. Journal of Reading Behavior, 14, 61-76.

Michael C. McKenna

GEORGIA SOUTHERN

UNIVERSITY

Richard D. Robinson

UNIVERSITY OF

MISSOURI-COLUMBIA

Copyright National Reading Conference Mar 1999

Provided by ProQuest Information and Learning Company. All rights Reserved