D-Lib Magazine
October 2006

Volume 12 Number 10

ISSN 1082-9873

Measuring Total Reading of Journal Articles

Donald W. King
Visiting Scholar
University of North Carolina, Chapel Hill
<donaldwking@gmail.com>

Carol Tenopir
Professor
University of Tennessee, Knoxville
<ctenopir@utk.edu>

Michael Clarke
Executive Editor
American Academy of Pediatrics*
<mtclarke@alumni.uchicago.edu>

*At the time of the study, Michael Clarke was at the American Academy of Pediatrics; however, he is now at the American Medical Association, and the email given for him is correct at the time of publication.)

Abstract

There have been hundreds, perhaps thousands, of studies of journal reading by professionals in such fields as science, engineering, medicine, law, social science and the humanities. These studies have been done for many reasons, including research to better understand professional communication patterns and the role this plays in their work. Some studies also focus on providing specific information to journal system participants such as publishers, librarians, other intermediaries and their funders. In this article we present a description of a little used but powerful method of observing reading by scientists (1). This method is designed to measure the amount of reading of specific journal articles and entire journals to complement exclusive observations of electronic journal hits and downloads, transaction logs, limited counts of citations to journals or articles and rough estimates of total amount of reading by professionals compared with total number of articles published.

Introduction

Amount of Reading by Scientists

Most surveys of reading by scientists from the 1960s to the 1990s observed average amount of time spent reading, but not average amount of reading per scientist (King et al. 1981, King & Tenopir 2001). Furthermore, few studies have provided estimates of total amount of reading of specific articles or journals in both print and/or electronic forms. Yet, from a cost and benefit analysis perspective the amount of reading is an essential metric to compare with article and journal publishing costs. Not only is the cost per reading a useful indicator for publishers and their funders, but also to the science and medical communities at large. The value derived from research or other sources of journal articles is the extent to and purposes for which information is used. Generally, the more the information is used the greater its value. Unfortunately, there are flaws in most measures of the amount of reading of scientific articles and journals and the benefits derived from this reading.

Other indicators of amount of use of articles and/or journals have been reported such as number of subscriptions, but this is only useful for comparing to the cost and/or price of journals. Furthermore, number of subscriptions may or may not reflect the extent to which personal and library subscriptions are actually read. Hits and downloads of electronic journals ignore reading from print versions, particularly from personal subscriptions. Citations to articles and journals are also used as an indicator of use and benefits (see, for example, Science and Engineering Indicators 2006). However, reading that leads to citations represents a small fraction of all reading, because more reading is done for purposes other than citing and much reading is done by scientists outside of the universities and research laboratories that encourage authorship. These indicators of article use and/or benefits are clearly useful and relatively easily observed, but they do not tell the entire story, which can be expanded through observing amount and consequences of actual reading.

The Myth of Low Use of Journal Articles

A myth that journal articles are read infrequently persisted over a number of decades (see, for example, Williams 1975, Lancaster 1978, Schauder 1994, Odlyzko 1996). In fact, early on this misconception led to a series of studies funded by the National Science Foundation (NSF) in the 1960s and 1970s to seek alternatives to traditional print journals, which were considered by many to be a huge waste of paper. The basis for this belief was generally twofold. First, many considered citation counts to be the principal indicator of reading articles, and studies showed that articles averaged about 10 to 20 citations to them (a number that has steadily grown over the past 25 years). Counts of citations to articles tend to be highly skewed with a few articles having a large number of citations and many with few or even no citation to them. This led to the perception that articles were read infrequently or simply not at all.

The second reason that many thought articles were not read is based on results of a series of National Science Foundation (NSF) studies done by the American Psychological Association (APA) in the 1960s (American Psychological Association 1965-1968). The first study determined the amount of reading of psychology articles, replicated in similar studies of other fields of science. The amount of reading of an article was observed from a statistical sample of psychologists who were surveyed. The respondents were provided Table-of-Contents of journals published two months prior to the survey and asked whether they had read each article. The APA reported that an average article was read by only 17 psychologists (2) and reading in other fields ranged from about 10 to twenty persons reading them. The problem is that the 17 readings per article (3) represented only the reading by sampled psychologists and not the universe from which the sample was chosen. This fact was not acknowledged in many articles that reported the sample counts of reading of articles over the years (including some referenced above). Furthermore, the reported amount of reading per article did not take into account reading that takes place beyond two months following publication. An in-depth reading of one APA report (Garvey & Griffith 1963) showed that the articles were actually read an average of 520 times when projected to the entire population, although not to reading beyond two months.

The Myth of Low Reading Exposed

The advent of electronic publishing and ability to observe server counts of hits and downloads has essentially demolished the myth that articles and journals are not well-read. While a great deal has been done to standardize metrics of electronic use, these data are suspect in that they again are merely indicators of amount of reading since there is not assurance that a hit or download results in a reading. Also a printed download could be passed on to others, in which case a download may represent several readings. Furthermore, our studies of reading by faculty in nine universities indicate that most reading from personal subscriptions and some reading from library collections remain from print copies.

Another approach to estimating amount of reading (4) per article was done under National Science Foundation (NSF) contract in 1977 (King, et al. 1981), replicated in 1984 and done again in 1995 (Tenopir & King 2000). In 1977 King Research conducted a national survey of scientists and estimated the average reading per scientist (i.e., 105 readings per scientist) and total reading of scientists in the US (i.e., 244 million readings). Note that we generally refer to number of readings rather than articles read because an article can be read more than once by a scientist. Our method of data collection does not make that distinction other than to indicate when a last reading was from an article that had previously been read. The 1977 NSF study also included a tracking survey of journals and articles published in the US. This led to an estimate of the total number of articles published in the U.S. (i.e., 382,000 articles). After adjustments made for age of articles read and so on, the average amount of reading per article was estimated roughly to be 640 readings per article. The average number of persons who read psychology articles in 1977 was estimated to be 685 readings per article compared with 520 persons who read psychology articles observed in the APA study done in 1963. In 1995 we estimated the average reading per science article to be 900 readings using this rough method (5).

Replicating the APA Method of Estimating Reading per Article

The method of sending Table-of-Contents to scientists or other professionals to estimate amount of reading of articles and journals is rarely done. In 1978 King Research conducted a journal cost and use study for John Bailar, editor of The Journal of The National Cancer Institute (JNCI). Here we replicated the APA survey method. With adjustments for amount of reading beyond the time at which the Table-of-Contents was sent to respondents, we estimated average reading to be 1,800 persons per JNCI article ranging from 380 to over 5,000 persons from the least read to the most read article. The survey yielded 521 responses (at a 63% response rate). Every article had been read at least 11 times by the 521 respondents. The total reading of the JNCI was estimated to be almost 750,000 for that year.

In 2004 the University of Tennessee, School of Information Sciences, conducted two surveys for the American Academy of Pediatrics – AAP (Tenopir, et al. 2007). These two surveys involved 2,000 initial samples each. One survey focused on reading patterns of pediatricians (i.e., how many articles were read, age of article last read, how it was identified, from what source was the article obtained, in print or electronic, consequences of reading, etc.). The other survey dealt with the journals read by the pediatricians and various aspects of journals that are read, with emphasis on reading and characteristics of Pediatrics.

The latter survey, conducted in June 2004, is where a portion of the article contents for the March 2004 issue of Pediatrics was presented to the survey respondents. They were asked: "For each of the sample of articles below (from the March 2004 issue of Pediatrics) indicate how much you read". Respondents were given the options of reporting: The Abstract, Part of the Text, Most of the Text, and Did Not Read. A total of 2,000 of the 60,000 AAP members were sampled and 685 responded to the survey (a 34% response rate). Some respondents did not check any of the four options. We treated these item non-responses as though the respondent did not read the article. Since our past definition of reading excluded reading just the abstract, we ignore that response in the results presented below. In order to reduce respondent burden we sampled the articles published in the March 2004 issue of Pediatrics, yielding a total of 19 sampled articles. Eight of the 19 articles were traditional articles; there was one review article, one special article, one commentary and one American Academy of Pediatrics; and seven "electronic page" articles. The following table gives the sample responses and estimated total number of readings for each of the 19 articles when projected to the population of 60,000 pediatricians who were members of AAP.

Table 1. The Number of Sampled Pediatricians Who Read Part of an Article Text, Most of the Text, or Did Not Read, and the Estimated Total Number Who Read an Article for 19 Articles Published in the March 2004 Issue of Pediatrics.

The 19 articles are estimated to have been read about 181,400 times by AAP members in the three months following publication which, when projected to over time, is estimated to be 280,000 times read. Thus, the average number of readings per article is about 9,500 readings per article, which increases to 14,700 readings when subsequent reading beyond three months is projected (6). Some have cautioned that reading should include only instances in which most of the text is read. With this definition of reading, the average amount of reading per article would be about 7,200 readings after projecting beyond three months.

Many pediatricians reported that they did not read the text of the articles, but just read the abstract of the articles. The number of respondents who said they read just the abstract of an article ranged from 42 to 140 (of the 685 total respondents). Thus, a total of 137,300 abstracts were exclusively read from the 19 articles or about 7,200 readings per article added to the estimated 14,700 readings from the text of articles. Even though only an abstract is initially read, the entire article may be read later at a time the information content of an article might be important for diagnosis, treatment, research, teaching or some other purpose.

Considering the total number of articles published in a month (i.e., 24 articles, not the 19 sampled), Pediatrics is estimated to be read an average of 5.9 readings per month per pediatrician based on the Table-of-Content reading. In the same survey of AAP members, the respondents were asked "how many articles do you read in a typical month from each" of 21 mentioned journals, including Pediatrics. The average reading of Pediatrics was 2.9 per month per pediatrician, which compares with 5.9 readings found in the Table-of-Contents question. However, since reading was not defined or clarified in the journal question, it may be that the respondents considered a reading to be one in which most of the text is read. If so, the two survey results would be similar. That is, the average reading of most of the text is 2.9 readings compared with 2.9 readings.

Electronic Format of Article Read

The AAP server reveals that there have been 175,479 downloads of the March 2004 issue of Pediatrics from March 2004 to February 2006. These downloads include 118,753 full-text HTML and 56,726 PDF. A rough extrapolation suggests that the total life-time downloads would be in the range of 250,000 to 300,000. Readers who take the time to download articles are more likely to read most of the text than only part of it. Therefore, assuming that AAP members average reading most of the text of 2.9 articles per month from Pediatrics, they average about 34.8 articles read per year. Thus, the 60,000 members would average a total reading of 2.1 million annual readings from this journal. The second survey of pediatrician reading patterns (Tenopir, et al. 2007) showed that 16 percent of readings were from electronic sources. If this pattern is true for Pediatrics one would expected about 340,000 of the estimated 2.1 million annual readings to be from electronic versions. Both estimates show that there is substantial reading of Pediatrics from electronic sources. However, medical practitioners such as pediatricians continued to rely much more on their personal print issues than electronic versions. Reading from library collections is much more likely to be in electronic format (i.e., 78% of reading from library collections).

Caveats of this Method of Estimating Amount of Reading per Article

One should not extend the above estimates of amount of reading per article to other professional fields or specialties, because many of them will not have the large population of potential readers found with Pediatrics, although some may be even larger. Also, medical professionals tend to average more readings than other professionals (Tenopir et al. 2004, Tenopir & King 2004). On the other hand, our estimate of amount of reading could be low because Pediatrics articles are read by non-members, students, etc., so potential readership is higher.

Another flaw in any survey is the potential bias of the population represented by non-respondents to a survey. We can reasonably assume that the 685 respondents from the 2,000 pediatricians sampled represented a population of 20,500 like pediatricians and the 1,315 non-respondents represent another 39,500 pediatricians whose reading habits are similar to the 1,315 non-responders. The two populations may be different in their information seeking patterns and interest in Pediatrics. Many of the "non-respondent" population may not read Pediatrics and only receive it as an AAP member service, although some may read Pediatrics even more than the average respondent. Even if the "non-respondent" population does not read at all, the absolute minimum average reading per article would be about 5,000 readings by the population represented by those who responded to the survey.

Purpose and Value of Reading

There are few estimates of total reading of articles and journals. Yet amount of reading provides a useful indicator of journal benefits and a means of comparing cost and benefits. Pediatricians' time is one of their most valuable resources and they would not expend an average of 20 minutes of their time reading an article if the information were not of value to them. The value of an article depends on reading its contents and applying the information to useful purposes. The principal purpose and other purposes of reading are summarized in Table 2 below.

Table 2. Proportion of Reading Journal Articles by Pediatricians for the Principal Purpose and Other Purposes (n=626)

The principal purpose for reading articles (i.e., the last article read) is current awareness, but evidence suggests that some of these articles will be read again for other purposes such as treatment, diagnosis, etc. For example, 16 percent of the last reading involved articles that they had read previously. The pediatricians were also asked: "In what ways did the reading of the article affect the principal purpose?" Some said it improved the results (43.5% of readings); inspired new thinking or ideas (42.2%); narrowed, broadened or changed the focus (17.0%); and resulted in other outcomes such as saving time, resolving problems, etc. Only a small proportion said the information was not helpful in meeting the principal purpose of reading (1.2%).

There are many ways to assess cost and benefits of journal articles. One straightforward means is to compare article publishing cost to amount of reading. For example, the average cost of publishing a Pediatrics article is about $9,800 per article. Therefore, the publishing cost is about $0.67 per reading with the most optimistic estimate of reading (14,700 readings per article) and nearly $2 per reading with the most pessimistic estimate (5,000 readings per article). If exclusive readings of abstracts are also included, the optimistic cost per reading is $0.45 and the pessimistic estimate is about $0.80 per reading. If printing costs are excluded, the cost per article is about $4,900 or one-half the total cost. Therefore, the first copy article cost per reading is one-half that observed above. Pediatricians average about $190,000 annual compensation and work an average of 2,330 hours per year (Tenopir & King 2004), therefore they would average paying about $27.20 per reading in their time or much more than the publishing cost per reading at 20 minutes per reading. Thus their determination in what they are willing to pay in their time far outweighs the publishing cost and/or price paid.

Possible Methods

There are several ways the amount of reading of articles can be observed so that surrogate measures can be made. The strengths and weaknesses of these measures are summarized below:

• Survey using Table-of-Contents. This method probably provides the most accurate measures. However, surveys are costly, they may be biased due to non-response, and some reader segments of the population may be missed.
• Electronic "hits and downloads". These measures are relatively inexpensive to observe. However, they may or may not reflect actual reading, they ignore reading from print versions (e.g., about 84% or pediatrician reading is from print), and there is no means of assessing the outcome of reading (Tenopir, et al. 2007).
• Survey of amount of reading divided by number of articles. This rough estimate is also relatively expensive and subject to bias. The comparison is between two independently collected sources of data. Total reading depends on accurate estimates of number of professionals who read and amount of reading per professional. The number of articles published in a year is also subject to error.
• Citation to articles. This method is also relatively inexpensive if the citation data are available and a citation is an indicator of value to research. However, the act of citing represents a very small proportion of readers and of reading. Some older data have suggested that citation counts are correlated with amount of reading, but this assertion should be re-evaluated in light of new methods of article dissemination (King et al. 1981).

Conclusion

Footnotes

(1) Surveys of "scientists" included nine fields of science designated in the U.S. National Science Foundation in the 1970's: Physical sciences, mathematics, computer sciences, environmental sciences, engineering, life sciences, psychology, social sciences, and other sciences.

(2) Note that this count of 17 average number of persons reading an article was about the same as the average number of citations to articles (about 10-20) at that time. Thus, the count of 17 should have been suspect to researchers who relied on this estimate.

(3) We assume that the number 17 represents both the number of persons who read an article as well as the numbers of readings of an article, even though a person may have read an article more than once in the two months following publication.

(4) Since 1977 our reading studies have defined a reading as "going beyond the table- of-contents, title and abstract to the body of the article". Recently we have also defined articles as follows: "Articles include those found in print or electronic journal issues, author websites, or separate copies such as preprints, reprints, and other electronic or paper copies".

(5) Note that Andrew Odlyzko has since recanted his belief that articles are not well read. Perhaps this is particularly due to the fact that his famous 1996 article "Tragic Lost or Good Riddance? The Impending Demise of Traditional Scholarly Journals" has had thousands of hits or downloads on his server alone.

(6) The survey of pediatrician reading patterns asked about the age of the article last read by them. This information was used to extrapolate beyond the proportion of reading the first two or three months following publication. It is estimated that 64.7 percent of reading takes place in that period (mostly for current awareness or in keeping up with the literature).

Bibliography

[American Psychological Association, 1965-1968] Reports of the American Psychological Association's Project on Scientific Information Exchange. Vol. 1 Overview Report and Reports 1-9, 1965; Vol. 2 Reports 10-15, 1966; Reports 16-19, 1968. Washington, D.C.: American Psychological Association.

[Garvey and Griffith, 1963] Garvey, W. D. and Griffith, B. C. (1963). The American Psychological Association's Project on Scientific Information Exchange in Psychology. Report No. 9. Washington, D.C.: American Psychological Association.

[King, et al., 1981] King, D. W., McDonald, D. D., Roderer, N. K. (1981). Scientific Journals in the United States: Their Production, Use and Economics. Stroudsberg, PA: Hutchinson Ross Publishing Co. (Division of Academic Press) dwking@unc.edu

[King and Tenopir, 2001] King, D. W. and Tenopir, C. (2001). "Using and Reading Scholarly Literature." In: Annual Review of Information Science and Technology, Ed. Martha E. Williams. Vol. 34. Medford, N.J.: Information Today, Inc.

[Lancaster, 1978] Lancaster, F. W. (1978). Toward Paperless Information Systems. New York: Academic Press.

[Odlyzko, 1996] Odlyzko, A. M. (1996). Tragic Loss or Good Riddance? The Impending Demise of Scholarly Journals. Int. J. Human Computer Studies. 42.

[Schauder, 1994] Schauder, D. W. (1994). Electronic Publishing of Professional Studies: Attitudes of Academics and Implications for the Scholarly Communication Industry. Journal of the American Society for Information Science. 45(2): 73-100.

[Science and Engineering Indicators, 2006] Science and Engineering Indicators. Washington, DC: National Science Foundation. Annual. <http://www.nsf.gov/statistics/seind06/>.

[Tenopir and King, 2000] Tenopir, C. and King, D.W. (2000). Towards Electronic Journals: Realities for Scientists, Librarians, and Publishers. Washington, D.C.: Special Libraries Association.

[Tenopir, et al., 2007] Tenopir, C., King, D. W., Clarke, M., Na, K., and Zhou, X. (2007). Journal of Reading Patterns and Preferences of Pediatricians. Journal of the Medical Library Association. Scheduled January, 2007.

[Tenopir, et al., 2004] Tenopir, C., King, D. W. and Bush, A. (2004). Medical Faculty's Use of Print and Electronic Journals: Changes Over Time and Comparison with Other Scientists. Journal of the Medical Library Association.

[Tenopir and King 2004] Tenopir, C. and King, D.W. Communication Patterns of Engineers. New York. IEEE Press and John Wiley & Sons. 2004.

[Williams, 1975] Williams, G. R. (1975). Library Subscription Decisions. IEEE Transactions on Professional Communication. 18(3): 207-09.

Top | Contents
Search | Author Index | Title Index | Back Issues
Previous Article | Conference Report
Home | E-mail the Editor

D-Lib Magazine Access Terms and Conditions

doi:10.1045/october2006-king