![[Board
accepted]](../graphics/r2.gif){kind=small}
Free Scholarly Electronic Journals: How Good Are They?
Physics and Earth and Atmospheric Sciences Librarian
Purdue University
fosmire@purdue.edu
Chemical Information Specialist
Purdue University
syu@purdue.edu
Abstract
Amidst the furor of serials price increases, there is a steadily growing segment of the serials landscape with prices that can't be beat, that is to say, free. Many scholarly journals are freely available in electronic format (no subscription fee or membership required), but there has been little assessment of their impact on scholarly research. A fairly comprehensive list of free scholarly electronic journals in the science, technology, and medical fields was compiled and was examined using citation analyses. The results indicate that, unlike the situation five years ago (Harter 1998), there are several free scholarly electronic journals that have a significant impact on their respective fields.
Introduction
Scholarly journals available in electronic format are fast becoming the rule and not the exception. Roes (1994) found 39 peer-reviewed electronic journals, and two years later, Harter and Kim (1996a) and Hitchcock, Carr and Hall (1996) found two to three times that number. Shortly thereafter mainstream society and commercial publishers started offering online versions of their print journals. Consequently, the Association of Research Libraries' Directory of Electronic Journals, Newsletters, and Academic Discussion Lists, 7th Edition claimed 1,002 peer-reviewed e-journals (Mogge 1997). Now, the vast majority of scientific publishers offer electronic versions of their journals.In their early development, there had been a tradition of creating/offering free access to electronic journals, perhaps taking advantage of the lower production costs of electronic distribution. Hitchcock, Carr, and Hall (1996) found 47 out of 83 STM journals were free and planned to stay that way, and, looking across all disciplines, Harter and Kim (1996b) found 90% of e-journals in their study were free. Currently (Fosmire and Young, In press) only 18% of e-journals are free, although the absolute number of titles is still increasing (from 67 in the Harter and Kim study, to 213 in the Fosmire and Young study).
Despite their increasing numbers, free electronic journals did not appear to be receiving the attention they deserved from the library community (measured in terms of inclusion in catalogs and on library web sites), commensurate with their inclusion in major abstracting and indexing services (Fosmire and Young, In press). However, inclusion by an abstracting and indexing service is only a crude measure of quality, and the authors decided to investigate other indicators of a journal's quality and influence within its discipline.
Harter (1998) carried out an impact study on electronic journals that started publishing before 1994. Of the 38 journals in the sample, 15 had no citations to them in ISI's citation databases, while 8 had ten or more. Those eight titles exhibited high impact factors for their fields (>70th percentile), although low total numbers of articles published (typically 15th percentile and below). Since the pool of free scholarly electronic journals has increased considerably since 1994, and potentially the attitudes and publishing habits of researchers have changed as well, the authors felt it was worth re-investigating the importance of free scholarly electronic journals. (N.B., Harter's study was not limited to free journals, although six of the eight journals were freely available.)
One of the standard methods of indicating the importance of a journal is through its impact factor. The impact factor is an unobtrusive and objective measure of the effect an article has on researchers. While much has been written about motivation for citations (e.g., Brooks 1986; Gilbert 1977; Small 1978) and suggestions have been made for improving the measure (Harter and Nisonger 1997; Moed and Van Leeuwen 1995; Nisonger 1994), no other quantity has gained widespread acceptance as a replacement. To provide a more complete picture of the effect of a journal on scholarship, other measures that appear in ISI's Journal Citation Reports, the immediacy index and current articles quantities, were also analyzed.
Methods
First, in carrying out this study, the term 'free scholarly electronic journal' needed to be defined, and a pool of free scholarly electronic journals assembled. For the purposes of this paper, 'free' refers to a journal, the access to which is not dependent on a subscription or membership in an organization. Thus, for example, 'free, with print subscription' titles do not meet this criterion. 'Scholarly' is defined to be peer-reviewed and with a scholarly treatment. Functionally, a journal is not considered to have a scholarly treatment unless its articles contain references. 'Electronic journal' refers not only to electronic-only journals, but also to journals with print counterparts. The restriction made for a journal to be considered electronic is that the electronic version of the journal contains at least as much content as the print counterpart and is published approximately simultaneously with the print version, if not sooner.Journals investigated in this study were selected from The Directory of Electronic Journals, Newsletters, and Academic Discussion Lists, 7th Edition, using the procedure described by Fosmire and Young (In press), and further limiting to just those titles in science, technology, and medicine (STM) fields. This yielded a final list of 84 titles. Once the titles had been selected, they were searched on the Institute for Scientific Information's (ISI) Web of Science database to find the number of citations to their articles. As the Web of Science searches citations based on a 20 character code, and their internal consistency is not complete, each title was searched using a three letter truncation of each word in the title (omitting stop words like 'of', 'the', etc.). Thus, for example, Emerging Infectious Diseases was searched as 'eme* inf* dis*'. In this way the authors attempted to capture as many citations as possible. The results lists were inspected to discard citations to other journals with similarly abbreviated titles.
The vast majority of free scholarly electronic journals are not indexed in Journal Citation Reports (JCR), so impact quantities for these journals were computed by the authors based on citations appearing in the Web of Science. The total number of articles published was determined by inspecting the journal's web site. In order to report the most current data, quantities for 1999 were computed, as follows:
Impact Factor = Number of citations received in 1999 to 1997 and 1998 articles/Number of articles published in 1997 and 1998.
Immediacy Index = Number of citations received in 1999 to 1999 articles/Number of articles published in 1999.
Current articles = Number of articles published in 1999.
The exception to the above method is for the British Medical Journal. Due to its large number of articles and citations, 1998 Journal Citation Reports data was used instead of 1999 Web of Science data. Also, the total number of 1997 and 1998 journal articles was not available for the International Journal of Mathematics and Mathematical Sciences on its web site, so no impact factor was computed for that title.
Generally speaking, "the impact factor measures the current impact of the typical recently published article" in a particular journal, as Harter and Nisonger (1997) so eloquently state. The immediacy index is an indication of how quickly readers incorporate articles from a journal into their research, and the number of articles published gives an indication of the total impact (multiplied by impact factor) of the journal on the field.
Since the 1999 version of Journal Citation Reports was not available for comparison (and will not be available until 2001), the authors compare the computed 1999 data to the 1998 JCR data. While this may be perceived as comparing apples and oranges, the purpose of this study is merely to get an indication of the general quality of different free scholarly electronic journals, not to rank the journals absolutely with their non-free peers. By computing quantities from 1999 data, the authors were able to include several titles that would not otherwise have been possible with 1998 data, including several strong new journals.
Since journal citation rates differ between different subject areas, comparisons of journal titles should be amongst those in the same subject area. Seven major subject areas were identified that representated all the free scholarly electronic journals. All the relevant categories in JCR were then grouped into those seven major subject areas, and the impact factor of the top journal, the 90th percentile journal, and the 75th, 50th, and 25th percentile journals were tabulated. The immediacy index and current articles quantities for the same percentiles were also tabulated.
Discussion
Of the 82 titles in the sample, 47 had received at least one citation in the Web of Science during 1999. Results for those 47 titles are given in Tables I-VII, organized by subject area (Agriculture, Biological Sciences, Computer Science, Engineering, Mathematics, Medicine, and Physics). Of the 47 journals, 34 are electronic-only start-up journals with no print-based past. A list of the 14 journals that started out in a print format are given in the appendix. Generally speaking, impact quantities do not differ significantly between the electronic-only journals and those with a print history, other than that the higher impact medical journals in our sample (Table VI) are dominated by the print-based titles.
Impact Factor
Thirteen of the journals that had citations in JCR (28%) rated above the 90th percentile of journals in their field, including Cybermetrics and Advances in Theoretical and Mathematical Physics (ATMP), which had higher rankings than the top journal in their subject area in Journal Citation Reports. Cybermetrics is a bit of an artifact, as it only published one article during 1997-1998, but ATMP is a bona fide high impact journal. It boasts an article with over 800 citations and another with over 500 citations. Certainly, high-energy and particle physics theory papers have high citation numbers within the field of physics, so ATMP and the Journal of High Energy Physics (JHEP) would be expected to have high impact factors (and JHEP ranks in the top 10%). However, ATMP excels even among high energy physics titles, as it had articles that were cited 2nd, 3rd, and 30th most often in 1999, among all citations in the HEP-SPIRES database to articles from any year (O'Connell 2000). The Engineering and Medical journals also had a high percentage of high-impact journals within their disciplines, while the Agriculture journals did not score well at all. Overall, the free scholarly electronic journals are competitive in their quality of articles with their non-free peers, although the overall impact factors are skewed to the top and bottom. 20 journals (43%) were above the 75th percentile, while 38% were below the 25th percentile.
Immediacy Index
In regard to the immediacy index, 13 (28%) of the journals rated above the 90th percentile, and 34% ranked in the above 75th percentile, while 47% were in the below the 25th percentile. These results that are similar to the impact factor rankings, although slightly lower. Again, the physics journals scored very well, with only one title below the 75th percentile. One might think that electronic journals would have high immediacy indexes relative to their print-based peers, based on a faster time to print. However, two factors negate this effect. First, many journals, especially in the STM disciplines, have moved to electronic publishing, so this pool of electronic journals does not comprise a significantly different publishing milieu than the non-free publications. Second, the time-dependent aspect of citations is how fast the citing journal is published. Since the vast majority of the journals investigated here are not indexed in the Web of Science, these journals are dependent on other journals to provide indications of quality. If the journals indexed in the Web of Science have a backlog for publication of more than one year, then there is no way to get an 'immediate' citation from that journal. Since the citation data are coming from the same pool of journals, electronic publishing of the original article does not, by itself, indicate a higher immediacy index.
Current Articles
The number of current articles is important if one is to gauge the total impact of a journal on a field of study. For example, if a journal has an impact factor of 6, but only publishes one article, is that a better journal than one that publishes 1000 articles and only has an impact factor of 2? If one had to choose between the journals, everything else being equal, obviously the one that garnered 2000 citations is more useful than the one that garnered 6. However, the total number of citations garnered does not tell the whole story either, as one does not want a lot of 'noise', (i.e., low-quality articles) to sort through on the off-chance of unearthing a gem in a journal. Harter (1998) found that the journals he studied scored very poorly with respect to current articles, ranking in the bottom 15% in their field. Given that history, the current results are very encouraging. Four percent of journals ranked above the 90th percentile of their field, and 23% are in the top half of journals in their field. While several of these productive journals are electronic versions of print journals (for example, Southern Medical Journal) or have migrated from successful print journals (e.g., Emerging Infectious Diseases), there are several productive electronic-only startup journals, such as Radiographics, JHEP, and the Materials Research Society Internet Journal of Nitride Semiconductor Research.Of course, impact factors are not the definitive answer of the importance of a journal in its field. Impact quantities are computed based on a pool of journals that are indexed by the Web of Science. Thus, journals in subject areas not well covered by this pool (for example, education-related STM journals like Journal of Extension and Compute-Ed) will have impact factors lower than their true influence on their field. Also, journals indexed by the Web of Science have the advantage of self-citations, whereas the bulk of the journals investigated here do not. For niche journals that cover a narrow subject area in great depth, if no counterpart is indexed by the Web of Science, the lack of self-citations can seriously underestimate the importance of the journal to researchers in its field.
Conclusion
Overall, it appears, that several high-quality, productive free scholarly electronic journals exist currently. These journals scored very well in impact factor and immediacy index, and they have reasonable numbers of articles published. The reasons for these journals' success are probably as unique as the journals themselves. Some are niche journals (MRS Internet Journal of Nitride Semiconductor Research), others are essentially government documents (Journal of Research of the National Institute of Standards and Technology, Emerging Infectious Diseases), and others are journals in mainstream topics, but with a devoted following of authors and readers that want low-cost, freely accessible information (ATMP, JHEP). Whatever the reason, these journals are proving that an alternative to the current 'serials crisis' of spiraling price increases is possible.
Appendix I
The following journals are the titles in our sample that started out as a print journal, rather than as an electronic journal.
- Agriculture:
- Journal of Extension
- Florida Entomologist
- Biology:
- Fungal Genetics Newsletter
- Computer Science:
- IBM Journal of Research and Development
- IBM Systems Journal
- Engineering:
- Journal of Research of the NIST
- Mathematics:
- Bulletin of the American Mathematical Society
- Journal de Theorie des Nombres Bordeaux
- Journal of Lie Theory
- Contributions to Algebra and Geometry
- Journal de Theorie des Nombres Bordeaux
- Medicine:
- Emerging Infectious Diseases
- British Medical Journal
- Southern Medical Journal
- Annals of Saudi Medicine
- British Medical Journal
Appendix II
- Agriculture:
- Florida Entomologist - http://www.fcla.edu/FlaEnt/
- Journal of Extension - http://joe.org/
- Biology:
- Conservation Ecology - http://www.consecol.org/Journal/
- Frontiers in Bioscience - http://www.bioscience.org/
- Fungal Genetics Newsletter - http://www.fgsc.net/newslet.html
- Molecular Vision - http://www.molvis.org/molvis/
- Frontiers in Bioscience - http://www.bioscience.org/
- Computer Science:
- Compute-Ed -
http://computed.coe.wayne.edu/index.html
- Cybermetrics - http://www.cindoc.csic.es/cybermetrics/cybermetrics.html
- IBM Journal of Research and Development - http://www.research.ibm.com/journal// [Note: Link moved; URL changed 11/12/00 by ald]
- IBM Systems Journal - http://www.research.ibm.com/journal/ [Note: Link moved; URL changed 11/12/00 by ald]
- Journal of Artificial Intelligence Research - http://www.cs.washington.edu/research/jair/home.html
- Journal of Memetics - http://www.cpm.mmu.ac.uk/jom-emit/
- Neural Computing Surveys - http://www.icsi.berkeley.edu/~jagota/NCS/
- Cybermetrics - http://www.cindoc.csic.es/cybermetrics/cybermetrics.html
- Engineering:
- Journal of Research of the National Institute of Standards and
Technology -
http://nvl.nist.gov/pub/nistpubs/jres/jres.htm
- MRS Internet Journal of Nitride Semiconductor Research - http://nsr.mij.mrs.org/
- Mathematics:
- Beiträge zur Algebra und Geometrie (Contributions to Algebra and
Geometry)
-
http://www.emis.de/journals/BAG/index.html
- Bulletin (New Series) of the American Mathematical Society - http://www.ams.org/bull/
- Documenta Mathematica - http://www.mathematik.uni-bielefeld.de/documenta/
- Electronic Journal of Combinatorics - http://www.combinatorics.org/ejc-wce.html
- Electronic Journal of Differential Equations - http://ejde.math.swt.edu/
- Electronic Journal of Linear Algebra - http://www.math.technion.ac.il/iic/ela/
- Electronic Journal of Probability - http://math.washington.edu/~ejpecp/
- Electronic Research Announcements of the American Mathematical Society - http://www.ams.org/era/
- Geometry and topology - http://www.maths.warwick.ac.uk/gt/
- Journal de Théorie des Nombres de Bordeaux - http://www.emis.de/journals/JTNB/index.html
- Journal of Graph Algorithms and Applications (JGAA) - http://www.emis.de/journals/JGAA/home.html
- Journal of Lie Theory - http://www.emis.de/journals/JLT/index.html
- New York Journal of Mathematics (NYJM) - http://nyjm.albany.edu:8000/nyjm.html
- Séminaire Lotharingien de Combinatoire - http://www.emis.de/journals/SLC/index.html
- Southwest Journal of Pure and Applied Mathematics - http://rattler.cameron.edu/swjpam/swjpam.html
- Theory and Applications of Categories - http://www.tac.mta.ca/tac/
- Bulletin (New Series) of the American Mathematical Society - http://www.ams.org/bull/
- Medicine:
- Alzheimer's Disease Review - http://www.coa.uky.edu/ADReview/
- Annals of Saudi Medicine - http://www.kfshrc.edu.sa/annals/
- British Medical Journal - http://www.bmj.com/bmj/
- Dermatology Online Journal - http://matrix.ucdavis.edu
- Emerging Infectious Diseases - http://www.cdc.gov/ncidod/eid/index.htm
- Eurosurveillance - http://www.ceses.org/eurosurv/
- Medicine & Global Survival - http://www.healthnet.org/MGS/
- Online Journal of Issues in Nursing - http://www.nursingworld.org/ojin/
- RadioGraphics - http://ej.rsna.org/
- Southern Medical Journal - http://www.sma.org/smj/index.htm
- Annals of Saudi Medicine - http://www.kfshrc.edu.sa/annals/
- Physics:
- Advances in Theoretical and Mathematical Physics -
http://www.intlpress.com/journals/ATMP/
- Journal of High Energy Physics - http://jhep.mse.jhu.edu/
- Living Reviews in Relativity - http://www.livingreviews.org/
- Mathematical Physics Electronic Journal - http://www.ma.utexas.edu/mpej/MPEJ.html
- New Journal of Physics - http://www.njp.org/
- Optics Express - http://epubs.osa.org/opticsexpress/
- Journal of High Energy Physics - http://jhep.mse.jhu.edu/
References
Brooks, T. A. 1986. "Evidence of complex citer motivations." Journal of the American Society for Information Science 37(1):34-36.
Fosmire and Young. In press. "Free scholarly
electronic journals: what access do college and university libraries
provide?" College & Research Libraries.
Gilbert, G. N. 1977. "Referencing as
persuasion." Social Studies of Science. 7:113-122.
Harter, Stephen P. 1998. "Scholarly
communication and electronic journals: an impact study." Journal of
the American Society for Information Science, 49.6 (1998): 507-516.
Harter, Stephen P. and Kim, Hak Joon.
1996a. "Electronic journals and scholarly communication: a citation and
reference study." Proceedings of the Midyear Meeting of the American
Society for Information Science, San Diego, CA, May 20-22, 1996:
299-315. [Online]. Available
http://php.indiana.edu/~harter/harter-asis96midyear.html. [August 8,
2000].
________. 1996b.
"Accessing electronic journals and other e-publications: an empirical
study." College and Research Libraries 57 (September 1996):
440-456.
Harter, Stephen P. and Nisonger, Thomas E.
1997. "ISI's impact factor as misnomer: a proposed new measure to assess
journal impact." Journal of the American Society for Information
Science 48(12):1146-1148.
Hitchcock, Steve, Carr, Leslie, and Hall,
Wendy. 1996. A Survey of STM Online Journals 1990-95:
The Calm Before the Storm. [Online] Available:
http://journals.ecs.soton.ac.uk/survey/survey.html#scope [August 8,
2000].
O'Connell, Heath. 2000. Most Cited
High-Energy Physics Articles 1999 Edition. [Online] Available:
http://www.slac.stanford.edu/library/topcites/top40.1999.html.
[August 8, 2000].
Moed, H. F. and Van Leeuwen, Th. N. 1995.
"Improving accuracy of Institute for Scientific Information's journal
impact factors." Journal of the American Society for Information
Science 46(6): 461-467.
Mogge, Dru W., Ed. 1997.
Directory of Electronic Journals, Newsletters and Academic
Discussion Lists Foreword. [Online]. Available:
http://db.arl.org/foreword.html. [August 8, 2000].
Nisonger, Thomas E. 1994. "A
methodological issue concerning the use of Social Sciences Citation Index
Journal Citation Reports impact factor data for journal ranking."
Library Acquisitions: Practice and Theory, 18(4): 447-458.
Roes, Hans. 1994. "Electronic journals:
a survey of the literature and the net." Journal of Information
Networking 2.3: 169-186. [Online]. Available:
http://cwis.kub.nl/~dbi/users/roes/articles/ej_join.htm [August 8,
2000].
Small, Henry G. 1978. "Cited documents
as concept symbols." Social Studies of Science 8: 327-340.
Table I
Comparisons of the 1999 Impact Factor, Immediacy Index,
and Number of Current Articles of Agriculture Free Scholarly Electronic
Journals with the 99th, 90th, 75th, 50th, and 25th percentile Agriculture
journals in ISI's Journal Citation Reports 1998.
Impact Factor
Immediacy Index
Current Articles
99th
19.00
99th
1.82
99th
986
90th
1.56
90th
0.29
90th
184
75th
1.00
75th
0.19
75th
101
50th
0.58
50th
0.10
50th
53
25th
0.31
25th
0.04
Journal of Extension
52
Journal of Extension
0.02
Journal of Extension
0
Florida Entomologist
44
Florida Entomologist
0.01
Florida Entomologist
0
25th
29
