D-Lib Magazine
|
|
Lynn Silipigni Connaway Stephen R. Lawrence |
AbstractThis exploratory study asked eleven Association of Research Libraries (ARL) librarians to identify the resources needed for the transition of an all-paper library to the all-digital library. Although the results cannot be generalized, the study identifies functions and resources associated with the provision of paper and electronic materials that can be used for further investigation of library resource allocations. IntroductionSince electronic materials are becoming more prevalent in libraries, there has been discussion and interest in identifying and comparing the costs of providing print and electronic materials [California State University, 2002; Lawrence et al.; Mackie-Mason et al.; Montgomery and King; and Sanett]. With the current world economy, budget restrictions, and librarians' current challenge of planning and integrating the digital library [Carlson], the study of resource allocation costs is a timely and practical endeavor. An exploratory study of the resources needed for the selection, acquisition, organization, and dissemination of paper materials and electronic materials was done in Boulder, CO, in May 2002. Eleven Association of Research Libraries (ARL) librarians were selected to identify the resources required for providing an all-paper library and an all-digital library and to complete an interactive electronic spreadsheet [Survey]. The digital library or the library of the future may have materialized sooner than projected and reflected in today's libraries. There have been numerous digital library projects and initiatives developed by libraries and educational institutions. These projects may have been sparked by the acceptance of and reliance on electronic journals, the wide acceptance of the Internet for scholarly communication, and the need for access to materials from locations outside of the traditional library setting. A recent article in the Chronicle of Higher Education identifies major university library building renovation projects [Carlson]. The author of this article highlights university libraries that are devoting space to special collections, multimedia stations and instruction rooms, information commons, group study rooms, and computer labs. At the same time, these libraries are installing compact shelving and are moving lesser-used and older collections to remote storage locations. In regard to these initiatives and the growing interest in electronic materials and communication, there have been several recent studies addressing the use and costs of electronic journals. The introduction of electronic journals and databases to libraries began in the 1980s; therefore, cost data are available for the comparison of print bound and unbound journals and electronic journals (ejournals). The Institute of Museum and Library Services (IMLS) funded a project "...to evaluate the economic implications of converting the current journal collection of a university library to an all-digital format" [Montgomery]". This study documented the costs of acquiring and maintaining print bound and unbound journals and electronic journals using overhead and fixed costs of Drexel University's W. W. Hagerty Library [Montgomery and King]. Since electronic books (eBooks) have only been introduced to libraries within the past four years, there is little available cost and usage data. Consequently, there have been very few studies that include the costs or resource allocations associated with the inclusion of eBooks in the collection. The research reported in this article is an attempt to identify resource allocations for an all-paper library and the all-digital library that provides ejournals, eBooks, electronic maps, and digital audio, video, and images, based on information gathered from a group of ARL librarians. The results of this study provide a foundation for future digital library costing studiesonly with follow-on comparative research can the results reported here be generalized. Literature ReviewResearch examining library costs has been ongoing for some decades. Rapidly evolving information technologies and media options provide substantial motivation for librarians to understand library cost structures so as to make informed decisions about acquiring and utilizing new technologies. Martin M. Cummings notes that "Analysis of costs in meaningful detail is essential to management, i.e., answering the question, 'Are there alternate, less costly ways to do the same thing?'" [Cummings]. Richard E. Quandt provides an overview of the determinants of journal pricing and the issues relating to pricing that includes an extensive literature review of the topic [Quant]. Lawrence, Connaway, and Brigham provide a recent review of the library costing literature [Lawrence et al.], so this review is restricted to research involving digital libraries that directly relates to the current article. For those interested, Stephen A. Roberts provides a good general discussion of library costing issues [Roberts]. The California State University Library System recently participated in an eBook pilot project that studied several different methods for acquiring and accessing eBooks [California State University, 2003]. The study reported usage statistics and patron satisfaction, and investigated the economics of eBooks. A cost model for the acquisition of each type of eBook and paper book (pBook) purchased by the libraries was developed. These costs include the purchase price of the books and the cost of processing and cataloging the books. A comparison of the break-even points for pBooks and eBooks was also included. The study concluded that eBooks are most economical for materials that receive high use over a short period of time, but did not estimate resource allocations required for the selection, acquisition, cataloging, maintenance, circulation, storage, and deselection of pBooks and eBooks. Mackie-Mason, et al. examined the different pricing and bundling models of electronic resources in terms of usage and publisher revenues [Mackie-Mason et al.]. The authors compared their data with "those established for the bundling of information goods in the economic literature." Their research did not examine the costs associated with the selection, acquisition, organization, and dissemination of electronic resources. As previously mentioned, Montgomery and King documented costs for space, systems, supplies and services, and staff by function for print bound and unbound journals and for ejournals [Montgomery and King]. These costs were calculated for a twenty-two week period at Drexel University's W. W. Hagerty Library. Montgomery, Dean of the W. W. Hagerty Library, had access to all of these costs and was able to initiate a study that required staff to document their time during a specified time frame. The results of the Drexel University library study support the findings of this exploratory study of the resource allocations associated with the print library and the digital library. Shelby Sanett proposed the development of a cost model for preserving authentic electronic records [Sanett]. Using existing digital preservation processes and cost models, Sanett identified the elements that should be included in a cost model for preserving electronic records. Her proposed cost model includes capital costs, direct operating costs, and indirect operating costs (overhead) for both the preservation and use of electronic records. Although Sanett's terminology varies from that of this article, her cost elements are the same as those identified in this study, including labor, space, materials, and equipment required to select, acquire, catalog, maintain, circulate, store, and deselect materials. Basic AssumptionsThis study was based on several assumptions developed during discussions with practicing librarians during a three-year period and on librarians' responses and reactions to two pre-tests conducted prior to the distribution of the formal cost allocation spreadsheet to the study group. The first pre-test conducted with catalog, collection development, acquisition, and reference librarians provided sufficient feedback to warrant a major revision of the cost allocation interactive spreadsheet. The pre-test of the revised cost allocation instrument demonstrated that the interactive spreadsheet accurately portrayed the functions associated with providing print materials and electronic materials, but also indicated the need for discussion before and after the completion of the cost allocations. It was then determined that it would be most beneficial to distribute the cost allocation interactive spreadsheet to a group of librarians who were together in the same location. The assumptions are:
MethodologyTo gather current thinking and experience regarding the potential benefits and costs associated with the paper and the digital library, a study group of librarians was convened. The group was comprised of eleven ARL librarians who have experience with the selection, acquisition, organization, circulation, maintenance, storing, and deselecting of both paper and electronic materials. There were ten academic librarians and one public librarian who represented different types of library services including two access services librarians (one public and one academic librarian), two collection development librarians, two reference librarians, two technical services librarians, one acquisitions librarian, one catalog librarian, and one electronic resources librarian. The group met on the campus of the University of Colorado at Boulder in early May 2002. The librarians were asked to consider two hypothetical types of libraries:
While no library of the recent past, of the present, or of the foreseeable future will match either of these hypothetical libraries [Sanett], considering these extremes helped focus the participants so that they could better evaluate anticipated differences in the resource requirements for paper materials versus electronic materials.
Participants were instructed that both hypothetical libraries were presumed to require the same set of operational functions needed to acquire and maintain its collection of paper materials or electronic materials, as shown in Figure 1. Principal functions include selection, acquisition, cataloging, maintenance, circulation, warehousing and storage, and deselection. Clearly, the resources required to execute these functions will vary significantly between a paper and a digital library. Each of these functions was subdivided into constituent subfunctions as shown in Tables 1a-b, which will be discussed later. Resources consumed by library functions were categorized into four major classes or categories; labor, space, materials, and equipment. LaborLabor allocations include professional, paraprofessional, and technical staff. Participants were instructed to evaluate labor using a metric of total labor hours consumed, not monetary expenditure for labor. SpaceSpace allocations include workspace, private space, and public space. Participants were instructed to evaluate space using a metric of square feet of floor space consumed. MaterialsParticipants were instructed not to include the purchase price of print materials or electronic materials in their comparisons, but only to consider the monetary costs of ancillary processing, maintenance, and security materials. The reason for this instruction was two-fold. First, the business model for selling and distributing electronic materials is rapidly evolving, so credible cost comparisons between print materials and electronic materials are difficult to estimate. Second, the purpose of this study is primarily to compare the life-cycle costs of ownership between print materials and electronic materials. The previous work of Lawrence, Connaway, and Brigham indicates that the purchase costs of pBooks are a relatively small fraction of the life-cycle costs of ownership [Lawrence et al.]. Montgomery and King report that the largest total expenditure for electronic journals is for publishers' packages, but "...the largest cost per title is for electronic journals that are subscribed to individually" [Montgomery and King]. The researchers also report that bound journals are the highest cost per use, although they do not represent the highest purchase cost per title. These findings indicate that the need for accurate purchase cost comparisons between paper materials and electronic materials are not critical to cost allocations since the costs of selecting, acquiring, organizing, maintaining, circulating, storing, and deselecting represent the highest costs associated with providing print and electronic materials. An implicit assumption of the study is that the total purchase cost of print materials and electronic materials are essentially equivalent, whatever the business model (single title or article purchase or licensing, per use fee, bundling, publisher packages, etc.). EquipmentEquipment resources included equipment for both library staff and patrons. Examples of equipment include bookshelves, tables, chairs, carts, network connectivity equipment, terminals, printers, and so forth. Participants were instructed not to include equipment necessary for digitizing materials, or for servers to store digitized materials. The reason for this instruction is that business models for in-print, copyrighted electronic materials are evolving toward one where third-party vendors undertake the digitization, storage, and network distribution of the materials, and where libraries are responsible for the network connectivity costs and for providing terminals, computers, printers, and other reading devices in the library itself. Participants were instructed to evaluate and compare equipment resources based on the monetary value of the equipment. Paper Library EstimatesGiven these preliminary instructions, participants were instructed to consider how resources would be allocated among the six principal functions of a paper library. For example, participants were asked to estimate the percentage of library operating space that would be allocated to selection, acquisition, cataloging, maintenance, circulation, warehousing and storage, and deselection. Participants were next instructed to consider the subfunctions within each major function and to allocate resources among the subfunctions. For example, participants assigned resources to the subfunctions authority control, catalog, classify, and maintain database that comprise the principal function of cataloging. The aggregate allocations of resources for all library subfunctions were subsequently determined by multiplying the overall allocation of each major function by the allocation of resources to each of its subfunctions. For example, if 20% of equipment resources were allocated to the cataloging function, and within cataloging, 25% of cataloging equipment was assigned to the classify subfunction, then 5% (20% of 25%) of all library equipment resources were assigned to classify. Results from this analysis are summarized in Table 1a and Figures 2a - 2d. These results provide estimated resource allocations for operational functions and subfunctions of a paper library. Digital Library EstimatesIn the second half of the cost allocation exercise, study participants were instructed to estimate the resources that would be required by a digital library as a percentage of the same resource required in a paper library. For example, consider the authority control subfunction of the cataloging function. In a digital library, authority control could require more labor (gt;100%), the same amount of labor (=100%), or less labor (<100%) than authority control in a paper library. For each of the subfunctions of the digital library, participants made percentage estimates of the relative amount of resources required as summarized in Table 1b and Figures 3a - 3d. FindingsThe aggregated results of the participants' cost allocations are summarized in Tables1a and 1b, and in Figures 2-4. Table 1a. Estimated mean resource use in a Paper Library, by operational function
Table 1b.Estimated mean resource use in a Digital Library relative to a Paper Library, by operational function
Figures 2a, 2b, 2c, and 2d show the estimated allocation of resources within a traditional paper library.
Figures 3a, 3b, 3c, and 3d show the estimated resource consumption of an all-digital library compared to a traditional paper library, by function.
Paper LibrariesFigures 2a - 2d and Table 1a show the estimated allocation of resources in a paper library. Figures 2a - 2d show the range of resource use estimates (minimum to maximum) as thin lines, while the solid bars represent the limits of the first and third response quartiles50% of responses fell within this band. Table 1a shows the mean estimates for all functions and subfunctions of the paper library. These data provide information for estimating the resource requirements of an all-digital library. Paper Library LaborOn average, the results indicate that cataloging and circulation consume almost 50% of labor resources in the paper library, followed closely by selection and acquisition. Participants estimate that these four functions consume about 80% of labor resources in a paper library. Paper Library SpaceNot surprisingly, circulation and warehousing are estimated to be the dominant consumers of space, accounting for almost 60% of the space requirements in a paper library. Paper Library MaterialsMaterial requirements are estimated to be more evenly distributed among library functions, with cataloging, maintenance, and circulation consuming about 64% of material resources in a paper library. Paper Library EquipmentAcquisition, cataloging, and circulation are estimated to be the dominant consumers of equipment in a paper library, accounting for about 61% of equipment requirements, followed by maintenance and warehousing. Digital LibrariesFigures 3a - 3d and Table 1b summarize the participants' comparison of resource requirements of an all-digital library to an all-paper library. As with Figures 2a - 2d, Figures 3a - 3d illustrate maximum and minimum responses as well as the first and third response quartiles. Table 1b shows mean estimates for digital library resource consumption for library subfunctions and as well as for their parent functions. Digital Library LaborOn average, participants estimated that labor requirements generally will be reduced in a digital library compared to a paper library. The largest labor savings are anticipated in circulation and warehousing, followed by maintenance and deselection. The smallest labor savings are anticipated for selection and cataloging, and in fact, several participants forecast increases in the labor requirements for these functions. Also of interest is the large dispersion (0 to 145%) of estimates for cataloging labor in a digital library, indicating that the role of the cataloging function within an all-digital library is highly uncertain. Digital Library SpaceParticipants estimated that space requirements in a digital library will be less than those in a paper library for all library functions. Space savings, not surprisingly, will be largest for the warehousing function, followed by deselection, circulation, and maintenance. Most participants forecast substantial space savings for selection, acquisition, and cataloging; however, several participants forecast no savings for selection, acquisition, and cataloging in a digital library. Digital Library MaterialsOn average, participants estimated that materials requirements in a digital library will be substantially lower than for a paper library. The greatest savings are forecast to occur in maintenance, circulation, warehousing, and deselection. In contrast, one participant forecast a significant (254%) increase in the materials requirements for selection and a modest increase (126%) in materials for cataloging. Digital EquipmentEstimates of equipment requirements in a digital library provide the greatest degree of variation and disagreement among participants. On average, participants expect equipment savings in all library functions except for the selection function. Small savings in equipment resources also are forecast for acquisition and cataloging. The largest equipment savings are anticipated for warehousing, circulation, and maintenance.
Vertical lines represent the maximum and minimum estimates; bars represent first and third quartiles and include 50% of all estimates.
Figure 4 shows weighted average resource use for each of the principal resource categories. These weighted averages were obtained by multiplying the relative use of a resource in each subfunction of a paper library by the forecast relative use of that resource in the same subfunction of a digital library. For example, if the jobber list maintenance subfunction is estimated to require 2.1% of all labor in a paper library, and a digital library is forecast to require 173.6% more labor for this subfunction, then the equivalent labor use in a digital library (as a percentage of labor in a paper library) would be 3.6% (2.1% of 173.6%). By summing the labor percentages in each subfunction category, function subtotals, and digital library totals can be obtained for each resource category. Figure 4 and Table 1b report these results. The assumption that an all-digital library will require less labor is supported by the forecasts of the participants. All participants estimated that aggregate labor requirements will be less for an all-digital library, with estimate ranges from a low of 39%, and high of 81%, and a mean forecast of 59%. The assumption that an all-digital library will require less space was strongly supported by the participants' estimates. As with labor, all participants estimated that aggregate space requirements would be smaller for an all-digital library. The mean of the estimates is 29% with a high estimate of 65% and a low of 6.5%. The assumption that material resources will be smaller in an all-digital library is also strongly supported by the results of the participant forecasts. The mean estimate for materials use was 34% for a digital library compared to a paper library. The minimum forecast was 0.2%, and the maximum 78.2%. In contrast with the first assumptions, the fourth assumption is that an all-digital library will require the same amount of equipment as does an all-paper library. The results of the study group suggest that aggregate equipment use in a digital library will be less than for a paper library: the mean forecast for equipment use in a digital library is 70% with a high of 119% and low of 26%. However, the large range between maximum and minimum (93%) suggests significant uncertainty among participants in eventual equipment requirements in an all-digital library. Given this uncertainty, it is difficult to forecast equipment requirements for an all-digital library with any assurance, except to say that equipment requirements will probably not increase significantly. DiscussionThis study used librarians, who are experienced in the functions required to provide both print materials and electronic materials, to estimate future resource requirements in a hypothetical all-digital library compared to an all-paper library. This contrived dichotomy served well to focus analysis and discussion on the most salient differences in resource consumption between paper and digital libraries, and by extension, between print materials and electronic materials. The transition from paper to digital media is far more complex than this dichotomy suggests, as reflected by the comments of participants both during the study group discussions and in a subsequent debriefing. In this section we report on some of the concerns and questions raised by participants as they completed the cost allocation exercise. LaborSeveral participants expressed concern that while overall staffing levels may decline; the knowledge level and technical skills needed by employees in a digital library may need to be significantly greater than in a paper library. This could cause the overall expense of labor to increase, even if total labor hours decline. For example, although cataloging records may be distributed with the electronic content, concern was expressed that libraries will need to hire more sophisticated staff to track and monitor outsourcing. There was also discussion on the allocation of time spent on maintaining the security of electronic content and rights management and licensing of electronic content. Staff and patron training and instruction, as well as marketing the digital library, may require more time from reference and instruction librarians. The participants suggested that more reference librarians be involved in future studies and that training and instruction be included when allocating labor resources to the all-digital library processes. This is supported in the Drexel University library print and electronic journal cost study. The highest staff-related cost for either format was reference support, with the higher costs reported for ejournals [Montgomery and King]. TransitionAnother concern expressed by participants related to the pace of transition from paper to digital media. While an all-digital library may require fewer resources than an all-paper library, it is plausible that a mixed paper-digital library may be more costly to operate than either an all-paper or all-digital library since a mixed library requires most of the resources of both. Participants expressed concern that the transition to predominantly digital media may burden libraries with significantly increased costs in the short- to medium-term, and that it may be some time (years, decades) before the full cost benefits of digital media are fully realized. New ServicesThe discussions and cost allocation exercise undertaken in this research explicitly limited the comparison of paper and digital libraries with the assumption that both library types provide the same scale and scope of services. A number of participants noted that the arrival of digital media is providing opportunities and pressures to increase the variety of services that libraries offer, such as Internet reading rooms, printing services, and new reference services. While such new services are needed and desirable from a service perspective, they will put additional pressure on scarce library resources. ConclusionsAs addressed in the literature review, there is a current need to examine the costs associated with providing library resources and services, yet there are few published studies that examine and compare the costs of providing these resources and services. Librarians need to have data points to make informed decisions. This is especially critical in a flat or declining economic environment where librarians are acquiring more materials in digital format, yet continue to maintain physical materials. The findings of this study indicate an agreement among the participating librarians that labor, aggregate space requirements, and material resources are estimated to be less in an all-digital library than in a paper library. Although the results suggest that aggregate equipment use in a digital library will be less than for a paper library, the greater range between maximum and minimum suggests an uncertainty among participants in equipment requirements in an all-digital library. The participants also expressed uncertainty and concern about the possibility of higher salaries needed to hire and retain a more knowledgeable and skilled level of staff to effectively manage an all-digital library. There is also an expressed concern for managing the costs of maintaining both the digital and paper library simultaneously, which seems to be the current scenario. Participants believed that this will not only manifest in the selection, acquisition, cataloging, maintenance, circulation, storage and warehousing, and deselection of materials, but also in the demand for more and different reference services, bibliographic instruction, and technology support. The later were not addressed in this study, but should be considered for further research. These uncertainties and the lack of available resource allocation data warrant the continued study of the resources needed for the transition from a paper to a digital library. This study provides a list of resource allocations by library function that can be used to monitor the resource allocations required to offer content in the paper library and the all-digital library. Since this study is exploratory and does not include a representative, random sample of librarians, its results cannot be generalized to all libraries. In an effort to provide information for library decision-making, researchers should continue to collect and compare data to the results generated by this study. References[The Book & The Computer] The Book and the Computer, Interview: "The Library Shall Endure: A Conversation with Michael Gorman." (Online Symposium 2003) Available: <http://www.honco.net/os/gorman.html> (April 27, 2003). [California State University, 2002] California State University Libraries Electronic Access to Information Resources Committee and e-Book Coordinating Team. Analysis of the Use of e-Books as Compared to Print. (December 2, 2002) Available: <http://www.calstate.edu/SEIR/eBk_FINAL_RPT/eBk_Final_Rpt_Sect_8.2.shtml> (April 27, 2003). [California State University, 2003] The California State University Libraries Electronic Access to Information Resources Committee and e-Book Coordinating Team. Ebook Project. (April 16, 2003) Available: <http://www.calstate.edu/SEIR/eBook.shtml> (April 27, 2003). [Carlson] Scott Carlson, "Do Libraries Really Need Books? Controversial Projects at Some Colleges Move the Printed Word Out of Sight," The Chronicle of Higher Education, (July 12, 2002) Available: <http://chronicle.com/weekly/v48/i44/44a03101.htm> (April 27, 2003). [Cummings] Martin M. Cummings, "Cost Analysis: Methods and Realities," Library Administration & Management, 3 (Fall 1989): 181. [Lawrence et al.] Stephen R. Lawrence, Lynn Silipigni Connaway, and Keith Brigham, "Life Cycle Costs of Library Collections: Creation of Effective Performance and Cost Metrics for Library Resources," College & Research Libraries, 62 (Nov. 2001): 541-553. [Mackie-Mason et al.] Jeffrey Mackie-Mason, Juan Riveros, Maria Bonn, and Wendy Lougee, "A Report on the PEAK Experiment: Usage and Economic Behavior." D-Lib Magazine, 5, no. 7/8 (July 1999). Available: <doi:10.1045/july99-mackie-mason>. (April 27, 2003). [Montgomery] Carol H. Montgomery. The Transition to an Electronic Journal Collection: Measuring the Operational and Economic Implications. (October 29, 2002) Available: <http://www.library.drexel.edu/facts/imls/default.html> (April 27, 2003). [Montgomery and King] Carol Hansen Montgomery and Donald W. King, "Comparing Library and User Related Costs of Print and Electronic Journal Collections: A First Step Towards a Comprehensive Analysis," D-Lib Magazine, 8, no. 10 (October 2002). Available: <doi:10.1045/october2002-montgomery> (April 27, 2003). [Quant] Richard E. Quandt, "Scholarly Materials: Paper or Digital?," Library Trends, 51 (Winter 2003): 349-375. [Roberts] Stephen A. Roberts, Cost Management for Library and Information Services, (London: Butterworths, 1985). [Sanett] Shelby Sanett, "Toward Developing a Framework of Cost Elements for Preserving Authentic Electronic Records into Perpetuity," College & Research Libraries, 63 (Sept. 2002): 388-404. [Survey] See <http://leeds.colorado.edu/faculty/Lawrence/eBooks/> for the survey instrument. Copyright © Lynn Silipigni Connaway and Stephen R. Lawrence |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Top | Contents | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
D-Lib Magazine Access Terms and Conditions DOI: 10.1045/december2003-connaway
|