D-Lib Magazine
April 2005

Volume 11 Number 4

ISSN 1082-9873

Initial Experiences in Developing a Chronologically Organized Digital Library for Continuing Education in Biodefense

Donna M. D'Alessandro, MD, Department of Pediatrics
Michael P. D'Alessandro, MD, Department of Radiology
University of Iowa Carver College of Medicine
Iowa City, IA

Point of contact for this article: Donna M. D'Alessandro, MD
<donna-dalessandro@uiowa.edu>

Abstract

To address an identified but unmet need, a chronologically organized digital library and learning collaboratory has been developed for delivering continuing education to investigators and trainees in the form of biodefense-related news stories, which can be integrated into their daily workflow in a time-efficient basis. We have begun to show how use of this program can broaden and deepen investigators and trainees knowledge and perspective of biodefense-related issues and thus give them a larger context for their work, and how over time these news stories aggregate into an unstructured curriculum of biodefense topics that closely parallels the structured curriculum of a formal biodefense graduate program. This technique of using a chronologically organized digital library dedicated to delivering a discipline-specific unstructured curriculum that unfolds in practice can be generalized to any discipline.

Introduction

The United States government is embarking upon an immense biodefense research initiative in order to develop drugs, vaccines, and diagnostics to mitigate the threat posed by biological weapons. A significant task of this biodefense research initiative is to initially educate investigators and trainees about the field of biodefense, and then to continue that education throughout their professional lives. Unfortunately, due to the investigators themselves as well as to the research process, this educational process is not straightforward.

By their nature, investigators are very focused in their work – on the agent they are studying, its sequence, its virulence – and they may lack the broader context and perspective of how their work fits into the overall biodefense field. Additionally, the demands of their research do not allow much time for initial, much less continuing, biodefense education. Although there are currently two graduate programs in biodefense studies in the United States, biodefense investigators and trainees often have neither the inclination nor the time to pursue a formal biodefense degree.

Furthermore, research is very often a solitary endeavor, and the research culture in this topic area has not traditionally been a culture of sharing [1]. The National Institutes of Health (NIH) has indicated that research must become more collaborative, and to overcome this hesitancy, the NIH is pushing more data sharing and collaboration in future projects it funds.

Therefore, the biodefense research initiative needs answers to the following questions with regard to how the current and next generation of biodefense investigators and trainees should be educated:

• How can the narrow focus that many investigators and trainees bring to their work be broadened to give them a deeper knowledge of biodefense-related issues so they can better put their work in context?
• How can the topics covered in a structured biodefense graduate program curriculum be covered in an unstructured manner for investigators and trainees, and integrated in a time-efficient way into their busy days?
• How can a sense of educational community and collaboration be developed among biodefense investigators and trainees separated by large physical distances? Can this educational collaboration, and the trust and relationships it embodies, then be extended to future research collaborations?

A digital library is defined as an organized collection of information on the Internet that can be used to answer questions or solve problems. A "learning collaboratory" is defined as a "center without walls" in which individuals – without regard to geographical location – can perform their work: interacting with colleagues, accessing instrumentation, sharing data and computational resources, and accessing information in digital libraries.

The goal of the project described in this article is to create a biodefense digital library and learning collaboratory that will serve as a national core facility enabling, enhancing, and intensifying education, communication, interaction and collaboration amongst all biodefense investigators and trainees.

The project's hypothesis is that biodefense investigators and trainees can broaden and deepen their knowledge of biodefense-related issues through the use of a continuing education program delivered daily in small granules and integrated into their workflow in the form of biodefense-related news stories. Over time, these news stories will aggregate into an unstructured curriculum of biodefense topics that will closely parallel the structured curriculum of a biodefense graduate program. Furthermore, as the investigators and trainees engage in discussions related to these news stories, the result will be the development of educational collaborative relationships.

The purpose of this article is to share our initial experiences in using a chronologically organized biodefense digital library and learning collaboratory for the continuous education of biodefense investigators and trainees.

Materials and Methods

This project has three interrelated specific aims:

1. Designing and implementing a biodefense digital library and learning collaboratory,
2. Operating the digital library and learning collaboratory, and
3. Evaluating the use of the digital library and learning collaboratory.

Designing and implementing a biodefense digital library and learning collaboratory

The biodefense digital library and learning collaboratory has three closely integrated components:

1. News Library
2. Reference Library
3. Learning Collaboratory

News Library – As learning is being recognized more as a process of apprenticeship, and therefore as an improvised practice, it is recognized that an unstructured learning curriculum unfolds in opportunities for engagement in practice [2]. The purpose of the News Library is to serve as a source of situated learning in the form of daily biodefense education and continuing professional development for biodefense investigators and trainees. Over time, the News Library reveals an unstructured biodefense learning curriculum that unfolds in practice. The News Library consists of an annotated list of links to non-classified (open source) biodefense news stories and recently published peer-reviewed journal articles that are posted daily after intelligent software agents scan several thousand international news sources for biodefense-related news stories (Google™, Mountain View, CA). A human curator exercises editorial control over the search results and decides which of the stories to link to and publish in the News Library. The biodefense news stories are then automatically archived by topic in reverse chronological order to facilitate review of biodefense news stories by topic at a later time. To assist investigators and trainees who are new to the biodefense field and who require an initial education in biodefense, the News Library contains a Reading List section of classic articles covering important topics in biodefense and an Online Courses section containing links to biodefense-related formal courses available on the Internet.

Reference Library – The Reference Library is a core library of current U.S. government unclassified biodefense textbooks and manuals. The purpose of this Reference Library is to serve as a biodefense knowledge management system, connecting the biodefense investigator or trainee faced with a question or a problem with the right information, at the right time, so he or she can make the right decision. The Reference Library is added to and revised as new biodefense textbooks and manuals are published by the U.S. government.

Learning Collaboratory – The Learning Collaboratory is a virtual community of individuals sharing common interests and ideas over the Internet through the use of a computer conferencing system. It thus serves as a virtual learning community or community of practice. The purpose of this Learning Collaboratory is to bring together online biodefense investigators and trainees so they may freely and easily discuss the numerous issues in biodefense research, education, or patient care. As an example of how the components of the biodefense digital library and learning collaboratory are closely integrated, daily news stories may become the focus of daily discussions within the Learning Collaboratory. The intent is to replicate the intellectual exchange that occurs informally amongst colleagues in hallway encounters, at the watercooler, or at scientific meetings – both within and outside of the conference halls and poster sessions.

All the components of the biodefense digital library and learning collaboratory consist of unclassified information in the public domain and are available to anyone via the Internet. Thus, the biodefense digital library and learning collaboratory can be used secondarily as a news, reference and discussion tool by health care providers, first responders, the media, and other interested parties.

The biodefense digital library and learning collaboratory is implemented using the Manila™ digital library and collaboratory software (UserLand Software, Danville, CA). Manila consists of an object-oriented database to manage the digital library at its core, surrounded by a content management system that includes editorial workflow to support publishing in the digital library. Manila also provides an integrated discussion system to support collaborative work between individuals, and it supports Really Simple Syndication (RSS) to facilitate user notification of new information (content or discussions) being added to the digital library and learning collaboratory.

Operating the digital library and learning collaboratory

The biodefense digital library and learning collaboratory is operated, first on behalf of our National Institute of Allergy and Infectious Diseases-funded Regional Center of Excellence for Biodefense and Emerging Infectious Diseases Research (RCE), second on behalf of the other eight national RCEs, and third on behalf of all other U.S. biodefense investigators and trainees, including those in the U.S. government, U.S. military and U.S. industry. This resource, in addition, is of value to a broader audience including, but not limited to, health care providers, first responders, and the media.

Evaluating the use of the digital library and learning collaboratory

We evaluate how biodefense researchers and trainees use the biodefense digital library and learning collaboratory to see if it meets their information and communication needs. Evaluation techniques include World Wide Web server log file analysis (Site Meter, Washington, DC), qualitative analysis of user-submitted online comment forms, citation frequency analysis (Google, Mountain View, CA), and a qualitative curriculum analysis of how the unstructured curriculum of the biodefense digital library and learning collaboratory compares to the structured curriculum of the biodefense graduate program at George Mason University, Manassas, VA.

Results

Design and Implementation

As part of the University of Iowa's Planning RCE grant, a prototype biodefense digital library and learning collaboratory was designed and implemented on the Internet at BiodefenseEducation.org (http://www.biodefenseeducation.org).

Operation

The biodefense digital library and learning collaboratory has been in operation since January 1, 2004. Specifically, one author (D.M.D.) has overseen daily updating and curatorship of the News Library and the evaluation of the project. The other author (M.P.D.) has overseen the design, implementation, and daily operation of BiodefenseEducation.org and has also served as the digital-librarian-in-chief of the Reference library.

Publicity for BiodefenseEducation.org has been handled in the following manner. E-mails announcing BiodefenseEducation.org were sent to the principal investigators of all RCEs, as well as to the Webmasters of the top 100 biodefense-related Web sites as determined by Google's citation frequency analysis (Google, Mountain View, CA). BiodefenseEducation.org was also enrolled in the leading Internet general and health-sciences specific search engines and directories.

From January 1, 2004 - December 31, 2004, 350 biodefense news stories were posted in the News Library (see Table 1). These news stories came from 130 different Internet news sources (e.g., newspapers, journals, newswires, television/radio networks, federal agencies, etc.) in twelve different countries. The five most heavily covered news topics, in descending order, were anthrax, research, smallpox, public policy, and public health.

The Reference Library currently consists of six searchable U.S. Government biodefense-related textbooks, a biodefense-delimited, one-click search of MEDLINE, and links to authoritative biodefense resources on the Internet.

From January 1, 2004 - December 31, 2004, three biodefense discussions occurred in the Learning Collaboratory.

Evaluation

The formative evaluation of BiodefenseEducation.org has provided results that are used to improve the design and implementation of the biodefense digital library and learning collaboratory through a process of continuous quality improvement.

Overall usage of BiodefenseEducation.org was determined from January 1, 2004 - December 31, 2004. BiodefenseEducation.org was used by approximately 8,200 visitors who accessed approximately 19,000 pages of information. The visitors reflected a broad audience, with 21% from U.S. universities, 14% from the U.S. government or U.S. military, 54% from other U.S. locations, and 11% from outside the U.S. Overall usage has continued to grow steadily over time.

Initial reviews from individuals in the biodefense field (10 comment forms) and journals have been favorable. Typical of comments received from individual users is that BiodefenseEducation.org "...has news articles and commentary I would otherwise miss...a very nice resource." The British Medical Journal, in its July 10, 2004 review stated "In the fast moving modern world, biodefense is becoming a new specialty that we may all have to know more about. So check out BiodefenseEducation.org and its library of information on this topic."

BiodefenseEducation.org has become one of the most highly regarded biodefense sites, when measured in terms of citation frequency analysis. When the term "biodefense" is entered into the Google search engine, BiodefenseEducation.org has a Google PageRank of 10, meaning that Google finds it to be one of the top 10 biodefense-related sites on the Internet. The Yahoo search engine and the MSN search engine return a similar PageRank for BiodefenseEducation.org, even though each of these search engines uses a unique algorithm to calculate PageRank. This serves to further confirm the authoritative nature in which BiodefenseEducation.org is held [3].

There is currently one graduate program in biodefense studies at U.S. universities that has its curriculum published online (the one at George Mason University). When the topics of the 350 biodefense news stories in the unstructured curriculum of the News Library are compared to the topics in the structured curriculum in the required courses in the Masters and Ph.D. programs (Table 2) in the Biodefense Graduate Program there [4], 7 out of 7 (100%) of the topics in the required courses are covered in various degrees of depth in the News Library. When the same topics in the unstructured curriculum in the News Library were compared to the topics in the structured curriculum in the elective courses in the Masters and Ph.D. programs (see Table 2), 23 out of 23 (100%) of the topics in the elective courses are covered in various degrees of depth.

Discussion

There are five ways to organize information: by location, by alphabet, by time, by category, and by hierarchy [5]. In the domain of digital libraries, organization by time appears to be the least-used organization scheme; however, considering how users interact with our digital library, organization by time is the most logical scheme for the biodefense digital library.

BiodefenseEducation.org is one of a growing number of chronologically organized digital libraries dedicated to delivering a discipline-specific unstructured curriculum that unfolds in practice. Perhaps the best known of these is Astronomy Picture of the Day [6], which is being used by students and teachers from grade school through university – as well as by individual adult learners – to pursue an unstructured curriculum in astronomy [7], complete with an online learning collaboratory [8].

We developed BiodefenseEducation.org to deliver continuing education in the form of biodefense-related news stories to investigators and trainees that can be integrated into their daily workflow in a time-efficient basis. We have begun to prove our hypothesis by showing how use of BiodefenseEducation.org can broaden and deepen investigators and trainees knowledge and perspective of biodefense-related issues, and thus give them a larger context for their work. We also show how over time these news stories aggregate into an unstructured curriculum of biodefense topics that closely parallels the structured curriculum of a formal biodefense graduate program.

BiodefenseEducation.org can be viewed as serving as a mentor, that on a daily basis suggests to its users concise readings related to current events in biodefense, thus allowing daily situated learning to occur. Thereby, an unstructured curriculum unfolds in practice for users. There are striking similarities between this educational program and the educational program embodied in the traditional Morning Report medical residency conference. Morning Report is a teaching conference, held early each day, in which resident physicians in training review with experienced staff physicians the care of the most interesting and challenging patients admitted to the hospital in the previous 24 hours. Morning Report has been shown to be an unstructured curriculum that unfolds in practice as these daily discussions of patients and knowledge imparted aggregate over time [9]. Both educational programs reinforce the idea of learning as a process of apprenticeship, as they can be done over morning coffee at work delivering a little information at a time. Yet over a long time period, the aggregated information delivered is quite large and covers a broad range of topics.

Ironically, this unstructured curriculum is beginning to cross into the structured curriculum of the classroom, as BiodefenseEducation.org has recently been adopted for use as a learning resource in the second graduate program in biodefense studies at U.S. universities, Georgetown University's Masters in Biohazardous Threat Agents and Emerging Infectious Diseases.

Ultimately, we hope to offer investigators and trainees who don't naturally collaborate a rallying point, by engendering educational collaboration amongst them through the Learning Collaboratory. We hope that in an area in which they perceive the personal risks to be low – education – they can begin the process of personal contact and social networking that build trust and create relationships that may lead to research collaborations.

Our plans for the future are to build upon this solid prototype and address three challenges we now face: refining, publicizing, and funding BiodefenseEducation.org. Specifically, BiodefenseEducation.org will have its design refined based upon the formative evaluation already completed as well as upon future evaluations, have the content in its News Library and Reference Library expanded, and undergo a summative evaluation. Additionally, we will work to further increase awareness and usage of BiodefenseEducation.org within the biodefense community. It is our hope that once a critical mass of participants is reached, usage of the learning collaboratory will begin to flourish. Finally, we are endeavoring to develop a sustainable business model for BiodefenseEducation.org. Funded as a prototype for one year by the National Institute of Allergy and Infectious Disease (NIAID), BiodefenseEducation.org will require stable, long-term funding to reach its full potential.

Our technique has several strengths. First is that health care professionals have described a preference for obtaining educational information on biodefense from Web-based news sources rather than from Web-based medical textbooks and continuing education courses [10]. By focusing its continuing education program in biodefense around the delivery of biodefense-related news stories, BiodefenseEducation.org addresses this preference.

Second are the broad numbers of authors and news sources that contribute stories. The vast majority of these stories are of high quality, as they are written by the science writers from national and international news sources (e.g., Associated Press, New York Times, Washington Post, Reuters, British Broadcasting Corporation.) The accuracy and timeliness of these news sources was confirmed recently by the Director of the Public Health Agency of Canada's Center for Emergency Preparedness and Response, who operates the Global Health Intelligence Network that data mines these news sources to create biodefense intelligence reports for national and international health agencies [11].

The third strength of our technique is the key role played by the BiodefenseEducation.org curators, who have expertise in library science, medical informatics, medicine, and education. This expertise allows them not only to find stories, but also to evaluate them for accuracy and relevancy before posting them into the News Library. The curators function in many of the same ways that teachers do, fulfilling a prophecy described in the early digital library literature. The first article describing a digital library defined curatorship as one of the key roles that needed to be performed within a digital library [12]. A role was foreseen for learned individuals called "trailblazers" who would serve as guides, weaving trails that others could follow through the vast amounts of information in the digital library. One of the first books describing digital libraries foresaw the development of a similar subculture of intellect, entitled the Xanadu Hypercorps, consisting of individuals that would be a combination of librarian, teacher and academic – using these skills to help users cope with an "avalanche, rather than a trickle, of ideas and information [13]."

Our technique also has several limitations. First, we are limited to English language news stories, as the software agents we use are set to search only English language sources, and the digital library's curators speak only English. This is not a major limitation, however, as many of our English language news sources have journalists stationed around the world (e.g., the British Broadcasting Corporation) and many of the larger foreign news sources make available English language editions of their sites (e.g., Agence France Press). Nonetheless, this limitation could be overcome through the allocation of more resources allowing the hiring of an international staff to make the site multilingual.

Second, the half-life of the stories linked to may be rather short. Many of the newspaper Web sites whose stories BiodefenseEducation.org links to allow free online access to these stories for a limited number of days. After that, the stories are moved to an archive that charges for access. Whenever possible, BiodefenseEducation.org links to Web sites with versions of these stories that remain freely available indefinitely. However, in order to read all the stories BiodefenseEducation.org links to, one must visit BiodefenseEducation.org on a regular basis, before the newspapers move their stories to their archives.

Third, there is a concern that over the long term the World-Wide Web may turn into a pay-as-you go medium. However, this has been a concern for the last decade that has still not come to pass, and such a situation is not imminent. Many news sites are profitable and therefore allow all of their content to be freely available, with some news sites only requiring a one-time user registration in order to view their content (e.g., New York Times, Washington Post), while others make a selection of their content available free of charge (e.g., The Economist). The variety of news site business models insures that there will remain a wide spectrum of high quality content information on the World Wide Web that is available free of charge for the foreseeable future.

Fourth, it should be pointed out that providing a daily reading list of stories does not guarantee the stories are read or understood, and therefore learning does not necessarily result. While this may be true, a similar situation exists in the classroom, whereby students showing up for lecture are not guaranteed to listen to or understand what they hear. Learning is truly in the hands of the learner. Our technique provides an alternative and complimentary learning technique to the traditional classroom, one that allows us to put adult learning theory to work for motivated individuals. While the online unstructured curriculum is different than the structured curriculum of a classroom, we believe it shows promise as a viable alternative for those who don't have the time, inclination or ability to attend a structured course of study in biodefense affairs.

In summary, we have developed a unique expertise and focus in the area of biodefense educational informatics. Specifically, BiodefenseEducation.org is developing into a well-used and highly regarded biodefense digital library and learning collaboratory, and it is off to a solid start as a source of continuing education on biodefense affairs. This program and technique can easily be generalized to other disciplines as well.

Acknowledgments

This project was funded by the National Institute of Allergy and Infectious Diseases as a part of the University of Iowa's Planning Regional Centers of Excellence for Biodefense and Emerging Infectious Diseases Research grant.

References

1. Arzberger P, Finholt TA. Data and Collaboratories in the Biomedical Community. Bethesda, MD: National Institutes of Health National Center for Research Resources, 2002: p. 13.

2. Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. 1991. Cambridge, United Kingdom: Cambridge University Press. Pp. 91-117.

3. Internet Search for the term "biodefense" on January 25, 2005 at www.google.com, search.yahoo.com, and search.msn.com.

4. Biodefense Graduate Program. Biodefense Degree Programs. [Web document] Manassas: George Mason University, 2003 [rev. 1 Sep 2004, cited 28 Jan 2005]. Available at <http://www.gmu.edu/departments/mmb/biodefense/academics.html>.

5. Wurman RS. Information Architects. 1996. Graphis Press. Zurich, Switzerland. p. 17.

6. Wilson B. Featured Collection: Astronomy Picture of the Day. [Web document] Reston: D-Lib Magazine, 2004 [rev. 15 Nov 2004, cited 28 Jan 2005]. Available at <http://www.dlib.org/dlib/november04/11featured-collection.html>.

7. Nemiroff R, Bonnell J. Astronomy Picture of the Day in the Classroom. [Web document] Greenbelt: NASA, 1995. [rev. 1 Jan 2005, cited 28 Jan 2005]. Available at <http://antwrp.gsfc.nasa.gov/apod/lib/apodclass.html>.

8. Nemiroff R, Bonnell J. Discuss an Astronomy Picture of the Day. [Web document] Houghton: The Night Sky Live, 2004 [rev. 28 Jan 2005, cited 28 Jan 2005] Available at <http://bb.nightskylive.net/asterisk/>.

9. D'Alessandro DM. Documenting the Educational Content of Morning Report. Arch Pediatr Adolesc Med. 1997 Nov; 151(11): Pp. 1151-6.

10. Chung S, Mandl KD, Shannon M, Fleisher GR. Efficacy of an Educational Web Site for Educating Physicians About Bioterrorism. Acad Emerg Med. 2004 Feb; 11(2): Pp. 143-148.

11. Associated Press. Global Health Alert Network Adds Languages. [Web document] New York: Associated Press, 2004. [rev. 17 Nov 2004, cited 28 Jan 2005.] Available at <http://www.ap.org>.

12. Bush V. As We May Think. Atlantic Monthly 1945; 176(1): Pp. 101-108.

13. Nelson TH. Literary Machines. 1981. Mindful Press. Sausalito. Pp. 3/9-3/11.

Table 1 - Categorization by Topic of News Stories Posted From January 1, 2004 - December 31, 2004*

*Each news story is categorized under only 1 topic using a pre-determined criteria set of which the main criterion is what is the most important subject of the news story. Specificity is also used as a criterion. For example, a news story with the title "Anthrax Vaccination Possible with Skin Patch" could be placed into the Anthrax or Vaccination category. Anthrax is the main subject of story, is listed first in the title, and is a more specific category than vaccines. Therefore this story would be categorized with Anthrax.

Table 2 - Topics in the Structured Curriculum in the Masters and Ph.D. Programs in the Biodefense Graduate Program at George Mason University [4]

A. Topics in the Required Courses
  Agroterrorism
  Bacillus anthracis
  Clostridium botulinum toxin
  Francisella tularensis
  Variola major
  Viral hemorrhagic fevers
  Yersinia pestis

B. Topics in the Elective Courses
  Approaches to biological warfare medical treatment and response
  Bioengineering and bioprocessing
  Coordinated response to a bioterror attack
  Counter-terrorism and civil rights
  Crisis and consequence management
  Detecting production of biological agents
  Detection techniques in a bioterror attack
  Dispersal patterns of biological agents
  Drug discovery and development
  Engineering immunology and immunomodulating drug development
  Epidemiology of a bioterror attack
  Ethical aspects of biodefense
  Examining terrorist groups
  History of biological agent use and treaties
  History of genetically engineered bioweapons
  How to conduct a bioterror attack
  Legal aspects of biodefense
  Non-proliferation
  Political aspects of biodefense
  Principles of toxicology
  Scientific aspects of biodefense
  Techniques in immunology
  Working in a high threat environment

Top | Contents
Search | Author Index | Title Index | Back Issues
Previous Article | Next article
Home | E-mail the Editor

D-Lib Magazine Access Terms and Conditions

doi:10.1045/april2005-dalessandro