Search D-Lib:

D-Lib Magazine

July/August 2017
Volume 23, Number 7/8
Table of Contents

Explorations of a Very-large-screen Digital Library Interface

Alex Dolski
Independent Consultant
alexd [at] n-ary.net

Cory Lampert
University of Nevada, Las Vegas Libraries
cory.lampert [at] unlv.edu

Kee Choi
University of Nevada, Las Vegas Libraries
kee.choi [at] unlv.edu

https://doi.org/10.1045/july2017-dolski

Abstract

While digital libraries accommodate remote access via the web and mobile devices, their physical presence tends to be minuscule. A locally-developed prototype digital library application called DLib Wall, connected to MultiTaction display hardware from MultiTouch Ltd., is an attempt to create a rich and engaging onsite presence for the digital collections of the University of Nevada, Las Vegas Libraries. DLib Wall is one of the first applications of its kind, and demonstrates several relatively unexplored interaction modalities. In late 2014, it was deployed in the Goldfield Room — a conference room in the Lied Library — on a wall-mounted array of six 42-inch touch displays. DLib Wall represents both application development and the work of a collaborative team charged with creating a large, interactive, and content-rich media wall experience on a limited resource budget. The group's work included: evaluating and recommending technologies, managing the custom development for the DLib Wall application, and creating an initial content plan for the public rollout. This article highlights the technical aspects of the application while sharing key decision points that the team encountered in the process of bringing the project from conception to completion.

Keywords: DLib Wall, Digital Library Applications, Visual Collections Display, Digital Exhibits, Digital Collection, Touchscreen Technology

1 Introduction

As a ubiquitous means of document exchange, the web is a natural fit for dissemination of digital library content. Some digital libraries even offer interfaces optimized for handheld devices. But it is all too common that their collections are quarantined within their own oft-visited "silo" within a labyrinthine library website, where they are hard to find and harder to enjoy.

Likewise, interaction with digital objects has remained a suboptimal experience. On a desktop computer, it is difficult to duplicate the experience of manual interaction with a mouse or touchpad. By removing the intermediary input device, touch screens may improve this, but the small size of a handheld device can be underwhelming.

DLib Wall is a custom application developed by UNLV Libraries to bring a rich and engaging physical presence to digital library collections. It can show visual collection resources (images and videos) in a variety of formats intended to engage passers-by who may have had no prior familiarity or even awareness of their existence. It is designed to be easy-to-use by patrons of any age or technical aptitude.

DLib Wall is scalable in terms of display resolution. It was developed on a laptop with a 15-inch, 1,440 by 900-pixel screen, and currently runs on an array of six 42-inch 1080p displays mounted side-by-side, shown in Figure 1. The dimensions of the array are 124 inches wide, 37 inches tall, and 128.9 inches (12 3/4 feet) on the diagonal, with a total display resolution of 6,480 by 1,920 pixels. (It should be noted that that these numbers are fairly modest relative to other MultiTaction installations.)

As video displays continue to grow in size while falling in price, it seems probable that onsite touch displays will grow to complement digital libraries' web and mobile presence.

2 Background Research

Within the cultural heritage community, the term "digital exhibits" is commonly used, but frequently misunderstood. The projects most often referred to when using this term can be classified into three main categories: curated digital content showcasing digital collections or data visualizations (accessed through traditional computer monitors, wall displays and/or mobile devices, digital signage and wayfinding kiosks used in facilities to guide traffic flow and provide visitor information, and digital galleries or digital media walls. These categories can also blur and be combined.

The DLib Wall falls primarily into the third category of large digital media walls. But as part of a growing exhibits program, the scope of the project included investigations on ways that the media wall could leverage digital collections content beyond eye-catching promotional purposes. This led to a focus on possibilities for development of a custom application that extended the media wall software and hardware to facilitate the ability of curators to create exhibits for the wall.

In the commercial world, digital signage was a robust and growing market at the time the group convened (in 2014). But, within libraries and specifically academic libraries, there were not many examples of this type of project in production. There had not yet been time for a deep body of literature to emerge from the academic library field. In researching technologies, conferences like the Digital Signage Expo 2017 and the Consumer Electronics Show 2017 were very helpful for inspiring possibilities, identifying products and vendors, and discussing technical specifications for the hardware and software. Once choices in products were narrowed to a smaller list, requesting detailed vendor information was key in researching technical support questions, compatibility, and costs. For implementation, looking to cultural heritage institution colleagues provided more real-world information about approaches to scoping projects. Some of the institutions with active digital exhibit projects were the Cleveland Art Museum (Cleveland Museum of Art, 2014), Brown University (Hills, 2011), Vanderbilt University's Wordsworth Project (Vanderbilt University, 2017), and the Duke University Digital Exhibits Working Group (Duke University, 2017).

3 Project Goals

The primary goal of the project was to create an engaging onsite way for users to experience digital collections content in an interactive touchscreen environment.

This broader goal was broken down into concrete expectations for a successful implementation:

• Showcase hardware capabilities for multiple simultaneous-user touch screen experience.
• Ingest and display full range of digital collections images (multiple collections, formats).
• Ingest and display metadata associated with digital collections images.
• Provide basic user interactions (selecting images, zooming, viewing metadata, linking to related images).
• Provide basic staff administration tools (ingest control, grouping images into sets/clusters, adjusting basic settings of display).

In addition, a secondary goal of providing to curators a simple and broadly accessible workflow for creating and displaying linear narratives with selected digital content was articulated, which was satisfied by a pair of "exhibit galleries" (see Exhibit Gallery A and Exhibit Gallery B).

4 Initial Approach and Challenges

Before beginning local development work, UNLV Libraries first approached the hardware manufacturer requesting a proposal to develop the initial prototype according to their specifications. At the time, MultiTaction did not provide software development services for their customers and software development was usually outsourced to expensive design firms and rarely done in-house by the client. In our case, however, MultiTaction agreed to collaborate with UNLV Libraries to develop a proof of concept prototype. The initial approach was for MultiTaction to take existing demo applications and modify them to meet the specifications for the DLib Wall prototype. Some months later, MultiTaction responded, not with the proof of concept, but rather proposing an entirely new application. This application, like others investigated from outside creative companies, was not in the range of the library's budget. The cost, as well as doubts about the vendor/library collaboration, ultimately drove UNLV Libraries to pursue in-house development as the primary option for DLib Wall development.

5 Project Timeline

Test hardware for DLib Wall was purchased in 2013. Full implementation of the MultiTaction hardware began in the fall of that year and was substantially complete in one month, with iterative testing and refinement of MultiTaction module configurations occurring over the following months. The second phase of the project, in the spring of 2014, involved work using the MultiTaction widgets to develop five custom gallery views to meet local specifications. Two additional galleries (see Section 6.5, Galleries) were designed and developed specifically to meet the needs of exhibit curators. Once the application was working satisfactorily on the test hardware, six additional MultiTaction wall displays and a computer to drive them were purchased for deployment. The completed application was deployed in late 2014 inside its own custom-built enclosure in the Lied Library's Goldfield Room. Later, the D-Lib Wall was formally named and an event was held to celebrate the opening of the Dr. Steven P. Shearing Exhibit Gallery at the University Libraries. A public web page with details about the gallery was created.

6 Software Architecture

To expedite development of applications for the MultiTaction platforms, MultiTouch, Ltd. offers a runtime library and software development kit (SDK) called Cornerstone, versions of which are available for Windows, Mac, and Linux. Cornerstone provides a high-level, object-oriented API and widget library that makes it relatively easy to develop applications in C++ or JavaScript. (It is also possible to bypass Cornerstone altogether and acquire touch information directly from the displays.)

Cornerstone is intended to be used with one or more MultiTaction displays. In development, however, it can also work with a standard display, using a mouse pointer and keyboard modifiers to simulate touch input.

In action, the application is quite simple. A root "Scene" object rotates through several "Gallery" objects on a timer, and opens object viewer objects in response to touch events occurring in the galleries. Gallery objects are high-level objects that marshal the necessary widgets into position, load them with appropriate content, set them in motion, and attach relevant event listeners to them (see Section 6.5, Galleries).

6.1 Widgets

Widgets are the basic graphical building blocks of Cornerstone applications, serving as the basis of all visual and interactive elements. The "MultiWidgets" namespace in the Cornerstone API contains several pre-built widget types, including a widget to display images (called ImageWidget), a video player (VideoWidget), a widget to abstract a view into a scene (ViewWidget), and widget to display text (TextWidget).

Widgets are organized in a tree; they can have one parent widget and zero or more child widgets (MultiTouch, 2014). They have their own coordinate systems, and may be oriented above or below other widgets. By default, widgets respond to the standard touch gestures of translation (dragging with one or more fingers), rotation (rotating with two or more fingers), and scaling (pinching with two or more fingers); but much of this behavior is customizable by adding particular "input flags" to a widget.

6.2 Data Import Process

Via a pair of scripts purpose-written for the task, DLib Wall is currently capable of ingesting data from the CONTENTdm digital asset management software used at the UNLV Libraries. CONTENTdm's main intellectual data structures include collections, objects, and "compound objects," or object aggregations. With the appropriate import scripts, DLib Wall could be adapted to support other idiomatically similar systems — which should include most — as well as simple file/folder hierarchies.

The process of importing data may take from hours to days depending on the number and size of objects to be imported and whether they first need to be converted to supported formats. It is as follows:

1. Object master files are copied into a data folder.
2. Object metadata is exported from CONTENTdm in XML format. This is an easy, point-and-click process, but must be performed manually for each collection that is to appear in the application.
3. A media conversion script scans the master files and, if necessary, converts them into a supported format using ImageMagick (images) or FFmpeg (audio/video). This includes converting JPEG2000-format images, which the Cornerstone codecs do not support, to JPEGs; QuickTime-format movies to MPEG-4; etc. The script also generates scaled-down image derivatives for all objects to serve as thumbnail images. For image objects, this is simply a reduced version of the image; for video objects, it is a frame from the video; and for compound objects, it is the first image within the object.
4. A metadata import script (written in Python) seeds the application's SQLite database with the content of the exported CONTENTdm XML files.
5. A final step involves pre-generating image mipmaps to be used by MultiWidgets::ImageWidget. The first time it loads an image, ImageWidget automatically generates derivatives at several scales to improve performance. However, this introduces an unacceptable delay before the image appears. Pre-loading all of the images gets this processing out of the way in advance so it doesn't have to happen in public.

After the initial import, adding and removing data both require repeating this process, but it proceeds much faster on subsequent runs since already-imported objects are skipped.

6.3 Running the Application

DLib Wall resides in its own self-contained folder on disk, and can be launched from the command line. Alternatively, it can be launched using special "Codice" markers placed directly on the screen (Figure 2). (These are like QR codes that the MultiTaction display can read.)

During operation, the application reads digital object binaries and metadata from a separate folder, populated during the import phase. Because all content resides on a local solid-state drive (SSD), access is fast and there is no need for a network connection.

6.4 Choosing Content

There is no practical upper limit to the number of objects that DLib Wall can support. As it loads only the objects it needs from the database and filesystem at one time, the simple SQL queries it executes should perform adequately into the millions of records.

That said, with typical settings, DLib Wall will be displaying no more than a few hundred objects at a time. Considering that each gallery takes only 45 seconds to cycle through by default, and that most users will be visiting the unit for only a few minutes, there is no need to populate the system with vast numbers of objects. On the contrary, a considerable degree of selectivity is desirable. Content was drawn from a series of thematic digital projects selected and created from UNLV Libraries Special Collections. DLib Wall is an inherently visual experience that calls for interesting, eye-catching content. Quality, in this context, matters more than quantity.

6.5 Galleries

In its current iteration, DLib Wall includes seven distinct "galleries," or scenes. Galleries cycle from one to the next in a repeating loop in a sequence defined in the application's configuration file. A gallery can appear more than once in this order, or not at all. The idea of distinct galleries was borrowed from the Collection Wall in Gallery One at the Cleveland Museum of Art (Cleveland Museum of Art, 2014).

The default, "out-of-the-box" gallery cycle order tries to avoid displaying similar galleries consecutively, to keep the user experience interesting. For example, because the Random Object Gallery and Collection Gallery are somewhat similar, they do not appear in succession.

6.5.1 Brand Gallery

The Brand Gallery is a simple gallery intended to display a brand, such as an institution or department logo, which will be displayed centered on the screen. It has a customizable background color and supports PNG images with alpha. Alternately, an image covering the whole screen can be used. Unlike the rest of the galleries, the Brand Gallery has its own cycle time, which defaults to six seconds, to retain users' attention.

6.5.2 Random Object Gallery

The Random Object Gallery features a random array of images arranged in a grid that scrolls from right to left. Touching an image causes its corresponding object to open in a viewer. Due to the number of ImageWidgets it must display, this is the most compute-intensive gallery. For any reasonably fine grid resolution, it demands a fast CPU and excellent storage read performance (see 6.5.11 Hardware Requirements) to load promptly. This gallery will automatically scroll to cover its entire scene — whose size is set in the configuration file — over the gallery cycle duration. The default scene width is twice the viewport's width. In this way, the apparent scroll speed can be changed.

6.5.3 Collection Gallery

The collection gallery features a vertical array of horizontally-scrolling rows of images. Each row corresponds to a collection of objects. Collections can be individually added or removed from this gallery in the configuration file, and the rows will vertically auto-size to adjust. Scroll speed is specified in the configuration file, but is randomly altered slightly for each row to cause them to drift out of sync, which is more visually interesting. When a row reaches the end of its contents, it reverses direction and starts scrolling in the opposite direction.

6.5.4 Tag Gallery

The tag gallery displays a random array of Dublin Core subject elements. When an element is touched, a multi-object viewer opens (see Section 6.5.10 Object Viewers), displaying a grid of thumbnail images corresponding to the objects to which the element value applies. These can then be touched to open ordinary object viewers. The appearance of the gallery is customizable with CSS. Each tag is randomly assigned a "TagRandomN" CSS class, where N is a number from 0 to 9; a "TagRankN" class, where N corresponds to the frequency with which the tag appears in the collections; and a "TagEven" or "TagOdd" class. Styles can be applied to any combination of these classes.

6.5.5 Giant Image Gallery

The Giant Image Gallery randomly selects a photo from the database, scales it slightly beyond filling the viewport, and scrolls upwards and left or right — the "Ken Burns effect." Consistent with this effect, the scrolling accelerates and decelerates. The upwards direction of the scrolling is intended to add emotional effect to waist-level and full-body portraits, in which faces are more likely to appear in the upper half. Reducing the zoom level in the configuration file will bring more of the image into view at once, thereby reducing the apparent scroll speed and potentially accommodating more types of images. This gallery benefits from a screen that does not diverge too greatly from the aspect ratio of the images that it is to display. Very wide screens, like the one pictured, can cause less of much taller images to be visible at once.

The image's Dublin Core title is displayed in the lower left corner. A QR code in the upper right corner points to the image's persistent URL.

6.5.6 Exhibit Gallery A

In Exhibit Gallery A, image objects are pre-selected and their identifiers coded into an XML file. If more object references live in the file than can be displayed at once, the gallery will choose as many from the file as there are screens, and display one per screen. The images will then refresh with the next set of images periodically. Each screen contains an image, a caption, and a QR code pointing to the object's persistent URL. This gallery is not interactive.

6.5.7 Exhibit Gallery B

Exhibit Gallery B displays one row of exhibit photos, chosen at random from the exhibits XML file, and scrolls them from right to left. Touching a photo will cause it to open in a viewer.

6.5.8 Gallery Transitions

Galleries run in a pre-defined cycle, for a duration specified in the application's configuration file. The default duration is 45 seconds, which attempts a compromise between usability and variety. At the end of a cycle, an "OutwardTransition" object disposes of the content in the scene by either fading or "dropping" it out, and then an "InwardTransition" object animates new content from the next gallery into place in the same manner.

6.5.9 Ideas for Future Galleries

Being essentially a high-level API on top of an OpenGL scene, it would seem natural to want to feature 3D models of digital objects. More types of galleries are conceivable, as well. Also, most of the galleries display a simple black background, which could be made more interesting by adding animated colors and/or designs.

6.5.10 Object Viewers

Currently, DLib Wall includes three distinct viewers, to support different types of digital objects. One is an image viewer (Figure 10), which displays an image and, beneath it, its collection name and Dublin Core title. Also in this area is an "info" button which, when touched, displays the image's description and a QR code pointing to its persistent URL. (These can be touched again to make them disappear.)

DLib Wall is highly effective at displaying high-resolution images. Using its automatic mip-mapping technique, MultiWidgets::ImageWidget is capable of displaying very large images with exceptional performance. Every detail of 100-plus-megapixel scanned maps is visible in a larger-than-life size. Arguably, the MultiTaction display provides a superior experience even to the actual physical object (Figure 11).

DLib Wall also supports video objects. Videos not already in a format supported by MultiWidgets::VideoWidget may be able to be converted by the media conversion script, which uses FFmpeg for transcoding. In most respects, the video viewer is similar to the image viewer, with similar close and info buttons.

Finally, there is a multi-object viewer, which displays object thumbnail images in a vertically-scrolling grid. This viewer is used to display compound objects (object aggregations) as well as lists of objects in the Tag Gallery. When a thumbnail is touched, an appropriate secondary viewer will open.

All object viewers can be moved and resized with standard swipe and pinch gestures, and the image and video viewers can be rotated with rotate gestures. All viewers have a close button in the top-left corner, which works predictably. Object viewers will auto-close after a certain length of inactivity — by default, 90 seconds — in order to provide a fresh experience for new passers-by.

6.5.11 Hardware Requirements

Basic Cornerstone applications can run well on any contemporary PC or Mac. In a digital library context, the most resource-intensive use is likely to be of the ImageWidget, which appears most prominently in the Random Object Gallery. Although individual ImageWidgets are quick to load, attempting to load large numbers of them at once can stress even above-average machines. Development experience has shown the likely bottlenecks to be CPU and hard drive performance. Using large numbers of ImageWidgets simultaneously, therefore, calls for a solid-state drive (SSD), or even an array of such drives in a RAID configuration to maximize read performance. A fast multi-core CPU is also helpful.

	Development	Production
CPU	Intel Core i7 3615QM 4-core (2.3GHz)	Intel Xeon E5-2630 6-core (2.6GHz)
RAM	10GB DDR3	64GB DDR3
Storage	1TB 5400RPM SATA	512GB SSD
Graphics	nVidia GeForce GT 650M	AMD FirePro W9000
Display	1440x900	6480x1920
Overall subjective performance (1-10 scale)	7	10

6.6 Display Size & Positioning

Although there is little research on optimizing the physical position of massive-screen digital libraries, it would seem obvious that this is of great importance. The authors propose the following principles:

• The display should be positioned in a high-traffic area where it can attract attention and foot traffic. The application can be enjoyed by any seeing person of any age or technical aptitude, whether through direct interaction or mere observation.
• The display should be raised so that its top edge is low enough to be touchable by most adults. A top edge that is too high will risk having elements on-screen that are untouchable, and could provoke straining or jumping.
• The bottom edge of the display should be high enough to be touchable by most adults without stooping. Keeping it off the floor also prevents kicking.
• If the target user group includes children, the bottom edge should not be out of their reach.
• The display should be located in a space where a reasonable level of volume is acceptable. It will be common for multiple people to want to use the application at once, and it will be natural for them to talk while doing so. Onlookers may comment on the display. Additionally, the application itself can play sounds upon various interaction events, which could be distracting in a quiet area.
• Choice of display size must balance patron traffic levels, available space, and budget. In terms of size, the main advantages of a larger screen are its greater impressiveness and ability to support more simultaneous users.

7 Assessments and Challenges

The selection of technology, development of the D-Lib Wall application, and implementation of the initial content were achieved and are undergoing assessment and evaluation. The event opening the gallery was seen as a successful culmination of the project and the American Library Association Annual Conference in Las Vegas in 2014 offered a unique opportunity for hundreds of librarians to tour the library and see the technology in action. Most visitors were drawn to the display by dynamic, interactive content, the sheer size of the display and the high quality of the images. Many noted that it was engaging and rewarding to experience special collections material in this new way. Most often, users were impressed with the ability to zoom in to extreme detail on images of maps and blueprints and enjoyed the ability to use the system alongside others who were opening and manipulating multiple images. Remarks about the promotional power of the display were common, while others noted that the screens could offer interesting teaching opportunities, if the cost were not prohibitive for wider use.

Going forward from project to ongoing production, some key issues are currently on the table for discussion. Most are concerned with sustaining and extending potential uses of the gallery now that it is in production. Some of these challenges include:

• Optimizing the gallery into the future will take targeted staff resources, beyond the current resources allocated to simply support and maintain the current project.
• Developing a content strategy for the gallery would engage more curators and optimize the user experience.
• Evaluating changing user expectations and needs will inform the usefulness of the technology; obsolescence and technology irrelevance are always risks associated with digital initiatives.
• Future space projects (the gallery is currently not in a public area; considering the role of public vs. closed areas for different types of digital exhibits).
• Gathering feedback through more structured assessment.

8 Future Areas of Research

Another area of high interest is how the gallery could be used beyond the primary goals of this initial project. There is strong interest in several areas for future development:

• Enhancing the gallery for events and customizing content for different audiences.
• Audio and video strategies, especially investigating use of the gallery for oral histories or other multimedia digital collections.
• Implications for teaching; exploring possibilities for the use of large-screen digital exhibits in other venues, such as the classroom.
• Defining and articulating the DLib Wall work within the larger digital exhibits ecosystem of digital signage/wayfinding, promotional digital information monitors, leisure spaces in the library, other physical exhibit spaces, and museum spaces on campus.

9 Conclusion

The DLib Wall has demonstrated its utility as a unique and innovative way of presenting digital library content to an on-site audience that can be enjoyed by anyone of any age, technical aptitude, or prior familiarity with the UNLV Libraries' digital collections. Its highly conspicuous nature makes it ideal for promotion, awareness-building, and even casual browsing of digital library content. As massive touch screens continue to grow in size while falling in price, it seems likely that onsite touch displays will grow to complement digital libraries' remote web and mobile presence. Libraries and other organizations seeking greater visibility and use of their digital collections should consider making them available on a massive touch screen in an appropriate location within their facilities.

Acknowledgements

The authors would like to acknowledge the major contributions made to the project by Peter Michel, Special Collections Curator (UNLV Libraries).

References

[1]	Choi, Kee (2017, February 15). Dr. Steven P. Shearing Gallery.
[2]	Cleveland Museum of Art. (2014, June 23). Collection Wall.
[3]	Duke University. (2017, February 15). Duke University Exhibits Working Group.
[4]	Hills, Alexander, "Museums at Your Fingertips", Computer, vol. 44, no. 12, pp. 87-89, December 2011. https://doi.org/10.1109/MC.2011.374
[5]	MultiTouch Ltd. (2014, April 15). MultiTouch Cornerstone Developer Guide.
[6]	Vanderbilt University. (2017, February 15). Wordsworth.

About the Authors

Alex Dolski worked on the MultiTaction project in an independent capacity prior to joining the University of Illinois at Urbana-Champaign Library as a Research Programmer for Digital Library Projects, where he maintains the Medusa Digital Library Service and Cantaloupe image server projects. Alex is interested in software development for digital multimedia repositories.

Cory Lampert is the Head of Digital Collections at the UNLV University Libraries where she is responsible for operations and strategic planning for a dynamic department that comprises digitization facilities, several digital collections systems/technologies, and grant-funded projects such as the Southern Nevada Jewish Heritage Project. Cory's research interests are in the areas of linked open data and innovation in user-centered digital library design. When not working on digital projects and metadata, she enjoys hiking the peaks of the Mountain West.

Kee Choi is a Technologist who is always excited about new technology which is why many of his colleagues refer to him as the "gadget guy". Kee Choi currently spends his time working with the newest emerging technologies at UNLV Libraries. His most recent focus is on exploring Virtual and Augmented Reality technologies for education.