Evaluation of Chat Reference Service Quality: Pilot Study

Appendix

Marilyn Domas White, Eileen Abels, and Neal Kaske
The University of Maryland

Directions for data collection:

1. Ask the question assigned to you sometime during the hour designated. Allow time for difficulties in reaching the library.

2. Please ask the question as it is written and do not embellish on the question unless asked. If asked, you may provide information in the scenario. For escalator questions, you may provide the next level of information + the scenario. Ask the question honestly and accurately. Record the descriptive information on the transaction immediately after completing the session.

3. Remember the purpose is to assess the accuracy of the information provided and the nature of the encounter. Do not mislead staff members or encourage an incorrect response. Alternatively, do not offer tips on conducting the search.

4. Be sure to request or make a copy of the interaction so that we can analyze the interchange.

5. For each question, please complete the attached form.

Question (number of question): _________
Library: _________
Date/time (actual time asked): _________
Questioner (initials): _________
Length of time from:
Logon to service to actual interchange (queuing time): _________
Beginning to end of interchange (length of interaction): _________

After session is over, please note:

Effectiveness of session: __________
Assess the effectiveness of the exchange (did it result in what appears at this point to be an acceptable answer?) on a scale of 1 to 5 with 1 being not effective and 5 being very effective.

Efficiency of session: __________
Assess the efficiency of the exchange on a scale of 1 to 5 with 1 being inefficient and 5 being very inefficient. Note: An efficient exchange would be one that proceeded fairly directly to the answer with few extraneous comments; an inefficient exchange would be one where the respondent eventually found the information but muddled through the search process.

Special note: It is possible to have an effective exchange that is done very inefficiently.

Nature of response:
For each source, indicate:
1) Source (give URL or title, etc.)
2) Type of source (Website, hardcopy, referral elsewhere)
3) Intermediate or final (Note: Final would actually have the answer; intermediate would be a preliminary step to getting to the final answer.)
4) Any concerns expressed by respondent about answer

Back to the Article