Facets of Quality for Digital Reference Services

Version 5

June 2003

This document outlines important characteristics and features (referred to as facets of quality) for building a digital reference service for all audiences, including the K–12 educational community. The facets are intended as a set of standards for organizations to achieve in creating and maintaining digital reference services participating in the Virtual Reference Desk (VRD) Network [a collaborative Internet-based question and answer service providing support to digital reference (or AskA) services by accepting out-of-scope and overflow questions], but can be applied to all digital reference services and consortia. (See Additional Resources for further discussion of the Facets of Quality.)

Background

The original facets of quality were identified by the 1997 Virtual Reference Desk Expert Panel. The list was based upon panel members' experiences in managing and coordinating exemplary digital reference services. This list has been revised to include results of discussions of the 1999 Expert Panel and October 13, 1999 AskA Consortium Meeting. (See http://vrd.org/training/facets.html for previous version of “Facets of Quality for K-12 Digital Reference Services.”) Revisions were made in June 2003 based on internal review and upon consultation and advice from Jo Bell Whitlatch, Ph.D.

The facets are divided into two main categories: user transaction and service development/management. The user transaction category includes those components that occur during the question-answering process (i.e., accessibility, prompt turnaround, clear response policy, interactive, and instructive); these features are generally visible to the user. The service development/management category involves decisions made in creating and maintaining the service that affect overall quality and user satisfaction (i.e., authoritative, trained experts, private, reviewed, provides access to related information, and publicized).

Levels of Quality

Digital reference services differ from each other in many aspects including policy and procedure, subject matter expertise, and available resources. For this reason, it is important that quality criteria accommodate the wide range of possibilities of various services and their readiness to meet certain expectations. Each "facet" is presented at two different levels:

• Essential - Required for network membership
• Recommendations – current state and goals of membership

To be included in the VRD Network all services must fulfill the essential level of requirements for each facet of quality. The recommendations level represents a wide range of possibilities and reflects the current state and goals to which all AskA services in the network will strive to achieve over time.

The following section presents each facet of quality, its definition, and its levels of participation.

User Transaction

1. Accessible: Digital reference services should be easily reachable and navigable by any Internet user regardless of equipment sophistication, physical disability or language barrier.

   Essential: Ensure that the service is reachable and navigable by either e-mail or Web-based query forms.

   Recommendations:

   • Allow users to contact the service via a Web-based query form.
   • Make reference service information easy to locate on the service's Web site.
   • Maintain compliance with the Americans with Disabilities Act (ADA).¹
   • Include access to the service for non-English speakers by identifying people within the organization who are able to translate non-English user questions.
   • Ensure that the service is easily reachable and navigable.
   • Plugins can be used but should not be required.

2. Prompt Turnaround: Questions should be addressed as quickly as possible. Actual turnaround time depends on a service's question-answer policy and available resources (e.g., staffing, funds, technology, etc.).

   Essential: Respond to at least 10% of incoming questions within two business days.

   Recommendations:

   • Answer at least 10% to 50% of questions within two to five days of receipt.
   • Acknowledge user questions within two to five days of receipt and notify users if they will not receive a response.
   • Allow users to indicate urgency of their requests on a Web-based query form.
   • For asynchronous services, the goal is to respond to 100% of appropriate (in scope) incoming questions within one to two business days.

3. Clear Response Policy: Clear communication should occur either before or at the start of every digital reference transaction in order to reduce opportunities for user confusion and inappropriate inquiries.

   Essential: Create and adhere to a clear response policy.

   Recommendations:

   • State question-answering procedures and services clearly in an accessible place on the service’s Web site or in an acknowledgement message to the user. The statement should indicate question scope, types of answers provided, what patrons can expect from experts regarding time and effort, and expected turnaround time.

4. Interactive: Digital reference services should provide opportunities for an effective reference interview, so that users can communicate necessary information to experts and to clarify vague user questions.

   Essential: Encourage sharing of important user information such as age or grade level and question subject-area either through a Web-based query form or follow-up e-mail communication.

   Recommendations:

   • Capture important information such as the user’s age or grade level, other sources checked, and contact information through Web-based query forms or other interactive communication tools, without compromising user privacy.
   • Provide opportunities for clarification of user questions through follow-up e-email messages or conversations using interactive communications tools.
   • Incorporate a follow-up method, such as assigning tracking numbers to questions, in order to identify related messages.
   • Offer real-time reference interviews or very thorough Web forms to gather as much information as possible without compromising user privacy.
   • Allow users the ability to return to a service for further information to clarify a question if the answer is insufficient within the policy guidelines of each service.
   • Link related question-answer sets using a common protocol to identify related messages to facilitate follow-up.²

5. Instructive: Digital reference services provide access to current information and expertise. Quality digital reference services offer more to users than straight, factual answers; they guide them in how to formulate questions, subject knowledge, and information literacy.

   Essential: Offer answers or pointers to information in responses to users. When unable to provide an answer, provide the user with appropriate notification.

   Recommendations:

   • Offer pointers and paths used to find the best resources, so users can learn to answer similar questions on their own. For example, responses can include the tools used to find resources (e.g., search engines, indexes, bibliographies, catalogs), specific search terms and processes used, and series of steps taken. (Subject specialists who draw upon their own backgrounds and knowledge of a particular topic should describe problem-solving processes and provide additional pointers when possible.)
   • When an answer or resources cannot be found, provide the user with a notice of paths, keywords, and processes that were attempted.
   • Promote information literacy by responding with detailed search paths and sets of resources that either provide the answer or allow users to investigate on their own. Subject experts provide answers drawing upon their own backgrounds and knowledge of a particular topic, and provide additional pointers when appropriate.

Service Development and Management:

6. Authoritative: Experts of a digital reference service should have the necessary knowledge and educational background in the service's given subject area or skill in order to qualify as an expert. Specific levels of knowledge, skill and experience are determined by each service and its related discipline or field.

   Essential: Ensure that the service staff includes at least one qualified expert who can oversee the quality and accuracy of staff responses.

   Recommendations:

   • Communicate to users the authority of the service in general as well as individual experts through an e-mailed acknowledgement or on a Web site.
   • Communicate any possible perspectives that may be reflected in responses to questions.
   • Staff the service with professionals (paid or volunteer) in the subject specialty of the AskA service. In information referral services, staff should consist of professional librarians.
   • Indicate on the Web site qualifications of those who answer questions. Participating organizations will provide evidence of certification for professionals who answer questions.
   • Communicate any possible perspectives that may be reflected in responses to questions.

7. Trained Experts: Services should offer effective orientation or training processes to prepare experts to respond to inquiries using clear and effective language and following service response policies and procedures. Training of information specialists is one of the most important aspects of planning and operating a digital reference service.

   Essential: Provide some training procedure for all AskA service staff, even if informal.

   Recommendations:

   • Provide a training component such as written documents containing response guidelines, opportunities for experts to practice responding to inquiries accompanied by feedback on performance, or mentoring by more experienced staff.
   • Ensure that trainees are practicing appropriate tone, “approachability”, and responsiveness in their replies
   • Ensure that trainees achieve mastery of training objectives.
   • Document each trainee’s participation in the training program for organizations providing academic credit or professional advancement.
   • Document each trainee’s performance during and after training.
   • Communicate to users that experts are trained in the service’s question-answer policies and procedures and that experts receive continuing education for updating and upgrading skills related to information sources, search strategies, and technology.
   • Create training materials and procedures that are well-documented, pilot tested, and revised as needed.
   • Recognize trained experts in some way for their successful completion of the process.
   • Offer credit for successful completion of the training program, whether it is for continuing education, basic coursework, or professional advancement (e.g., through the workplace, the certifying organization, etc.).

8. Private: All communications between users and experts should be held in complete privacy.

   Essential: Deny public access to user names, e-mail or mailing addresses, and questions unless there is written notice prior to the information being made available, such as on a Web-based query form.

   Recommendations:

   • Receive consent from users before sharing transaction data or identifying information (e.g., e-mail address) with a third party (e.g., questions and answers posted in a Web-based archive).
   • Post privacy policies on the Web site, especially the page with the query form.³
   • Establish appropriate privacy policies and make them readily accessible on the Web site.
   • Remove all identifying information from the question-answer sets before posting in a public archive.
   • Obtain parental permission for children under 18 if identifying information is posted publicly.
   • Include information related to the extent of protection available for individuals from harmful surveillance, for both patrons and experts.
   • Advise patrons as to what data is retained by the service.
   • Provide patrons with the opportunity to amend their profiles and questions.

9. Reviewed: Digital reference services should regularly evaluate their processes and services. Ongoing review and assessment help ensure quality, efficiency, and reliability of transactions as well as overall user satisfaction.

   Essential: Periodically monitor responses to users for quality, accuracy, adherence to service policy, and to improve skills.

   Recommendations:

   • Evaluate user and staff satisfaction with the service.
   • Monitor responses either before sending to users or at some point afterwards on a periodic basis. (Provide corrections to responses if needed.)
   • Provide formative feedback to trained experts to remind, teach, and encourage them to adhere to service policies and procedures.
   • Evaluate user and staff satisfaction with the service.
   • Monitor responses and online resources on an ongoing basis.
   • Provide formative feedback to trained experts to remind, teach, and encourage them to adhere to service policies and procedures and the ethical and legal use of resources, such as subscription databases.
   • Collaborate with other AskA services to offer peer review.
   • Run responses against a link-checking application to assure suggested Web sites are valid for the user.

10. Provides access to related information: Besides offering direct response to user questions, digital reference services should offer access to supporting resources and information. Services can reuse results from question-answer exchanges in resources such as archives and frequently-asked questions (FAQs).

    Essential: Include basic resources on the Web site to supplement the question-answering component.

    Recommendations:

    • Provide access to information on the service’s content area, including access to a knowledge base of previously asked questions and answers, or links or references to external resources.
    • Post selection policies on the Web site to indicate criteria for selection of external resources.
    • Review and update Web sites at least 3 times per year to ensure that content is correct and links and references are active.
    • Provide access to information on the service’s content area, including access to a knowledge base of previously asked questions and answers, or links or references to external resources.
    • Post criteria for Web resource selection.
    • Evaluate Web-based resources such as lists of links, FAQs, and other resources before posting, and check resources periodically for accuracy and reliability.
    • Annotate suggested Web links when possible.
    • Identify partnerships within network of collaborating services.

11. Publicize: Services should inform potential users of the value that can be gained from use of the service. A well-defined public relations plan can ensure that services are well-publicized and promoted on a regular basis. Publicity should not create more demand than the service has capacity to handle.

    Essential: Create and post publicly a description of the purpose and practices of the service.

    Recommendations:

    • Create a publicity plan that may include some of the following methods:
      • Promotional message to appropriate electronic discussion groups (e.g., listservs) and on the service Web site
      • Links to the service from related Web sites
      • Direct postal mailings to potential users
      • Articles in print publications and press releases
      • Presentations at conferences, meetings, etc.
    • Publicity should not create an inundation of questions to the point of overwhelming the service.
    • Develop a publicity plan and publicize their service to potential constituents to the degree with which they can manage the resultant traffic to the service. Included in this plan should be an evaluation of publicity for reaching the intended audience.

Conclusion

It is clear that there is a need for standards as digital reference evolves from a handful of AskA services and a set of experiments in libraries to a common means of interacting with users. These facets of quality provide an example of standards of operation for digital reference services in a cross-domain environment. It is hoped that these standards will be adopted in growing reference consortia, or used as a model for the development of new digital reference standards.

Footnotes

2. Sources for determining ADA compliance can be found in the following sites: Bobby 3.2 (CAST) and Accessible Web Page Design.
3. See background information on Question Interchange Profile.
4. A source for assisting with writing privacy policies can be found at TRUSTe.

Additional Resources on Facets of Quality

Bennett, A. B., A. Kasowitz, and R. D. Lankes (2000). Digital reference quality criteria. In R. D. Lankes, J. W. Collins III, and A. S. Kasowitz (Eds.), Digital Reference Service in the New Millennium: Planning, Management, and Evaluation (pp. 69-80). New York: Neal-Schuman Publishers.

Kasowitz, A., B. Bennett, and R. D. Lankes (2000). Quality standards for digital reference consortia. Reference & User Services Quarterly, 39(4), 355-63.

Revised 06/27/03