REALM Project Recruiting for Evaluator
The purpose of this RFP is to solicit competitive proposals from individuals or agencies to serve as independent evaluator of the IMLS-funded project, REopening Archives, Libraries, and Museums (REALM): A COVID-19 Research Partnership, as described below.
OVERVIEW OF PROJECT
OCLC is working on a project funded by the Institute of Museum and Library Services (IMLS), “REopening Archives, Libraries, and Museums (REALM): A COVID-19 Research Partnership.”
Project activities began on April 22, 2020, and will continue through September 2021. The goal of this project is to draw upon scientific research and conduct laboratory work to produce, widely disseminate, and periodically update designed to reduce the risk of transmission of COVID-19 through libraries, archives, and museums (LAMs) to their staff and visitors. Information about the project is available at https://oc.lc/realm-project. OCLC has contracted Battelle Memorial Institute to conduct the scientific research, and is working with an Executive Project Steering Committee, Scientific Working Group, and Operations Working Group with representatives from across the field of libraries, archives, and museums.
EVALUATION REQUIREMENTS
The evaluator will, in consultation and collaboration with project staff, document and execute a plan that will provide a formative evaluation of the project over its three phases and a summative evaluation of the project’s near-term outcomes. The deliverables will include formative evaluations in Phase 1 and 2, and a final evaluation report at the conclusion of the project. Aspects of the evaluation that should be considered for the strategy and framework are distributed over the three phases of the project. Data collection plans must adhere to the clearance requirements of the Paperwork Reduction Act.
Phase 1 (May – August 2020)
The focus of the Phase 1 formative evaluation will be to assess the following:
- communication among OCLC, Battelle, IMLS, Steering Committee, and Working Groups;
- clarity among those stakeholders about which research questions are in scope and why;
- the project’s capacity to reach and suitably communicate to a broad spectrum of archives, libraries, and museum staff and their allied organizations;
- impact of shifting external conditions, such as adjusted public health guidelines or government-issued timelines.
The evaluation consultant will also prepare for formative evaluation of toolkit resources that will take place in Phase 2.
Phase 2 (June – October 2020)
The formative evaluation in Phase 2 will continue to provide iterative feedback to inform any incremental changes to the project design or activities. The independent evaluation consultant will continue to observe project structures and activities and review documents; a particular focus will be on how the project manages the expanded scope of research described in this phase, while retaining stakeholder clarity about what research questions are in scope. Phase 2 evaluation will also assess audience reaction to a selection of toolkit resources created thus far; this assessment will take the form of a brief survey and/or focus groups with representatives from across the various LAM sectors.
Phase 3 (October 2020 – September 2021)
A Phase 3 evaluation strategy and collection methods will be determined at the start of this phase, in collaboration with the evaluation consultant, based on the emergent learning from the previous phases and a more current understanding of what activities and outputs will be part of this phase. This planning process will include exploring whether there are near-term outcomes that could be measured through a summative evaluation.
In addition to the data collection methods conducted by the evaluation consultant, OCLC will track and report basic metrics about the project’s reach and engagement.
PROJECT BUDGET & TIMELINE
The total opportunity for program evaluation and measurement will be a maximum of $80,000.00, to include activities in the June 2020 through September 2021 timeframe.
CRITERIA FOR AWARD
Responses will be evaluated on the following points: how well the individual/organization meets the needs of the project, price, experience working on projects in libraries, archives and museums, and the qualifications of the project team.
SUBMISSIONS
Applications should be submitted via email to Nancy Lensenmayer, [email protected] by June 19, 2020 at 8:00 pm Eastern. Please include the following information in your response:
- names and resumes of key members of the team who would support the project
- narrative response outlining the approach to creating and supporting a formative evaluation
- pricing for evaluation activities, including rates of team members
- links to, or examples of, similar final work product that can demonstrate writing and presentation skill
Any questions regarding the RFP can be submitted via email to Nancy Lensenmayer, [email protected] by June 16, 2020 at 8:00 pm Eastern. Any submitted questions will be publicly answered by June 17, 2020 at 5:00 pm Eastern and posted to this page.
QUESTIONS RECEIVED
Will be updated through June 17, 2020.
Q1. The RFP describes Phase 1 as lasting from April-August and Phase 2 in June-October. Have there been any changes or updates to the project timeline? And is there a (tentative) release date for the toolkit?
A1. There have been no updates or changes to the timeline yet. There is no firm release date for the toolkit, but components of the toolkit will be released incrementally as they are developed.
Q2. How often do the working groups and steering committee meet? Would it be possible to join in/observe these meetings as part of the evaluation?
A2. The project steering committee and working groups are currently meeting online weekly, but that is expected to move to once every two weeks as the project progress. It will be possible to view the recordings of the meetings. Attending the meetings will have to be discussed.
Q3. Do you have an estimate of approximately how many LAMs may be involved in assessing the toolkit during Phase 2?
A3. This has not been determined, but our mailing list of interested LAM professionals is already over 6,400 subscribers. Feedback from the evaluator on a representative sample size will be solicited.
Q4. How do the project steering committee, working groups, and others exchange information? If they meet, how often?
A4. The project steering committee and working groups are currently meeting online weekly (three separate meetings), but that is expected to move to once every two weeks in the near future. They provide consultation and subject matter expertise to the project; they are not decision-making bodies.
Q5. What is the function of the working group?
A5. There is a Scientific Working Group which looks to articulate and advance the scientific basis for the remediation of COVID-19 among the libraries, archives and museums fields, as well as offering their scientific expertise and support. They also identify, review and help validate materials for research. An Operations Working Group brings their knowledge of workflows, services, and spaces that are impacted by the pandemic, which contextualizes the scientific research into real-world library, archives, and museums settings.
Q6. Is there a dedicated project manager for REALM?
A6. Yes, IMLS, OCLC and Battelle all have project managers for the activities. OCLC is charged with driving the project schedule and deliverables.
Q7. Does Phase 3 encompass planning the summative evaluation, or planning and carrying out the evaluation?
A7. Summative evaluation plan should be developed during Phase 2, to be executed in Phase 3.
Q8. Knowing the relative risks of the virus on materials such as buckram cloth, paper, etc is the crucial component of REALM’s work. Has the project team gathered any information on the public’s perception of the safety these items? Are there plans to do so? (As reopening will require understanding of public perception and effectively communicating the relative safety and/or decontamination.)
A8. Assessing public perception of the materials is out of scope for this project. But, we do anticipate that the project results will be shared with the public by libraries, archives and museums seeking to inform the public about their policies and procedures around COVID-19.
Q9. What is the structure and cadence of Steering Committee and Work Group meetings? Have the Work Groups already convened?
A9. The Steering Committee and Working Groups are currently meeting weekly. It is expected that these will transition to bi-weekly when the volume of work/decisions which need input are reduced.
Q10. Is the membership generally new (convened specifically for REALM) or do they have a longer history of working together?
A10. The groups are convened specifically for REALM, but many of the participants have engaged professionally in the past.
Q11. Has the REALM timeline (and subsequently the evaluation timeline) shifted at all?
A11. No, we are on track with the schedule at this time.
Q12. Are you anticipating a summative or formative evaluation for Phase 3?
Yes, Phase 3 evaluation strategy and collection methods will be determined at the start of this phase, in collaboration with the evaluation consultant, based on the emergent learning from the previous phases and a refined definition of the activities and outputs that will be part of this phase. This planning process will include identifying any near-term outcomes that can be measured through a summative evaluation.
Q13. Are there prior projects or evaluations this scope is modeled after?
A13. No.