You are here

Home » The Road to COVID Recovery

The Road to COVID Recovery

Actionable Research on District Strategies for Student Advancement

 

In the wake of instructional challenges faced over the past two school years, districts across the country are currently making important decisions about which interventions and strategies to implement to aid with COVID recovery. Understanding which interventions work best for students, and how to best implement them, is critical as we collectively move forward. Researchers from CALDER at the American Institutes for Research (AIR), the Center for Education Policy and Research (CEPR) at Harvard, and NWEA are partnering with a coalition of districts across the country to help determine which COVID recovery interventions are working (or not working) and why. We aim to maximize the potential of this research to practically inform each district’s recovery efforts, as well as offer insights to the larger field.

Objectives

  1. Learn how various recovery strategies (e.g., summer enrichment programs, Saturday school, tutoring, etc.), implemented in different ways, are affecting student achievement in districts across the country;
  2. Provide rapid and practical feedback to practitioners in participating NWEA school districts so they can make appropriate adjustments to those strategies; and
  3. Inform the field more broadly about the efficacy of district efforts to help students recover academically from the COVID-19 pandemic. These data are important to gather now, as recovery from the pandemic will likely be a multi-year process. And, this project affords participating districts the opportunity to determine how well their recovery efforts are working in the moment, and, when appropriate, make mid-year adjustments to their recovery strategies. 

For more information or to get involved, please contact recovery@nwea.org. 

Publications

Impacts of Academic Recovery Interventions on Student Achievement in 2022-23, CALDER Working Paper

Pandemic Learning Loss by Student Baseline Achievement: Extent and Sources of Heterogeneity, CALDER Working Paper

Summer School as a Learning Loss Recovery Strategy After COVID-19: Evidence From Summer 2022, CALDER Working Paper

The Impact and Implementation of Academic Interventions During COVID: Evidence from the Road to Recovery Project, CALDER Working Paper

A Comprehensive Picture of Achievement Across the COVID-19 Pandemic Years: Examining Variation in Test Levels and Growth Across Districts, Schools, Grades, and Students, CALDER Working Paper

The Consequences of Remote and Hybrid Instruction During the Pandemic, CALDER Working Paper

In the News

Has summer school helped in the fight against pandemic learning loss?, KUOW NPR Network

PROOF POINTS: Summer school programs too short and not popular enough to reverse pandemic learning loss, researchers say, The Hechinger Report

Research Notes: Summer Learning’s Impact on Academic Recovery, FutureEd

Pandemic-era summer school boosts math scores, but barely makes a dent in steep learning loss, Chalkbeat

What Two New Studies Reveal About Learning Recovery, EducationWeek

The Pandemic's Toll on Academic Growth Wasn't Uniform. Recovery Efforts Can't Be Either, EducationWeek

Kids Are Far, Far Behind in School, The Atlantic

Events

Webinar: Where are School Districts on the Road to Academic Recovery? What Comes Next?

PAST EVENT: February 15, 2023, 12:00 pm PT

Third in a series of CALDER webinars on Academic Recovery & Acceleration

Webinar:  The Road to COVID Recovery:  How Districts are Seizing the Once-in-a-Generation Opportunity to Learn from ESSER Interventions

PAST EVENT: November 4, 2021, 9-10 am PT

Second in a series of CALDER webinars on Academic Recovery & Acceleration

Webinar:  Academic Recovery & Acceleration: District and State Interventions for Monitoring Implementation & Effectiveness

PAST EVENT: August 02, 2021,10-11 am PT

Research Partners

The Road to COVID Recovery project is jointly led by CALDER at the American Institutes for Research (AIR), Center for Education Policy Research (CEPR) at Harvard University, and NWEA.

About CALDER at the American Institutes for Research

Established in 1946, the American Institutes for Research (AIR) is a nonpartisan, not-for-profit organization that conducts behavioral and social science research and delivers technical assistance both domestically and internationally in the areas of education, health and the workforce. AIR’s work is driven by its mission to generate and use rigorous evidence that contributes to a better, more equitable world. With headquarters in Arlington, Virginia, AIR has offices across the U.S. and abroad. For more information, visit www.air.org. The National Center for the Analysis of Longitudinal Data in Education Research (CALDER) is a joint effort of AIR and scholars at eleven universities. CALDER research uses longitudinal data from multiple states and districts to examine how policies and practices impact student outcomes over a number of years. CALDER researchers mobilize quickly to study how current and recent events affect real-time educational and social outcomes. For more information, visit CalderCenter.org.

The Road to COVID Recovery is part of AIR's Equity Initiative. Learn more about AIR's commitment to equity at https://www.air.org/air-equity-initiative-bridge-more-equitable-world.

About the Center for Education Policy Research at Harvard University

The Center for Education Policy Research at Harvard University, based at the Harvard Graduate School of Education, seeks to transform education through quality research and evidence. CEPR and its partners believe all students will learn and thrive when education leaders make decisions using facts and findings, rather than untested assumptions. Learn more at cepr.harvard.edu.

About NWEA

NWEA®, a division of HMH, is a mission-driven organization that supports students and educators in more than 146 countries through research, assessment solutions, policy and advocacy services, professional learning and school improvement services that fight for equity, drive classroom impact and push for systemic change in our educational communities. For more than 40 years, NWEA has been a pioneer in educational research and assessment methodology with a focus on improving learning outcomes for every student. Our research explores foundational issues in education, practical challenges in today’s schools, and the evolving role of technology in the lives of students. Visit NWEA.org to learn more about how we’re partnering with educators to help all kids learn.

For more information or to get involved, please contact recovery@nwea.org. 

Project Overview

The first phase of this project involves a coalition of large, urban, and suburban districts that use MAP® Growth™ assessments from NWEA and have systems in place to identify student participation in COVID recovery programs and interventions. 

  1. Responding to the Need -- April 2020 - August 2021
    When the COVID-19 pandemic disrupted schooling across the country for sustained periods of time in 2020, it was clear student learning would be impacted dramatically. Using MAP data from across the nation, NWEA began projecting and studying the impact of the pandemic on students’ reading and math achievement and growth. As described in two reports issued in 2021, NWEA found significant impacts on student outcomes, especially in mathematics, and also showed that the effect was most pronounced for students of color and students in high-poverty schools.

    Shortly thereafter, the federal Elementary & Secondary School Emergency Relief (ESSER) funds provided $190 billion in education funding to states and school districts, $110 billion of which was designated for addressing “learning loss” due to pandemic-related schooling impacts, using “evidence-based interventions.” Researchers at the Center for the Analysis of Longitudinal Data in Education Research (CALDER), along with CALDER’s Policymakers Council, issued a Policy Brief in support of districts and states taking early steps to learn from this unprecedented federal investment. 

  2. Engage Coalition of the Willing -- June - October 2021
    A subset of districts already partnering with NWEA to administer the MAP Growth assessments were approached by researchers from the three organizations to share their COVID recovery plans and determine if they were interested in joining with other districts to lead a national effort to learn about the efficacy of their catch up strategies. Researchers sought to identify a group of districts that represented a broad range of student demographics, district sizes, and geographic locations, and were implementing a number of different programs and interventions specifically targeted at COVID recovery efforts. 
  3. Initial District Convening - September 2021
    Researchers from the three organizations convened the district leaders in late September 2021 to meet one another, begin to identify COVID catch-up programs and interventions, and review what to expect over the course of the year. Participants included superintendents, central office leaders in instruction, out-of-school time, and research/assessment.
  4. Develop Data Collection Instruments - August 2021
    Based on the information districts shared about the programs they were offering, researchers identified the types of data that would be needed in order to examine the efficacy of interventions. 
  5. Execute Data Sharing Agreements & Protecting Student Privacy - June - November 2021
    Ensuring student privacy was a critical requirement of this project. NWEA has executed (or will execute) data sharing agreements with each participating district, and has obtained (or will obtain) approval from the Institutional Review Board, a formal group that is required by the federal government to review and monitor research involving human subjects.
  6. District Leader Interviews -- September - November 2021
    Leaders from each district have participated (or will participate) in 90-minute virtual interviews with project researchers. The focus of the interviews was to identify the major academic COVID recovery initiatives each district was offering in grades 3-8 in the 2021-22 school year. In particular, interviews focused on new or expanded programs offered throughout the district or targeting specific groups of students (e.g., summer enrichment, additional classroom instructional blocks/extended class time, weekend academies, out-of-school time programs, virtual learning, tutoring, etc.).

    For each identified intervention, researchers asked district leaders about the program and student eligibility (i.e., were programs offered district wide or to specific schools and/or students), participation, program content, providers, modality, dosage/frequency, and student attendance.

    In advance of each interview, researchers sent district leaders the interview template including all of the questions that they would ask, so they could be prepared to answer to the best of their ability and have staff participate in the interview who had working knowledge of the interview content. 

    During the interview, researchers took notes in real time using a Google Document that was visible to all participants via shared screen mode in the virtual meeting. 

    In some cases, a second interview was conducted, when it was not possible to complete all questions in the interview during the time allotted, or when an additional district leader needed to speak with researchers. After the interview(s), researchers sent a draft of the interview notes, as well as any follow-up questions, to district leaders to review and issue changes as they saw fit.

  7. Collect Student Attendance Data - September - May 2022
    Starting at the beginning of the school year, districts began or continued tracking student attendance in each program. 
  8. Convene to Discuss Recovery Strategies - October 2021
    Districts will convene to discuss the recovery strategies and programs districts are using across the coalition, as well as gather feedback on the types of reports and analysis that will be helpful. 
  9. Practice Data Upload - October 2021
    Districts will practice uploading student-level attendance data and student roster data, which included subgroup identification (e.g., free/reduced meal status, students with disabilities, English learners, and race/ethnicity). The purpose of the practice upload was to identify any barriers to uploading data in the format necessary to analyze student learning for each intervention.
  10. Convene to Discuss Fall Data Reports - December 2021
    After fall 2021 MAP testing is complete, NWEA will provide a report to each district on how student achievement in grades 3-8 is trending relative to students across the nation and in comparison to students in their district in prior years. The report will also include comparisons of trends for the district as a whole and for each school and student subgroup in the district.

    Districts will convene to discuss findings from the Fall Report. 

  11. Convene to Discuss Course-Corrections - January 2022
    Districts will convene with researchers to share and discuss any programmatic changes districts are making related to COVID recovery efforts for the second semester.
  12. Convene to Discuss Winter Data Reports - March-May 2022
    After winter 2022 MAP testing is complete, researchers will provide a report to each district showing how the different COVID recovery strategies districts are implementing and their characteristics are related to students’ academic recovery relative to the other similar districts participating in the study.

    Districts will convene to discuss findings from the Winter Report.

  13. Spring Data Convening & Reports
    After spring 2022 MAP testing is complete, researchers will provide a report to each district following up to the winter report analyses.

    Districts will convene to discuss findings from the Spring Report

  14. Public Report on aggregate academic recovery strategies used and students’ academic progress in different types of interventions across the districts.

Project Deliverables

Over the course of this school year, we plan to consult with districts on a series of reports that examine MAP Growth outcomes across various different COVID recovery strategies and interventions. The first set of reports will be specifically tailored to participating districts and will not be shared outside approved district personnel. Additional report(s), which will be made publicly available (while protecting district information), will provide an overview of the academic recovery strategies used and students’ academic progress in different types of interventions across the coalition of districts participating in the study. These reports will use information from participating districts, but no individual districts will be identified. 

District Convenings

For the 2021-22 school year, virtual district convenings will be held as follows:

  • October - Review of Recovery Strategies: Discuss the recovery strategies and programs districts are using across the coalition, gather district feedback on the types of reports and analyses that would be helpful.

  • December - Fall MAP Growth Results: Discuss findings from the fall report

  • January - Updates to Recovery Strategies: Discuss any changes to districts’ recovery strategies and programs for the second semester

  • March-May - Winter MAP Growth Results: Discuss findings from the winter report

  • July-August - Spring MAP Growth Results: Discuss findings from the spring report

 

For more information or to get involved, please contact recovery@nwea.org. 

Frequently Asked Questions

What is the cost to participate?

This project is internally and philanthropically funded at no cost to participating districts. The goal is to provide actionable information to districts to inform their short- and long-term COVID recovery strategies. 

What data are participating districts collecting?

To learn how interventions impact student learning, districts need to collect, track, and securely share:

  • The new or expanded interventions they are implementing for COVID recovery, and how each intervention is implemented (e.g., type of intervention, how students are identified, subject matter focus, grade level, duration, modality, etc.)

  • Individual student participation/attendance for each intervention

  • Individual student MAP Growth assessment results for fall, winter, and spring

  • Individual student demographics

Who should I contact if I want more information about this project or if my district is interested in participating?

Please contact recovery@nwea.org. 

Which districts are participating?

Given that the project is in its early stages, we are not currently sharing the names of the school systems/districts that are in this project. If those districts give us permission to do so, we will make that information available at a future point. 

Logo of American Institutes for Research. Advancing Evidence. Improving Lives.
Logo of Center for Education Policy Research, Harvard University
Logo of NWEA