A Comprehensive Picture of Achievement Across the COVID-19 Pandemic Years: Examining Variation in Test Levels and Growth Across Districts, Schools, Grades, and Students

In this paper, we use NWEA MAP test data to examine variation in students' achievement and growth during the pandemic across multiple dimensions. Consistent with prior evidence, we find that students' test scores in fall 2021, on average, were substantially below historic averages. Moreover, the average scores of students of color, students attending high poverty schools, and students in elementary school were more negatively impacted, and more so in math than reading. We present novel evidence on the distributions of test scores and growth in fall 2021 relative to pre-pandemic distributions, finding disproportionately larger declines for students with lower previous achievement levels across districts. However, between districts, there was considerable variation in the extent to which their fall 2021 achievement and growth distributions shifted from their historical distributions by subject, student subgroups, and baseline achievement levels. Therefore, accurately targeting students and choosing interventions for pandemic-related recovery will require careful assessment by districts of their students' achievement and growth in the 2021-22 school year (and into the future): assuming that students in a district reflect the national trends of achievement will often lead to incorrect conclusions about the degree to which they suffered pandemic-related learning losses and the amount of support they will need to recover.