Research Areas

In this paper we examine academic recovery in 12 mid- to large-sized school districts across 10 states during the 2021–22 school year. Our findings highlight the challenges that recovery efforts faced during the 2021–22 school year. Although, on average, math and reading test score gains during the school year reached the pace of pre-pandemic school years, they were not accelerated beyond that pace. This is not surprising given that we found that districts struggled to implement recovery programs at the scale they had planned. In the districts where we had detailed data on student participation in academic interventions, we found that recovery efforts often fell short of original expectations for program scale, intensity of treatment, and impact. Interviews with a subsample of district leaders revealed several implementation challenges, including difficulty engaging targeted students consistently across schools, issues with staffing and limitations to staff capacity, challenges with scheduling, and limited engagement of parents as partners in recovery initiatives. Our findings on the pace and trajectory of recovery and the challenges of implementing recovery initiatives raise important questions about the scale of district recovery efforts. 

This is an updated version of CALDER Working Paper 275-1222 titled, "The Challenges of Implementing Academic COVID Recovery Interventions: Evidence from the Road to Recovery Project" originally released in December 2022.

Citation: Maria V. Carbonari, Miles Davison, Michael DeArmond, Daniel Dewey, Elise Dizon-Ross, Dan Goldhaber, Ayesha Hashim, Thomas J. Kane, Andrew McEachin, Emily Morton, Atsuko Muroga, Tyler Patterson, Douglas O. Staiger (2024). The Impact and Implementation of Academic Interventions During COVID: Evidence from the Road to Recovery Project. CALDER Working Paper No. 275-0624-2

While grading has been a topic of research for well over a century, teacher grading standards are receiving increased attention—and with good reason. There is widespread speculation (e.g., Johnson, 2021; Klinger et al., 2022; Mathews, 2022; Walker, 2021) and some evidence (e.g., Sanchez & Moore, 2022, Sanchez, 2023) that grading standards have changed over the course of the pandemic, making higher grades relatively easier to achieve and less reflective of objective measures of learning. It is possible—even likely—that shifting grading standards give parents, guardians, and students a confusing or inaccurate picture of what students know and can do, especially considering pandemic-related learning losses (Dorn et al., 2021; Goldhaber et al., 2023; Kuhfeld et al., 2022). Indeed, recent pieces in the Los Angeles Times, New York Times and The Associated Press raise this alarm, observing that many parents are left in the dark by the lack of information from their children’s schools. Public opinion surveys point to a discrepancy between what parents believe about their student’s level of achievement, i.e., that students have recovered academically, and what test results like NAEP suggest about their achievement (Esquivel, 2022; Kane & Reardon, 2023; Vázquez Tonnes, 2023). 

Despite considerable theoretical work about grading standards and some evidence of changes in grading over the pandemic, we have limited research on the extent to which eased grading standards continued post-pandemic, as teachers and students returned to normal schooling. In this research brief, we use administrative data on student grades from Washington state to assess whether grading standards have returned to pre-pandemic levels.

Citation: Dan Goldhaber, Maia Goodman Young (2023). Course Grades as a Signal of Student Achievement: Evidence on Grade Inflation Before and After COVID-19. CALDER Policy Brief No. 35

Reading has been at the forefront of early-grade educational interventions, but addressing the educational needs of students in math early on is also critical given that early gaps in math skills widen further over the course of schooling. In this study, we examine the effects of Kentucky’s Math Achievement Fund – a unique state-level program that combines targeted interventions, peer-coaching, and close collaboration among teachers to improve math achievement in grades K-3 – on student outcomes and the costs associated with this policy. We find significant positive effects of the program not only on math achievement, but also on test scores in reading and non-test outcomes including student attendance and disciplinary incidents. The benefits exist across racial/ethnic groups and students from different socioeconomic statuses, and they are slightly higher for racial minorities. These findings, along with the cost estimate of the program, suggest that this program could provide a cost-effective blueprint to address the educational needs of students in math in early grades.

Citation: Zeyu Xu, Umut Özek, Jesse Levin, Dong Hoon Lee (2023). Effects of Large-Scale Early Math Interventions on Student Outcomes: Evidence from Kentucky’s Math Achievement Fund. CALDER Working Paper No. 279-0323

Measures of student disadvantage—or risk—are critical components of equity-focused
education policies. However, the risk measures used in contemporary policies have significant limitations, and despite continued advances in data infrastructure and analytic capacity, there has been little innovation in these measures for decades. We develop a new measure of student risk for use in education policies, which we call Predicted Academic Performance (PAP). PAP is a flexible, data-rich indicator that identifies students at risk of poor academic outcomes. It blends concepts from emerging “early warning” systems with principles of incentive design to balance the competing priorities of accurate risk measurement and suitability for policy use. PAP is more effective than common alternatives at identifying students who are at risk of poor academic outcomes and can be used to target resources toward these students—and students who belong
to several other associated risk categories—more efficiently.

Citation: Ishtiaque Fazlul, Cory Koedel, Eric Parsons (2022). Using Predicted Academic Performance to Identify At-Risk Students in Public Schools. CALDER Working Paper No. 261-0922

More than one out of eight students have an identified disability, and students with disabilities tend to lag behind their typical peers on a variety of outcomes. It has been well established that teacher quality is an important determinant of student success, and there are persistent shortages of special education teachers. The available evidence suggests that, on average, students with disabilities are not being assigned to teachers of different quality in comparison to students without disabilities. However, there are many challenges to studying the quality of the teachers who instruct special education students. As a result, we know relatively little about the quality of special education teachers and what factors determine special education teacher quality. One existing study suggests that the determinants of teacher effectiveness for students with disabilities may be rather different than for teachers of nondisabled students; certification status and advanced degree attainment are positively correlated with a teacher’s ability to increase achievement for students with disabilities, but not so for the general student population. Given the difficulty that districts face in hiring and retaining special education teachers, more research on special education teacher quality would be valuable when assessing potential policies such as recruiting bonuses, loan forgiveness, or differential pay.

CALDER Policy Brief No. 11-0918-1

Citation: Curran Prettyman, Tim Sass (2018). Teacher Quality and Outcomes for Students with Disabilities. CALDER Policy Brief No. 11

CALDER Policy Brief No. 7-0918-1

Citation: Umut Özek (2018). The Effects of Instruction Time on Student Outcomes. CALDER Policy Brief No. 7

CALDER Policy Brief No. 4-0918-2

Citation: Carrie Conaway, Dan Goldhaber (2018). Appropriate Standards of Evidence for Education Policy Decision-Making. CALDER Policy Brief No. 4

This paper examines the value of strategically assigning disproportionately larger classes to the strongest teachers in order to optimize student learning in the face of differential teacher effectiveness. The rationale is straightforward: Larger classes for the best teachers benefit the pupils who are reassigned to them; they also help the less effective teachers improve their instruction by enabling them to concentrate on fewer students. But just how much of a difference could manipulating class sizes in this way make for overall student learning and access to effective teaching? This study performs a simulation based on North Carolina data to estimate plausible student outcomes under this approach. In the North Carolina data, I find there is a very slight tendency to place more students in the classes of effective teachers; but still only about 25 percent of students are taught by the top 25 percent of teachers. Intensively reallocating eighth-grade students—so that the most effective teachers have up to twelve more pupils than the average classroom—may produce gains equivalent to adding roughly two-and-a-half extra weeks of school. Even adding a handful of students to the most effective eighth-grade teachers (up to six more than the school’s average) produces gains in math and science akin to extending the school year by nearly two weeks or, equivalently, to removing the lowest 5 percent of teachers from the classroom. The potential impacts on learning are more modest in fifth grade, where the large majority of teachers are in self-contained classrooms. Results show that this strategy shows an overall improvement in student access to effective teaching, yet gaps in access for economically disadvantaged students persist. For instance, disadvantaged eighth-grade students are about 8 percent less likely than non-disadvantaged peers to be assigned to a teacher in the top 25 percent of performance. This gap in access changes little in spite of the policy putting more students in front of effective teachers — because the pool of available teachers in high-poverty schools does not change under this strategy. Thus, this policy alone shows little promise in reducing achievement gaps.

Citation: Michael Hansen (2014). Right-Sizing the Classroom: Making the Most of Great Teachers. CALDER Working Paper No. 110