You are here
Research Areas
Education policy and program impact
Full Abstract
To make up for pandemic-related learning losses, many U.S. public school districts have increased enrollment in their summer school programs. We assess summer school as a strategy for COVID-19 learning recovery by tracking the academic progress of students who attended summer school in 2022 across eight districts serving 400,000 students. Based on students’ spring to fall progress, we find a positive impact for summer school on math test achievement (0.03 standard deviation, SD), but not on reading tests. These effects are predominantly driven by students in upper elementary grades. To put the results into perspective, if we assume that these districts have losses similar to those present at the end of the 2022–23 school year (i.e., approximately -0.2 SD), we estimate summer programming closed approximately 2% to 3% of the districts’ total learning losses in math, but none in reading.
Citation: Ian Callen, Maria V. Carbonari, Michael DeArmond, Daniel Dewey, Elise Dizon-Ross, Dan Goldhaber, Jazmin Isaacs, Thomas J. Kane, Megan Kuhfeld, Anna McDonald, Andrew McEachin, Emily Morton, Atsuko Muroga, Douglas O. Staiger (2023). Summer School as a Learning Loss Recovery Strategy After COVID-19: Evidence From Summer 2022. CALDER Working Paper No. 291-0823
Full Abstract
Reading has been at the forefront of early-grade educational interventions, but addressing the educational needs of students in math early on is also critical given that early gaps in math skills widen further over the course of schooling. In this study, we examine the effects of Kentucky’s Math Achievement Fund – a unique state-level program that combines targeted interventions, peer-coaching, and close collaboration among teachers to improve math achievement in grades K-3 – on student outcomes and the costs associated with this policy. We find significant positive effects of the program not only on math achievement, but also on test scores in reading and non-test outcomes including student attendance and disciplinary incidents. The benefits exist across racial/ethnic groups and students from different socioeconomic statuses, and they are slightly higher for racial minorities. These findings, along with the cost estimate of the program, suggest that this program could provide a cost-effective blueprint to address the educational needs of students in math in early grades.
Citation: Zeyu Xu, Umut Özek, Jesse Levin, Dong Hoon Lee (2023). Effects of Large-Scale Early Math Interventions on Student Outcomes: Evidence from Kentucky’s Math Achievement Fund. CALDER Working Paper No. 279-0323
Full Abstract
In this paper we examine academic recovery in 12 mid- to large-sized school districts across 10 states during the 2021–22 school year. Our findings highlight the challenges that recovery efforts faced during the 2021–22 school year. Although, on average, math and reading test score gains during the school year reached the pace of pre-pandemic school years, they were not accelerated beyond that pace. This is not surprising given that we found that districts struggled to implement recovery programs at the scale they had planned. In the districts where we had detailed data on student participation in academic interventions, we found that recovery efforts often fell short of original expectations for program scale, intensity of treatment, and impact. Interviews with a subsample of district leaders revealed several implementation challenges, including difficulty engaging targeted students consistently across schools, issues with staffing and limitations to staff capacity, challenges with scheduling, and limited engagement of parents as partners in recovery initiatives. Our findings on the pace and trajectory of recovery and the challenges of implementing recovery initiatives raise important questions about the scale of district recovery efforts.
Citation: Maria V. Carbonari, Miles Davison, Michael DeArmond, Daniel Dewey, Elise Dizon-Ross, Dan Goldhaber, Ayesha Hashim, Thomas J. Kane, Andrew McEachin, Emily Morton, Tyler Patterson, Douglas O. Staiger (2022). The Challenges of Implementing Academic COVID Recovery Interventions: Evidence from the Road to Recovery Project. CALDER Working Paper No. 275-1222
Full Abstract
Taking advanced courses in high school predicts a broad array of positive postsecondary and labor market outcomes. Yet students from historically disadvantaged groups and low-income backgrounds have long been underrepresented in these courses. To address this problem, more than 60 districts in Washington state implemented a policy that automatically enrolled all qualified high school students in advanced coursework. The policy relied on a simple behavioral nudge: It made advanced courses “opt out” rather than “opt in” for all qualified students. The districts implemented the policy in waves, beginning in the 2014–15 school year. In this descriptive paper, we examine enrollment patterns by comparing districts that adopted the policy at different times. We found that students in districts that implemented the policy between 2014–15 and 2016–17 were more likely to enroll in at least one advanced course in any subject relative to students in districts without the policy. This was the case for students who “qualified” for advanced courses based on their test scores and for students whose scores did not qualify them for advanced courses. We also found that the policy was associated with a higher probability of enrollment in advanced mathematics courses but only for qualified students. Qualified students across demographic groups experienced similar changes in the probability of advanced course enrollments. But among all students—regardless of qualified status—enrollments in advanced mathematics and advanced English language arts/social studies courses increased more for students from racial/ethnic groups underrepresented in advanced courses and for students who were eligible for free or reduced-price lunch (FRPL) than for non-underrepresented students and students not eligible for FRPL. These across the board increases in advanced course enrollment for students who were historically underrepresented in these courses suggests that districts may have looked beyond standardized assessment scores to identify students for automatic enrollment.
Citation: Megan Austin, Benjamin Backes, Dan Goldhaber, Dory Li, Francie Streich (2022). Leveling Up: A Behavioral Nudge to Increase Enrollment in Advanced Coursework. CALDER Working Paper No. 271-1022
Full Abstract
Measures of student disadvantage—or risk—are critical components of equity-focused
education policies. However, the risk measures used in contemporary policies have significant limitations, and despite continued advances in data infrastructure and analytic capacity, there has been little innovation in these measures for decades. We develop a new measure of student risk for use in education policies, which we call Predicted Academic Performance (PAP). PAP is a flexible, data-rich indicator that identifies students at risk of poor academic outcomes. It blends concepts from emerging “early warning” systems with principles of incentive design to balance the competing priorities of accurate risk measurement and suitability for policy use. PAP is more effective than common alternatives at identifying students who are at risk of poor academic outcomes and can be used to target resources toward these students—and students who belong
to several other associated risk categories—more efficiently.
Citation: Ishtiaque Fazlul, Cory Koedel, Eric Parsons (2022). Using Predicted Academic Performance to Identify At-Risk Students in Public Schools. CALDER Working Paper No. 261-0922
Full Abstract
CALDER Policy Brief No. 16-0519
The Community Eligibility Provision (CEP) is a recently-implemented policy change to the federally-administered National School Lunch Program (NSLP). It allows schools and districts serving low-income populations to identify all students as eligible for free lunch, regardless of students’ individual circumstances. The purpose of the CEP is to expand meal access to students who attend low income schools, while at the same time reducing paperwork and streamlining the process of participating in the NSLP. An unintended consequence of the CEP is that it reduces the informational content of NSLP-based measures of student poverty because all students at CEP schools are classified as eligible for free lunch. This has important implications for school accountability and finance policies at all levels of government as these policies have become highly dependent on the use of free and reduced-price lunch (FRL) data to proxy for student disadvantage.
Citation: Cory Koedel, Stacey Preis (2019). The Community Eligibility Provision, Free and Reduced-Price Lunch, and Measurement of Poverty: Implications for Education Policy. CALDER Policy Brief No. 16
Full Abstract
More than one out of eight students have an identified disability, and students with disabilities tend to lag behind their typical peers on a variety of outcomes. It has been well established that teacher quality is an important determinant of student success, and there are persistent shortages of special education teachers. The available evidence suggests that, on average, students with disabilities are not being assigned to teachers of different quality in comparison to students without disabilities. However, there are many challenges to studying the quality of the teachers who instruct special education students. As a result, we know relatively little about the quality of special education teachers and what factors determine special education teacher quality. One existing study suggests that the determinants of teacher effectiveness for students with disabilities may be rather different than for teachers of nondisabled students; certification status and advanced degree attainment are positively correlated with a teacher’s ability to increase achievement for students with disabilities, but not so for the general student population. Given the difficulty that districts face in hiring and retaining special education teachers, more research on special education teacher quality would be valuable when assessing potential policies such as recruiting bonuses, loan forgiveness, or differential pay.
CALDER Policy Brief No. 11-0918-1
Citation: Curran Prettyman, Tim Sass (2018). Teacher Quality and Outcomes for Students with Disabilities. CALDER Policy Brief No. 11
Full Abstract
CALDER Policy Brief No. 8-0918-1
Citation: Steven Hemelt, Matthew Lenard (2018). Career Academies and the Resurgence of Career and Technical Education in the United States. CALDER Policy Brief No. 8
Full Abstract
CALDER Policy Brief No. 7-0918-1
Citation: Umut Özek (2018). The Effects of Instruction Time on Student Outcomes. CALDER Policy Brief No. 7
Full Abstract
CALDER Policy Brief No. 6-0918-1
Citation: Roddy Theobald (2018). Career and Technical Education for Students with Disabilities. CALDER Policy Brief No. 6
Full Abstract
CALDER Policy Brief No. 4-0918-2
Citation: Carrie Conaway, Dan Goldhaber (2018). Appropriate Standards of Evidence for Education Policy Decision-Making. CALDER Policy Brief No. 4
Full Abstract
Instructional time is a fundamental educational input, yet we have little causal evidence about the effect of longer school days on student achievement. This paper uses a sharp regression discontinuity design to estimate the effects of lengthening the school day for low-performing schools in Florida by exploiting an administrative cutoff for eligibility. Our results indicate significant positive effects of additional literacy instruction on student reading achievement. In particular, we find effects of 0.05 standard deviations of improvement in reading test scores for program assignment in the first year, though long-run effects are difficult to assess.
Citation: David Figlio, Kristian Holden, Umut Ozek (2018). Do Students Benefit from Longer School Days? Regression Discontinuity Evidence from Florida's Additional Hour of Literacy Instruction. CALDER Working Paper No. 201-0818-1
Full Abstract
This paper examines the value of strategically assigning disproportionately larger classes to the strongest teachers in order to optimize student learning in the face of differential teacher effectiveness. The rationale is straightforward: Larger classes for the best teachers benefit the pupils who are reassigned to them; they also help the less effective teachers improve their instruction by enabling them to concentrate on fewer students. But just how much of a difference could manipulating class sizes in this way make for overall student learning and access to effective teaching? This study performs a simulation based on North Carolina data to estimate plausible student outcomes under this approach. In the North Carolina data, I find there is a very slight tendency to place more students in the classes of effective teachers; but still only about 25 percent of students are taught by the top 25 percent of teachers. Intensively reallocating eighth-grade students—so that the most effective teachers have up to twelve more pupils than the average classroom—may produce gains equivalent to adding roughly two-and-a-half extra weeks of school. Even adding a handful of students to the most effective eighth-grade teachers (up to six more than the school’s average) produces gains in math and science akin to extending the school year by nearly two weeks or, equivalently, to removing the lowest 5 percent of teachers from the classroom. The potential impacts on learning are more modest in fifth grade, where the large majority of teachers are in self-contained classrooms. Results show that this strategy shows an overall improvement in student access to effective teaching, yet gaps in access for economically disadvantaged students persist. For instance, disadvantaged eighth-grade students are about 8 percent less likely than non-disadvantaged peers to be assigned to a teacher in the top 25 percent of performance. This gap in access changes little in spite of the policy putting more students in front of effective teachers — because the pool of available teachers in high-poverty schools does not change under this strategy. Thus, this policy alone shows little promise in reducing achievement gaps.
Citation: Michael Hansen (2014). Right-Sizing the Classroom: Making the Most of Great Teachers. CALDER Working Paper No. 110
Full Abstract
In 2002/03, the Charlotte-Mecklenburg Schools initiated a broad program of accelerating entry into algebra coursework. The proportion of moderately-performing students taking 8th grade algebra increased from less than half to nearly 90%, then reverted to baseline levels, in the span of just six age cohorts. We use this policy-induced variation to infer the impact of accelerated entry into algebra on student performance in math courses as students progress through high school. Students affected by the acceleration initiative scored significantly lower on end-of-course tests in Algebra I, and were either no more likely or significantly less likely to pass standard follow-up courses, Geometry and Algebra II, on a college-preparatory timetable. We also find that the district assigned teachers with weaker qualifications to Algebra I classes in the first year of the acceleration, but this reduction in teacher quality accounts for only a small portion of the overall effect.
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor (2012). The Aftermath of Accelerating Algebra: Evidence from a District Policy Initiative. CALDER Working Paper No. 69