You are here
Research Areas
Educational accountability
Full Abstract
The success of many students with disabilities (SWDs) depends on access to high-quality general education teachers. Yet, most measures of teacher value-added measures (VAM) fail to distinguish between a teacher’s effectiveness in educating students with and without disabilities. We create two VAM measures: one focusing on teachers’ effectiveness in improving outcomes for SWDs, and one for non-SWDs. We find top-performing teachers for non-SWDs often have relatively lower VAMs for SWDs, and that SWDs sort to teachers with lower scores in both VAMs. Overall, SWD-specific VAMs may be more suitable for identifying which teachers have a history of effectiveness with SWDs and could play a role in ensuring that students are being optimally assigned to these teachers.
Citation: W. Jesse Wood, Ijun Lai, Neil R. Filosa, Scott A. Imberman, Nathan Jones, Katharine O. Strunk (2022). Are Effective Teachers for Students with Disabilities Effective Teachers for All?. CALDER Working Paper No. 268-0622
Full Abstract
In this paper, we use NWEA MAP test data to examine variation in students’ achievement and growth during the pandemic across multiple dimensions. Consistent with prior evidence, we find that students’ test scores in fall 2021, on average, were substantially below historic averages. Moreover, the average scores of students of color, students attending high poverty schools, and students in elementary school were more negatively impacted, and more so in math than reading. We present novel evidence on the distributions of test scores and growth in fall 2021 relative to pre-pandemic distributions, finding disproportionately larger declines for students with lower previous achievement levels across districts. However, between districts, there was considerable variation in the extent to which their fall 2021 achievement and growth distributions shifted from their historical distributions by subject, student subgroups, and baseline achievement levels. Therefore, accurately targeting students and choosing interventions for pandemic-related recovery will require careful assessment by districts of their students’ achievement and growth in the 2021-22 school year (and into the future): assuming that students in a district reflect the national trends of achievement will often lead to incorrect conclusions about the degree to which they suffered pandemic-related learning losses and the amount of support they will need to recover.
Citation: Dan Goldhaber, Thomas J. Kane, Andrew McEachin, Emily Morton (2022). A Comprehensive Picture of Achievement Across the COVID-19 Pandemic Years: Examining Variation in Test Levels and Growth Across Districts, Schools, Grades, and Students. CALDER Working Paper No. 266-0522
Full Abstract
Using testing data from 2.1 million students in 10,000 schools in 49 states (plus D.C.), we investigate the role of remote and hybrid instruction in widening gaps in achievement by race and school poverty. We find that remote instruction was a primary driver of widening achievement gaps. Math gaps did not widen in areas that remained in-person (although there was some widening in reading gaps in those areas). We estimate that high-poverty districts that went remote in 2020-21 will need to spend nearly all of their federal aid on academic recovery to help students recover from pandemic-related achievement losses.
Citation: Dan Goldhaber, Thomas J. Kane, Andrew McEachin, Emily Morton, Tyler Patterson, Douglas O. Staiger (2022). The Consequences of Remote and Hybrid Instruction During the Pandemic. CALDER Working Paper No. 267-0522
Full Abstract
We present new estimates of the importance of teachers in early grades for later grade outcomes, but unlike the existing literature that examines teacher “fade-out,” we directly compare the contribution of early-grade teachers to later year outcomes against the contributions of later year teachers to the same later year outcomes. Where the prior literature finds that much of the contribution of early teachers fades away, we find that the contributions of early-year teachers remain important in later grades. The difference in contributions to eighth-grade outcomes between an effective and ineffective fourth-grade teacher is about half the difference among eighth-grade teachers. The effect on eighth-grade outcomes of replacing a fourth-grade teacher who is below the 5th percentile with a median teacher is about half the underrepresented minority (URM)/non-URM achievement gap. Our results reinforce earlier conclusions in the literature that teachers in all grades are important for student achievement.
Citation: Dan Goldhaber, Zeyu Jin, Richard Startz (2022). How Much Do Early Teachers Matter?. CALDER Working Paper No. 264-0422
Full Abstract
In this study, we use microdata from 12 Florida county-level school districts to examine the effects of early grade retention on the short-, medium-, and long-term outcomes of English learners in a regression discontinuity design. We find that retention in the third grade coupled with instructional support substantially improves the English skills of these students, reducing the time to proficiency by half and decreasing the likelihood of taking a remedial English course in middle school by one-third. Grade retention also roughly doubles the likelihood of taking an advanced course in math and science in middle school, and triples the likelihood of taking college credit-bearing courses in high school for English learners. We do not find any adverse effects of the policy on disciplinary problems or absences among English learners.
CALDER WP 211-0119-1 was originally released in January 2019. An updated version was released in March 2020.
Citation: David Figlio, Umut Özek (2019). An Extra Year to Learn English? Early Grade Retention and the Human Capital Development of English Learners. CALDER Working Paper No. 211-0119-1
Full Abstract
CALDER Policy Brief No. 12-1118-1
Citation: Dan Goldhaber, Umut Özek (2018). How Much Should We Rely on Student Test Achievement as a Measure of Success?. CALDER Policy Brief No. 12
Full Abstract
Educational accountability policies are a popular tool to close the achievement gaps between advantaged and disadvantaged students. However, these policies may exacerbate inequality if families from advantaged backgrounds are better able to advocate for their children and thus circumvent policy. We investigate this possibility in the context of the early grade retention policy in Florida, which requires all students with reading skills below grade level to be retained in the third grade, yet grants exemptions under special circumstances. We find that Florida’s third-grade retention policy is in fact enforced differentially depending on children’s socioeconomic background, especially maternal education. Holding exemption eligibility constant, scoring right below the promotion cutoff results increases the retention probability 14 percent more for children whose mothers have less than a high school degree as compared to children whose mothers have a bachelor’s degree or more. We also find that the discrepancies in retention rates are mainly driven by the fact that students with well-educated mothers are more likely to be promoted based on subjective exemptions such as teacher portfolios.
Citation: Christina LiCalsi, Umut Özek, David Figlio (2016). The Uneven Implementation of Universal School Policies: Maternal Education and Florida’s Mandatory Grade Retention Policy. CALDER Working Paper No. 167
Full Abstract
Studies of the charter sector typically compare charters and traditional public schools at a point in time. These comparisons are potentially misleading because many charter-related reforms require time to generate results. We study quality dynamics among Texas charter schools from 2001-2011. School quality in the charter sector was initially highly variable and on average lower than traditional public schools. However, exits, improvement of existing charter schools, and expansion of higher-performing charter management organizations increased charter effectiveness relative to traditional public schools. We present evidence that reduced student mobility and an increased share of charters adhering to No Excuses- style curricula contribute to these improvements.
Citation: Patrick Baude, Marcus Casey, Eric Hanushek, Steven Rivkin (2016). The Evolution of Charter School Quality. CALDER Working Paper No. 159
Full Abstract
This paper examines the effect of school turnaround in North Carolina elementary and middle schools. Using a regression discontinuity design, we find that turnaround led to a drop in average school-level math and reading passing rates and an increased concentration of low-income students in treated schools. We use teacher survey data to examine how teacher activities changed. Treated schools brought in new principals and increased the time teachers devoted to professional development. The program also increased administrative burdens and distracted teachers, potentially reducing time available for instruction. Teacher turnover increased after the first full year of implementation. Overall, we find little success for North Carolina’s efforts to turn around low-performing schools under its federally funded Race to the Top grant.
Citation: Helen Ladd, Jennifer A. Heissel (2016). School Turnaround in North Carolina: A Regression Discontinuity Analysis. CALDER Working Paper No. 156
Full Abstract
One of the concerns over high-stakes testing is the incentive for teachers to alter the scores of their students. We investigate the effects of teacher cheating on subsequent student achievement, attendance, behavior and educational attainment. We find that test scores drop below expected levels in the first year post-cheating year. These effects persist for reading and ELA, but not for math. The drop in later test scores appears to be due in part to a reduction in access to remediation services. We also find some evidence that cheated middle-school students may be more likely to drop out of high school.
Citation: Jarod Apperson, Carycruz Bueno, Tim Sass (2016). Do the Cheated Ever Prosper? The Long-Run Effects of Test-Score Manipulation by Teachers on Student Outcomes. CALDER Working Paper No. 155
Full Abstract
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine the evolution of the charter school sector in North Carolina between 1999 and 2012 through the lens of a market model. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, the quality of the match between parental preferences in charter schools relative to the quality of match in traditional public schools, and the distribution of test score performance across charter schools relative those in traditional public schools serving similar students over time. Taken together, our findings imply that the charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools.
Citation: Helen Ladd, Charles Clotfelter, John B. Holbein (2015). The Growing Segmentation of the Charter School Sector in North Carolina. CALDER Working Paper No. 133
Full Abstract
In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also has the unintended effect, however, of increasing the number of student misbehaviors such as suspensions, fights, and offenses reportable to law enforcement. Further, this negative response is most pronounced among minorities and low performing students, who are the most likely to be left behind. iii
Citation: John B. Holbein, Helen Ladd (2015). Accountability Pressure and Non-Achievement Student Behaviors. CALDER Working Paper No. 122
Full Abstract
This study examines the multi-faceted public school choice environment in the District of Columbia and the effects of alternative public schools on the achievement levels of students who exercise this type of school choice. The results indicate that students who attend out-of-boundary public schools and charter schools significantly outperform similar students who attend in-boundary public schools in both reading and math tests. We rely on instrumental variables framework to disentangle the underlying reasons behind this achievement gap and find that the observed differences are likely due to the positive effects of alternative public schools.
Citation: Austin Nichols, Umut Özek (2014). Public School Choice and Student Achievement in the District of Columbia. CALDER Working Paper No. 53
Full Abstract
A critical element in the sustainability of any public policy is the fair treatment of ‘similar’ individuals. This paper introduces a new dimension of merit to evaluate public school assignment mechanisms based on this notion of horizontal equity. The findings reveal that all of the prominent assignment mechanisms discussed in the literature fail to satisfy this ‘equal treatment’ criterion. I also show that there exists no student-optimal stable mechanism that also satisfies equal treatment, illustrating the tradeoff between constrained efficiency and horizontal equity. These findings surface a serious cause for concern about the public school assignment procedures used in major school districts.
Citation: Umut Özek (2013). Equal Treatment as a Means of Evaluating Public School Assignment Mechanisms. CALDER Working Paper No. 99
Full Abstract
Current federal policy emphasizes a focus on turning around schools that consistently fail to serve their students and communities. At first glance, the identification of chronically low-performing schools and successful turnarounds may seem straightforward. Characterizing school performance, however, requires resolving multiple dilemmas about schools that have not been previously addressed within the literature on turnaround. Furthermore, the literature lacks empirical evidence on the frequency of turnaround in these low-performing schools. This paper addresses these issues involved in identifying low performance and turnaround among schools. Additionally, it examines the long-term performance trajectories of chronically low-performing (CLP) elementary and middle schools in multiple states to identify schools that have shown rapid improvement (designated turn around [TA] schools), schools that have shown moderate improvement (MI), and schools that are persistently not improving (NI). The findings indicate school turnaround is an uncommon event in these low-performing schools, though not rare—approximately 10 to 30 percent of CLP schools, depending on the state and school level, are identified as TA schools based on improvements in performance.
Citation: Michael Hansen, Kilchan Choi (2013). Chronically Low-performing Schools and Turnaround: Evidence from Three State. CALDER Working Paper No. 60
Full Abstract
Test-based accountability has become the new norm in public education over the last decade. In many states and school districts nationwide, student performance on standardized tests plays an important role in high-stakes decisions such as grade retention. This study examines the effects of grade retention on student misbehavior in Florida, which requires students with reading skills below grade level to be retained in the 3rd grade. The regression discontinuity estimates suggest that grade retention increases the likelihood of disciplinary incidents and suspensions in the short run, yet these effects dissipate over time. The findings also suggest that these short term adverse effects are concentrated among economically disadvantaged and male students.
Citation: Umut Özek (2013). Hold Back to Move Forward? Early Grade Retention and Student Misbehavior. CALDER Working Paper No. 100
Full Abstract
This paper examines the effects of policies that increase the number of students who take the first course in algebra in 8th grade, rather than waiting until 9th grade. Extending previous research that focused on the Charlotte-Mecklenburg school system, we use data for the 10 largest districts in North Carolina. We identify the effects of accelerating the timetable for taking algebra by using data on multiple cohorts grouped by decile of prior achievement and exploiting the fact that policy-induced shifts in the timing of algebra occur at different times in different districts to different deciles of students. The expanded data make it possible to examine heterogeneity across students in the effect of taking algebra early. We find negative effects among students in the bottom 60% of the prior achievement distribution. In addition, we find other sources of heterogeneity in effect
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor (2013). Algebra for 8th Graders: Evidence on its Effects from 10 North Carolina Districts. CALDER Working Paper No. 87
Full Abstract
We use North Carolina data to explore whether the quality of teachers in the lower elementary grades (K-2) falls short of teacher quality in the upper grades (3-5) and to examine the hypothesis that school accountability pressures contribute to such quality shortfalls. Our concern with the early grades arises from recent studies highlighting how children’s experiences in those years have lasting effects on their later outcomes. Using two credentials-based measures of teacher quality, we document within-school quality shortfalls in the lower grades, and show that the shortfalls increased with the introduction of No Child Left Behind. Consistent with that pattern, we find that schools responded to accountability pressures by moving their weaker teachers down to the lower grades and stronger teachers up to the higher grades. These findings support the view that accountability pressure induces schools to pursue actions that work to the disadvantage of children in the lower grades.
Citation: Sarah C. Fuller, Helen Ladd (2013). School Based Accountability and the Distribution of Teacher Quality Across Grades in Elementary Schools. CALDER Working Paper No. 75
Full Abstract
Using longitudinal data on spanning the 2002-03 through 2007-08 school years in Florida and North Carolina, this paper decomposes the workforce dynamics among teachers and principals in low-performing schools that significantly improved their performance. In general, I find strong, consistent evidence of human capital development (i.e., improvements in the productivity of the teachers and principals already in the school) accounting for the increased performance in turnaround schools. These findings are robust to the inclusion of school random effects, alternative categorizations of both teachers and turnaround schools, and are observed across elementary and middle school samples in both states. There is also general evidence of productive incoming teachers helping to improve these turnaround schools, but little evidence to support negative attrition specific to these schools played a role. These findings are important as they document large improvements in the joint productivity of teachers in low-performing schools, a finding which is out of step with current federal efforts to improve schools that implicitly assume teacher productivity is essentially fixed over time.
Citation: Michael Hansen (2013). Investigating the Role of Human Resources in School Turnaround: A Decomposition of Improving Schools in Two States. CALDER Working Paper No. 89
Full Abstract
How to incorporate mobile students, who enter schools/classrooms after the start of the school year, into educational performance evaluations remains to be a challenge. As mandated by the No Child Left Behind Act of 2001 (NCLB), all states currently require that a school is accountable only if the student has been enrolled in the school for a full academic year. This paper investigates the school response to this eligibility requirement in a regression-discontinuity framework. Comparing students who enter schools right before and after the eligibility cutoff, I find no evidence that schools behave strategically in response to this requirement.
Citation: Umut Özek (2012). One Day Too Late? Mobile Students in an Era of Accountability. CALDER Working Paper No. 82