You are here
School choice and educational accountability
We use rich administrative microdata from Missouri to examine the potential to expand and diversify the production of STEM degrees at universities by tapping into the population of community college students. We find that the scope for expansion is modest, even at an upper bound, because most
community college students have academic qualifications that make them unlikely to succeed in a STEM field at a university. We also find there is almost no scope for community college students to improve the racial/ethnic diversity of four-year STEM degree recipients. We conclude that it will be challenging to
expand and diversify STEM degree production at universities with interventions targeted toward community college students.
Citation: Cheng Qian , Cory Koedel (2020). The Potential for Community College Students to Expand and Diversify University Degree Production in STEM Fields . CALDER Working Paper No. 244-1020
In this study, we use microdata from 12 Florida county-level school districts to examine the effects of early grade retention on the short-, medium-, and long-term outcomes of English learners in a regression discontinuity design. We find that retention in the third grade coupled with instructional support substantially improves the English skills of these students, reducing the time to proficiency by half and decreasing the likelihood of taking a remedial English course in middle school by one-third. Grade retention also roughly doubles the likelihood of taking an advanced course in math and science in middle school, and triples the likelihood of taking college credit-bearing courses in high school for English learners. We do not find any adverse effects of the policy on disciplinary problems or absences among English learners.
CALDER WP 211-0119-1 was originally released in January 2019. An updated version was released in March 2020.
Citation: David Figlio, Umut Özek (2019). An Extra Year to Learn English? Early Grade Retention and the Human Capital Development of English Learners. CALDER Working Paper No. 211-0119-1
Instructional time is a fundamental educational input, yet we have little causal evidence about the effect of longer school days on student achievement. This paper uses a sharp regression discontinuity design to estimate the effects of lengthening the school day for low-performing schools in Florida by exploiting an administrative cutoff for eligibility. Our results indicate significant positive effects of additional literacy instruction on student reading achievement. In particular, we find effects of 0.05 standard deviations of improvement in reading test scores for program assignment in the first year, though long-run effects are difficult to assess.
Citation: David Figlio, Kristian Holden, Umut Ozek (2018). Do Students Benefit from Longer School Days? Regression Discontinuity Evidence from Florida's Additional Hour of Literacy Instruction. CALDER Working Paper No. 201-0818-1
Educational accountability policies are a popular tool to close the achievement gaps between advantaged and disadvantaged students. However, these policies may exacerbate inequality if families from advantaged backgrounds are better able to advocate for their children and thus circumvent policy. We investigate this possibility in the context of the early grade retention policy in Florida, which requires all students with reading skills below grade level to be retained in the third grade, yet grants exemptions under special circumstances. We find that Florida’s third-grade retention policy is in fact enforced differentially depending on children’s socioeconomic background, especially maternal education. Holding exemption eligibility constant, scoring right below the promotion cutoff results increases the retention probability 14 percent more for children whose mothers have less than a high school degree as compared to children whose mothers have a bachelor’s degree or more. We also find that the discrepancies in retention rates are mainly driven by the fact that students with well-educated mothers are more likely to be promoted based on subjective exemptions such as teacher portfolios.
Citation: Christina LiCalsi, Umut Özek, David Figlio (2016). The Uneven Implementation of Universal School Policies: Maternal Education and Florida’s Mandatory Grade Retention Policy. CALDER Working Paper No. 167
Studies of the charter sector typically compare charters and traditional public schools at a point in time. These comparisons are potentially misleading because many charter-related reforms require time to generate results. We study quality dynamics among Texas charter schools from 2001-2011. School quality in the charter sector was initially highly variable and on average lower than traditional public schools. However, exits, improvement of existing charter schools, and expansion of higher-performing charter management organizations increased charter effectiveness relative to traditional public schools. We present evidence that reduced student mobility and an increased share of charters adhering to No Excuses- style curricula contribute to these improvements.
This paper examines the effect of school turnaround in North Carolina elementary and middle schools. Using a regression discontinuity design, we find that turnaround led to a drop in average school-level math and reading passing rates and an increased concentration of low-income students in treated schools. We use teacher survey data to examine how teacher activities changed. Treated schools brought in new principals and increased the time teachers devoted to professional development. The program also increased administrative burdens and distracted teachers, potentially reducing time available for instruction. Teacher turnover increased after the first full year of implementation. Overall, we find little success for North Carolina’s efforts to turn around low-performing schools under its federally funded Race to the Top grant.
Citation: Helen Ladd, Jennifer A. Heissel (2016). School Turnaround in North Carolina: A Regression Discontinuity Analysis. CALDER Working Paper No. 156
One of the concerns over high-stakes testing is the incentive for teachers to alter the scores of their students. We investigate the effects of teacher cheating on subsequent student achievement, attendance, behavior and educational attainment. We find that test scores drop below expected levels in the first year post-cheating year. These effects persist for reading and ELA, but not for math. The drop in later test scores appears to be due in part to a reduction in access to remediation services. We also find some evidence that cheated middle-school students may be more likely to drop out of high school.
Citation: Jarod Apperson, Carycruz Bueno, Tim Sass (2016). Do the Cheated Ever Prosper? The Long-Run Effects of Test-Score Manipulation by Teachers on Student Outcomes. CALDER Working Paper No. 155
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine the evolution of the charter school sector in North Carolina between 1999 and 2012 through the lens of a market model. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, the quality of the match between parental preferences in charter schools relative to the quality of match in traditional public schools, and the distribution of test score performance across charter schools relative those in traditional public schools serving similar students over time. Taken together, our findings imply that the charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools.
Citation: Helen Ladd, Charles Clotfelter, John B. Holbein (2015). The Growing Segmentation of the Charter School Sector in North Carolina. CALDER Working Paper No. 133
In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also has the unintended effect, however, of increasing the number of student misbehaviors such as suspensions, fights, and offenses reportable to law enforcement. Further, this negative response is most pronounced among minorities and low performing students, who are the most likely to be left behind. iii
Citation: John B. Holbein, Helen Ladd (2015). Accountability Pressure and Non-Achievement Student Behaviors. CALDER Working Paper No. 122
This study examines the multi-faceted public school choice environment in the District of Columbia and the effects of alternative public schools on the achievement levels of students who exercise this type of school choice. The results indicate that students who attend out-of-boundary public schools and charter schools significantly outperform similar students who attend in-boundary public schools in both reading and math tests. We rely on instrumental variables framework to disentangle the underlying reasons behind this achievement gap and find that the observed differences are likely due to the positive effects of alternative public schools.
Citation: Austin Nichols, Umut Özek (2014). Public School Choice and Student Achievement in the District of Columbia. CALDER Working Paper No. 53
A critical element in the sustainability of any public policy is the fair treatment of ‘similar’ individuals. This paper introduces a new dimension of merit to evaluate public school assignment mechanisms based on this notion of horizontal equity. The findings reveal that all of the prominent assignment mechanisms discussed in the literature fail to satisfy this ‘equal treatment’ criterion. I also show that there exists no student-optimal stable mechanism that also satisfies equal treatment, illustrating the tradeoff between constrained efficiency and horizontal equity. These findings surface a serious cause for concern about the public school assignment procedures used in major school districts.
Citation: Umut Özek (2013). Equal Treatment as a Means of Evaluating Public School Assignment Mechanisms. CALDER Working Paper No. 99
Current federal policy emphasizes a focus on turning around schools that consistently fail to serve their students and communities. At first glance, the identification of chronically low-performing schools and successful turnarounds may seem straightforward. Characterizing school performance, however, requires resolving multiple dilemmas about schools that have not been previously addressed within the literature on turnaround. Furthermore, the literature lacks empirical evidence on the frequency of turnaround in these low-performing schools. This paper addresses these issues involved in identifying low performance and turnaround among schools. Additionally, it examines the long-term performance trajectories of chronically low-performing (CLP) elementary and middle schools in multiple states to identify schools that have shown rapid improvement (designated turn around [TA] schools), schools that have shown moderate improvement (MI), and schools that are persistently not improving (NI). The findings indicate school turnaround is an uncommon event in these low-performing schools, though not rare—approximately 10 to 30 percent of CLP schools, depending on the state and school level, are identified as TA schools based on improvements in performance.
Citation: Michael Hansen, Kilchan Choi (2013). Chronically Low-performing Schools and Turnaround: Evidence from Three State. CALDER Working Paper No. 60
Test-based accountability has become the new norm in public education over the last decade. In many states and school districts nationwide, student performance on standardized tests plays an important role in high-stakes decisions such as grade retention. This study examines the effects of grade retention on student misbehavior in Florida, which requires students with reading skills below grade level to be retained in the 3rd grade. The regression discontinuity estimates suggest that grade retention increases the likelihood of disciplinary incidents and suspensions in the short run, yet these effects dissipate over time. The findings also suggest that these short term adverse effects are concentrated among economically disadvantaged and male students.
Citation: Umut Özek (2013). Hold Back to Move Forward? Early Grade Retention and Student Misbehavior. CALDER Working Paper No. 100
This paper examines the effects of policies that increase the number of students who take the first course in algebra in 8th grade, rather than waiting until 9th grade. Extending previous research that focused on the Charlotte-Mecklenburg school system, we use data for the 10 largest districts in North Carolina. We identify the effects of accelerating the timetable for taking algebra by using data on multiple cohorts grouped by decile of prior achievement and exploiting the fact that policy-induced shifts in the timing of algebra occur at different times in different districts to different deciles of students. The expanded data make it possible to examine heterogeneity across students in the effect of taking algebra early. We find negative effects among students in the bottom 60% of the prior achievement distribution. In addition, we find other sources of heterogeneity in effect
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor (2013). Algebra for 8th Graders: Evidence on its Effects from 10 North Carolina Districts. CALDER Working Paper No. 87
We use North Carolina data to explore whether the quality of teachers in the lower elementary grades (K-2) falls short of teacher quality in the upper grades (3-5) and to examine the hypothesis that school accountability pressures contribute to such quality shortfalls. Our concern with the early grades arises from recent studies highlighting how children’s experiences in those years have lasting effects on their later outcomes. Using two credentials-based measures of teacher quality, we document within-school quality shortfalls in the lower grades, and show that the shortfalls increased with the introduction of No Child Left Behind. Consistent with that pattern, we find that schools responded to accountability pressures by moving their weaker teachers down to the lower grades and stronger teachers up to the higher grades. These findings support the view that accountability pressure induces schools to pursue actions that work to the disadvantage of children in the lower grades.
Citation: Sarah C. Fuller, Helen Ladd (2013). School Based Accountability and the Distribution of Teacher Quality Across Grades in Elementary Schools. CALDER Working Paper No. 75
Using longitudinal data on spanning the 2002-03 through 2007-08 school years in Florida and North Carolina, this paper decomposes the workforce dynamics among teachers and principals in low-performing schools that significantly improved their performance. In general, I find strong, consistent evidence of human capital development (i.e., improvements in the productivity of the teachers and principals already in the school) accounting for the increased performance in turnaround schools. These findings are robust to the inclusion of school random effects, alternative categorizations of both teachers and turnaround schools, and are observed across elementary and middle school samples in both states. There is also general evidence of productive incoming teachers helping to improve these turnaround schools, but little evidence to support negative attrition specific to these schools played a role. These findings are important as they document large improvements in the joint productivity of teachers in low-performing schools, a finding which is out of step with current federal efforts to improve schools that implicitly assume teacher productivity is essentially fixed over time.
Citation: Michael Hansen (2013). Investigating the Role of Human Resources in School Turnaround: A Decomposition of Improving Schools in Two States. CALDER Working Paper No. 89
How to incorporate mobile students, who enter schools/classrooms after the start of the school year, into educational performance evaluations remains to be a challenge. As mandated by the No Child Left Behind Act of 2001 (NCLB), all states currently require that a school is accountable only if the student has been enrolled in the school for a full academic year. This paper investigates the school response to this eligibility requirement in a regression-discontinuity framework. Comparing students who enter schools right before and after the eligibility cutoff, I find no evidence that schools behave strategically in response to this requirement.
Citation: Umut Özek (2012). One Day Too Late? Mobile Students in an Era of Accountability. CALDER Working Paper No. 82
School closures are increasingly common among U.S. public schools, driven by both budgetary constraints and accountability pressures to turnaround low-performing schools. This paper contributes to the nascent literature on school closures by evaluating student achievement and mobility outcomes in a large-scale restructuring effort in Washington, D.C. in which 32 elementary and middle school campuses were closed or consolidated in the summer of 2008. Using longitudinal data, we investigate how student outcomes change in relation to this initiative with an instrumental variables strategy that counters the endogeneity of student assignment across schools before and after the restructuring occurred. The results show that the academic performance of students directly affected by the school restructuring experienced a temporary decline, but it rebounded by the second school year after restructuring occurred. Additionally, we find no evidence of closure adversely inducing further mobility among affected students.
Citation: Umut Özek, Michael Hansen, Thomas Gonzalez (2012). A Leg Up or a Boot Out?: Student Achievement and Mobility under School Restructuring. CALDER Working Paper No. 78
Increasing parental choice has been a leading theme of recent education policy intended to enhance the academic achievement of low-performing students in the United States. These policies aim to “level the playing field” in access to high-quality education for disadvantaged students who cannot otherwise afford higher-quality schooling options. Public school choice programs in D.C. are successful; disadvantaged students are able to attend higher-performing schools than their neighborhood public schools, even with prolonged commutes. Overall, the findings provide evidence that the relatively advantaged students are taking advantage of public school choice programs. However, choice exacerbates student quality disparities between low- and high-poverty schools, casting some doubt on the benefits of such programs.
Citation: Umut Özek (2011). Public School Choice in the District of Columbia: A Descriptive Analysis. CALDER Working Paper No.
Struggling schools that come under increased accountability pressure face a number of challenges, including changing instructional policies and practices to facilitate student improvement. But what effect does school accountability have on teachers’ mobility decisions? This study is the first to exploit policy variation within the same state to examine the effects of school accountability on teacher job changes. Using student-level data from Florida State the authors measure the degree to which schools and teachers were “surprised” by the change in Florida’s school grading system (A+ Plan for Education) in the summer of 2002— what they refer to as an “accountability shock.” They observed the mobility decisions of teachers in the years before and after the school grading change and found over half of all schools in the state experienced an accountability “shock” due to this grading change. Teachers were more likely to leave schools facing increased accountability pressure; even more likely to leave schools shocked downward to a grade of “F”; and less likely to leave schools facing decreased accountability pressure. Schools facing increased accountability pressure also saw a rise in the average quality of the teachers who stayed. If these schools were able to retain more of their high- quality teachers, perhaps through increased incentives to remain in the school, the performance gains associated with school accountability pressure could be greater than those already observed.