You are here
A sizeable body of evidence has documented the effectiveness of Teach For America (TFA) corps members at raising the mathematics test scores of their students, though little is known about the program’s impact at the school level. TFA’s recent placement strategy in the Miami-Dade County Public Schools, in which large numbers of TFA corps members are placed as clusters into a targeted set of disadvantaged schools, provides an opportunity to evaluate the impact of the TFA program on broader school performance. This study examines whether the influx of TFA corps members led to a spillover effect on other teachers’ performance. We find that many of the schools chosen to participate in the cluster strategy experienced large subsequent gains in mathematics achievement. These gains were driven in part by the composition effect of having larger numbers of effective TFA corps members. However, we do not find any evidence that the clustering strategy led to any spillover effect on schoolwide performance. In other words, our estimates suggest that the extra student gains for TFA corps members under the clustering strategy would be equivalent to gains resulting from an alternate placement strategy in which corps members were evenly distributed across schools.
Revised August 31, 2015
Citation: Michael Hansen, Benjamin Backes, Victoria Brady, Zeyu Xu (2015). Examining Spillover Effects from Teach For America Corps Members in Miami- Dade County Public Schools. CALDER Working Paper No. 113
There is increased policy interest in extending test-based evaluations in K-12 education to include student achievement in high school. High school achievement is typically measured by performance on end-of-course exams (EOCs), which test course-specific standards in a variety of subjects. However, unlike standardized tests in the early grades, students take EOCs at different points in their schooling careers. The timing of the test is a choice variable presumably determined by input from administrators, students and parents. Recent research indicates that school and district policies that determine when students take particular courses can have important consequences for achievement and subsequent outcomes like advanced course taking. We develop an approach for modeling EOC test performance that disentangles the influence of school and district policies regarding the timing of course taking from other factors. After separating out the timing issue, better measures of the quality of instruction provided by districts, schools and teachers can be obtained. Our approach also offers diagnostic value because it separates out the influence of school and district course-timing policies from other factors that determine student achievement.
Citation: Eric Parsons, Cory Koedel, Michael Podgursky, Mark Ehlert , P. Brett Xiang (2015). Incorporating End-of-Course Exam Timing into Educational Performance Evaluations. CALDER Working Paper No. 137
This study uses detailed administrative data on teachers and students from the state of North Carolina to revisit the empirical evidence on master’s degrees, with attention to teachers at the middle and high school levels. It provides descriptive information on which types of teachers obtain master’s degrees, for which subjects, at which institutions, and during what phase of their career. The study estimates returns to master’s degrees using teacher fixed effects to control for time-invariant characteristics of teachers, thus separating the effects of teacher decisions to get an advanced degree from the effects of having one. Even with this careful attention to selection bias, we confirm the findings of prior studies showing that teachers with master’s degrees are no more effective than those without. The only consistently positive effect of attaining a master’s degree emerging from this study relates not to student test scores but rather to lower student absentee rates in middle school.
Citation: Helen Ladd, Lucy C. Sorensen (2015). Do Master’s Degrees Matter? Advanced Degrees, Career Paths, and the Effectiveness of Teachers. CALDER Working Paper No. 136
We investigate patterns of teacher mobility in districts with different collective bargaining agreement (CBA) transfer provisions. We use detailed teacher-level longitudinal data from Washington State to estimate the probability that teachers of varying experience and effectiveness levels transfer out of their schools to other schools in the district, to other districts, or out of Washington kindergarten through 12th grade (K–12) public schools. We find consistent evidence that within-district transfer probabilities increase for veteran teachers with the proportion of disadvantaged students in a school but decrease for novice teachers with the proportion of disadvantaged students, and that the strength of these relationships is associated with the strength of seniority transfer provisions in CBAs. Specifically, the pattern of veteran teachers’ leaving disadvantaged schools and novice teachers’ staying in disadvantaged schools is more pronounced in districts with strong CBA seniority transfer protections. CBA transfer provisions do not, however, appear to be an important factor in teacher transfers out of school districts or the K–12 public school workforce in Washington. Finally, we find some evidence that more effective teachers are more likely to stay in advantaged schools when seniority is not a factor in transfer decisions.
Citation: Dan Goldhaber, Lesley Lavery, Roddy Theobald (2015). Inconvenient Truth? Do Collective Bargaining Agreements Help Explain the Mobility of Teachers Within School Districts?. CALDER Working Paper No. 135
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine the evolution of the charter school sector in North Carolina between 1999 and 2012 through the lens of a market model. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, the quality of the match between parental preferences in charter schools relative to the quality of match in traditional public schools, and the distribution of test score performance across charter schools relative those in traditional public schools serving similar students over time. Taken together, our findings imply that the charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools.
Citation: Helen Ladd, Charles Clotfelter, John B. Holbein (2015). The Growing Segmentation of the Charter School Sector in North Carolina. CALDER Working Paper No. 133
One consequence of the Great Recession is that teacher layoffs occurred at a scale previously unseen. In this paper we assess the effects of receiving a layoff notice on teacher mobility using data from Los Angeles and Washington State. We find strong evidence that the receipt of a layoff notice increases the likelihood that teachers leave their schools, even in the absence of actually losing their position due to a layoff. Placebo tests suggest that it is the layoff process that induces “structural churn” rather than differential mobility of the teachers who are targeted by this process.
Citation: Dan Goldhaber, Katharine O. Strunk, Nate Brown, David S. Knight (2015). Lessons Learned from the Great Recession: Layoffs an the RIF-Induced Teacher Shuffle. CALDER Working Paper No. 129
This study provides a first look at how student college- and career-readiness have progressed in the early years of the Common Core State Standards (CCSS) implementation. It is motivated by concern that changes triggered by the standards transition might be disruptive to student learning in the short run, even when those changes may become beneficial once fully implemented. Using longitudinal administrative data from Kentucky, an early adopter of the CCSS, we followed three cohorts of students from the end of the 8th grade to the end of the 11th grade and found that students exposed to the CCSS—including students in both high- and low-poverty schools—made faster progress in learning than similar students who were not exposed to the standards. Although it is not conclusive whether cross-cohort improvement was entirely attributable to the standards reform, we found that students made large gains in proficiency in the years immediately before and after the transition. Additionally, we found student performance in subjects that adopted CCSS-aligned curriculum framework experienced larger, more immediate improvement than student performance in subjects that carried over last-generation curriculum framework.
Citation: Zeyu Xu, Kennan Cepa (2015). Getting College and Career Ready During State Transition Toward the Common Core State Standards. CALDER Working Paper No. 127
In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also has the unintended effect, however, of increasing the number of student misbehaviors such as suspensions, fights, and offenses reportable to law enforcement. Further, this negative response is most pronounced among minorities and low performing students, who are the most likely to be left behind. iii
Citation: John B. Holbein, Helen Ladd (2015). Accountability Pressure and Non-Achievement Student Behaviors. CALDER Working Paper No. 122
The relatively low status of teaching as a profession is often given as a factor contributing to the difficulty of recruiting teachers, the middling performance of American students on international assessments, and the well-documented decline in the relative academic ability of teachers through the 1990s. Since the turn of the 21st century, however, a number of federal, state, and local teacher accountability policies have been implemented toward improving teacher quality over the objections of some who argue the policies will decrease quality. In this paper we analyze 25 years of data on the academic ability of teachers in New York State and document that since 1999 the academic ability of both individuals certified and those entering teaching has steadily increased. These gains are widespread and have resulted in a substantial narrowing of the differences in teacher academic ability between high and low poverty schools and between white and minority teachers. We interpret these gains as evidence that the status of teaching is improving.
Citation: Hamilton Lankford, Susanna Loeb, Andrew McEachin, Luke Miller, James Wyckoff (2015). Who Enters Teaching? Encouraging Evidence that the Status of Teaching is Improving. CALDER Working Paper No. 124
Evidence suggests that teacher hiring in public schools is ad hoc and often fails to result in good selection among applicants. Some districts use structured selection instruments in the hiring process, but we know little about the efficacy of such tools. In this paper, we evaluate the ability of applicant selection tools used by the Spokane Public Schools to predict three outcomes: measures of teachers’ value-added contributions to student learning, teacher absence behavior, and attrition rates. We observe all applicants to the district and are therefore able to estimate sample selection-corrected models, using random tally errors in selection instruments and differences in the quality of competition across job postings. These two factors influence the probability of being hired by Spokane Public Schools but are unrelated to measures of teacher performance. We find that the screening instruments predict teacher value added in student achievement and teacher attrition but not teacher absences. A onestandard- deviation increase in screening scores is associated with an increase of between 0.03 and 0.07 standard deviations in student achievement and a decrease in teacher attrition of 2.5 percentage points.
Citation: Dan Goldhaber, Cyrus Grout, Nick Huntington-Klein (2014). Screen Twice, Cut Once: Assessing the Predictive Validity of Teacher Selection Tools. CALDER Working Paper No. 120
In this paper, we present a closer look at the student achievement trends in the District of Columbia between 2006-07 and 2012-13. We have three main conclusions. First, we find that overall, math scores in the District have improved. The improvements in reading scores during this time frame, however, were primarily limited to the first year after the PERAA implementation. While almost all student subgroups have experienced test score gains in math, these improvements were higher among the more affluent black and Hispanic students. Second, we find that these observed trends in math scores persist even after controlling for the cross-cohort differences in observed student characteristics. In particular, the estimates indicate that less than 10 percent of the year-to-year improvements in test scores can be attributed to the changing student composition in the District over this time frame. Finally, we show that existing students have also experienced gains in math even though the students who are new to the District’s public school system score at higher levels on standardized tests when compared to existing students.
Citation: Umut Özek (2014). A Closer Look at the Student Achievement Trends in the District of Columbia between 2006-07 and 2012-13. CALDER Working Paper No. 119
Teacher and principal evaluation systems now emerging in response to federal, state and/or local policy initiatives typically require that a component of teacher evaluation be based on multiple performance metrics, which must be combined to produce summative ratings of teacher effectiveness. Districts have utilized three common approaches to combine these multiple performance measures, all of which introduce bias and/or additional prediction error that was not present in the performance measures originally. This paper investigates whether the bias and error introduced by these approaches erodes the ability of evaluation systems to reliably identify high- and low-performing teachers. The analysis compares the expected differences in long-term teacher value-added among teachers identified as high- or low-performing under these three approaches, using simulated data based on estimated inter-correlations and reliability of measures in the Gates Foundation’s Measures of Effective Teaching project. Based on the results of our simulation exercise presented here, we conclude these approaches can undermine the evaluation system’s objectives in some contexts. Depending on the way these performance measures are actually combined to categorize teacher performance, the additional error and bias can be large enough to undermine the district’s objectives.
Citation: Michael Hansen, Mariann Lemke, Nicholas Sorensen (2014). Combining Multiple Performance Measures: Do Common Approaches Undermine Districts’ Personnel Evaluation Systems?. CALDER Working Paper No. 118
This paper describes teacher tenure reforms first enacted by the New York City Department of Education (NYCDOE) during the 2009-10 academic year (AY) and the changes in the district’s teacher workforce following the reforms. We show that the reforms dramatically changed the proportion of eligible teachers receiving tenure, as well as the career paths of early career teachers, more generally.
Citation: Susanna Loeb, Luke Miller, James Wyckoff (2014). Performance Screens for School Improvement: The Case of Teacher Tenure Reform in New York City. CALDER Working Paper No. 115
This study examines the multi-faceted public school choice environment in the District of Columbia and the effects of alternative public schools on the achievement levels of students who exercise this type of school choice. The results indicate that students who attend out-of-boundary public schools and charter schools significantly outperform similar students who attend in-boundary public schools in both reading and math tests. We rely on instrumental variables framework to disentangle the underlying reasons behind this achievement gap and find that the observed differences are likely due to the positive effects of alternative public schools.
Citation: Austin Nichols, Umut Özek (2014). Public School Choice and Student Achievement in the District of Columbia. CALDER Working Paper No. 53
Most studies that have fueled alarm over the attrition and mobility rates of teachers have relied on proxy indicators of teacher quality, even though these proxies correlate only weakly with student performance. This paper examines the attrition and mobility of early-career teachers of varying quality using value-added measures of teacher performance. Unlike previous studies, this paper focuses on the variation in these effects across the effectiveness distribution. On average, more effective teachers tend to stay in their initial schools and in teaching. But the lowest performing teachers, who are generally the most likely to transfer between schools, appear to "churn" within the system, and teacher mobility appears significantly affected by student demographics and achievement levels.
Citation: Dan Goldhaber, Betheny Gross, Daniel Player (2014). Teacher Career Paths, Teacher Quality, and Persistence in the Classroom: Are Schools Keeping their Best?. CALDER Working Paper No. 29
Teacher pension systems target retirements within a narrow range of the career cycle by penalizing individuals who separate too soon or remain employed too long. The penalties result in the retention of some teachers who would otherwise choose to leave, and the premature exit of some teachers who would otherwise choose to stay. We examine how the effects of teachers' pension incentives on workforce composition influence teacher quality. Teachers who are held in by the "pull" incentives in the pension systems are not more effective, on average, than the typical teacher. Teachers who are encouraged to exit by the "push" incentives are more effective on average. We conclude that the net effect of teachers' pension incentives on workforce quality is small, but negative. Given the substantial and growing costs of current systems, and the lack of evidence regarding their efficacy, experimentation by traditional and charter schools with alternative retirement benefit structures would be useful.
Citation: Cory Koedel, Michael Podgursky (2014). Teacher Pension Systems, the Composition of the Teaching Workforce, and Teacher Quality. CALDER Working Paper No. 72
This paper studies the pension preferences of Washington State public school teachers by examining two periods of time during which teachers were able to choose between enrolling in a traditional defined benefit plan and a hybrid plan with defined benefit and defined contribution components. Our findings suggest that a large share of teachers are willing to transfer from a traditional DB plan to a hybrid pension plan, and that the probability that a teacher will choose to transfer is related to financial incentives and factors related to risk preferences. There is evidence that more effective teachers are more likely to enroll in the hybrid pension plan, suggesting that states could reduce the financial risk associated with strict defined benefit pension systems without sacrificing the desirability of pension plans to employees.
Citation: Dan Goldhaber, Cyrus Grout (2014). Which Plan to Choose? The Determinants of Pension System Choice for Public School Teachers. CALDER Working Paper No. 111
This paper examines the value of strategically assigning disproportionately larger classes to the strongest teachers in order to optimize student learning in the face of differential teacher effectiveness. The rationale is straightforward: Larger classes for the best teachers benefit the pupils who are reassigned to them; they also help the less effective teachers improve their instruction by enabling them to concentrate on fewer students. But just how much of a difference could manipulating class sizes in this way make for overall student learning and access to effective teaching? This study performs a simulation based on North Carolina data to estimate plausible student outcomes under this approach. In the North Carolina data, I find there is a very slight tendency to place more students in the classes of effective teachers; but still only about 25 percent of students are taught by the top 25 percent of teachers. Intensively reallocating eighth-grade students—so that the most effective teachers have up to twelve more pupils than the average classroom—may produce gains equivalent to adding roughly two-and-a-half extra weeks of school. Even adding a handful of students to the most effective eighth-grade teachers (up to six more than the school’s average) produces gains in math and science akin to extending the school year by nearly two weeks or, equivalently, to removing the lowest 5 percent of teachers from the classroom. The potential impacts on learning are more modest in fifth grade, where the large majority of teachers are in self-contained classrooms. Results show that this strategy shows an overall improvement in student access to effective teaching, yet gaps in access for economically disadvantaged students persist. For instance, disadvantaged eighth-grade students are about 8 percent less likely than non-disadvantaged peers to be assigned to a teacher in the top 25 percent of performance. This gap in access changes little in spite of the policy putting more students in front of effective teachers — because the pool of available teachers in high-poverty schools does not change under this strategy. Thus, this policy alone shows little promise in reducing achievement gaps.
Citation: Michael Hansen (2014). Right-Sizing the Classroom: Making the Most of Great Teachers. CALDER Working Paper No. 110
We examine the efficiency implications of imposing proportionality in teacher evaluation systems. Proportional evaluations force comparisons to be between equally-circumstanced teachers. We contrast proportional evaluations with global evaluations, which compare teachers to each other regardless of teaching circumstance. We consider a policy where administrators use the ratings from the evaluation system to help shape the teaching workforce, and define efficiency in terms of student achievement. Our analysis indicates that proportionality can be imposed in teacher evaluation systems without efficiency costs under a wide range of evaluation and estimation conditions. Proportionality is efficiency-enhancing in some cases. These findings are notable given that proportional teacher evaluations offer a number of other policy benefits.
Citation: Cory Koedel, Jiaxi Li (2014). The Efficiency Implications of Using Proportional Evaluations to Shape the Teaching Workforce. CALDER Working Paper No. 106
Measures of teachers’ “value added” to student achievement play an increasingly central role in k-12 teacher policy and practice, in part because they have been shown to predict teachers’ long-term impacts on students’ life outcomes. However, little research has examined variation in the long-term effects of teachers with similar value-added performance. In this study, we investigate variation in the persistence of teachers’ value-added effects on student achievement in New York City. We separate persistent effects into general effects that improve both the subject taught (math or English language arts (ELA)) and the other area of measured achievement and subject-specific effects which improve only the subject taught. Two findings emerge. First, a teacher’s value-added to ELA achievement has substantial crossover effects on long-term math performance. That is, having a better ELA teacher affects both math and ELA performance in a future year. Conversely, math teachers have only minimal long-term effects on ELA performance; their effects are far more subject-specific. Second, we identify substantial heterogeneity in the persistence of English Language Arts (ELA) teachers’ effects across observable student, teacher, and school characteristics. In particular, teachers in schools serving more poor, minority, and previously low-scoring students have less persistence than other teachers with the same value-added scores. Moreover, ELA teachers with stronger academic backgrounds have more persistent effects on student achievement, as do schools staffed with a higher proportion of such teachers. The results indicate that teachers’ effects on students’ long-term skills can vary as a function of instructional content and quality in ways that are not fully captured by value-added measures of teacher effectiveness.
Citation: Ben Master, Susanna Loeb, James Wyckoff (2014). Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge. CALDER Working Paper No. 104