You are here
Teach For America (TFA) is an alternative certification program that intensively recruits and selects recent college graduates and midcareer professionals to teach in schools serving high-need students. Prior rigorous evaluations of the program have generally found positive effects of TFA teachers on students’ learning in math and science and no significant differences in reading or language arts, compared with non-TFA teachers’ effects in the same schools. No priorstudies, however, have specifically focused on TFA effects in the Atlanta region.
This report examines the efficacy of TFA teachers in the Atlanta region spanning the 2005-06 through 2013-14 school years. Using longitudinal administrative data from three major school districts with significant numbers of recent TFA placements, we generate TFA effect estimates based on two series of Georgia’s standardized tests—the end-of-grade Criterion-Referenced Competency Tests (CRCTs) and end-of-course tests (EOCTs).
We find evidence of a positive effect in student learning due to the hiring of TFA teachers in these three districts, compared with the performance of non-TFA colleagues in the same schools. Estimated TFA effects are positive and statistically significant in social studies and science on the state’s CRCTs, and in American literature on the state’s EOCTs. We find no significant differences in performance between TFA and non-TFA teachers in the other subjects we analyzed. Supplementary analyses show these results are not sensitive to the inclusion of data from a period of well-documented test score manipulation in Atlanta Public Schools.
Citation: Michael Hansen, Tim Sass (2015). Performance Estimates of Teach For America Teachers in Atlanta Metropolitan Area School Districts. CALDER Working Paper No. 145
A sizeable body of evidence has documented the effectiveness of Teach For America (TFA) corps members at raising the mathematics test scores of their students, though little is known about the program’s impact at the school level. TFA’s recent placement strategy in the Miami-Dade County Public Schools, in which large numbers of TFA corps members are placed as clusters into a targeted set of disadvantaged schools, provides an opportunity to evaluate the impact of the TFA program on broader school performance. This study examines whether the influx of TFA corps members led to a spillover effect on other teachers’ performance. We find that many of the schools chosen to participate in the cluster strategy experienced large subsequent gains in mathematics achievement. These gains were driven in part by the composition effect of having larger numbers of effective TFA corps members. However, we do not find any evidence that the clustering strategy led to any spillover effect on schoolwide performance. In other words, our estimates suggest that the extra student gains for TFA corps members under the clustering strategy would be equivalent to gains resulting from an alternate placement strategy in which corps members were evenly distributed across schools.
Revised August 31, 2015
Citation: Michael Hansen, Benjamin Backes, Victoria Brady, Zeyu Xu (2015). Examining Spillover Effects from Teach For America Corps Members in Miami- Dade County Public Schools. CALDER Working Paper No. 113
There is increased policy interest in extending test-based evaluations in K-12 education to include student achievement in high school. High school achievement is typically measured by performance on end-of-course exams (EOCs), which test course-specific standards in a variety of subjects. However, unlike standardized tests in the early grades, students take EOCs at different points in their schooling careers. The timing of the test is a choice variable presumably determined by input from administrators, students and parents. Recent research indicates that school and district policies that determine when students take particular courses can have important consequences for achievement and subsequent outcomes like advanced course taking. We develop an approach for modeling EOC test performance that disentangles the influence of school and district policies regarding the timing of course taking from other factors. After separating out the timing issue, better measures of the quality of instruction provided by districts, schools and teachers can be obtained. Our approach also offers diagnostic value because it separates out the influence of school and district course-timing policies from other factors that determine student achievement.
Citation: Eric Parsons, Cory Koedel, Michael Podgursky, Mark Ehlert , P. Brett Xiang (2015). Incorporating End-of-Course Exam Timing into Educational Performance Evaluations. CALDER Working Paper No. 137
This study uses detailed administrative data on teachers and students from the state of North Carolina to revisit the empirical evidence on master’s degrees, with attention to teachers at the middle and high school levels. It provides descriptive information on which types of teachers obtain master’s degrees, for which subjects, at which institutions, and during what phase of their career. The study estimates returns to master’s degrees using teacher fixed effects to control for time-invariant characteristics of teachers, thus separating the effects of teacher decisions to get an advanced degree from the effects of having one. Even with this careful attention to selection bias, we confirm the findings of prior studies showing that teachers with master’s degrees are no more effective than those without. The only consistently positive effect of attaining a master’s degree emerging from this study relates not to student test scores but rather to lower student absentee rates in middle school.
Citation: Helen Ladd, Lucy C. Sorensen (2015). Do Master’s Degrees Matter? Advanced Degrees, Career Paths, and the Effectiveness of Teachers. CALDER Working Paper No. 136
Evidence suggests that teacher hiring in public schools is ad hoc and often fails to result in good selection among applicants. Some districts use structured selection instruments in the hiring process, but we know little about the efficacy of such tools. In this paper, we evaluate the ability of applicant selection tools used by the Spokane Public Schools to predict three outcomes: measures of teachers’ value-added contributions to student learning, teacher absence behavior, and attrition rates. We observe all applicants to the district and are therefore able to estimate sample selection-corrected models, using random tally errors in selection instruments and differences in the quality of competition across job postings. These two factors influence the probability of being hired by Spokane Public Schools but are unrelated to measures of teacher performance. We find that the screening instruments predict teacher value added in student achievement and teacher attrition but not teacher absences. A onestandard- deviation increase in screening scores is associated with an increase of between 0.03 and 0.07 standard deviations in student achievement and a decrease in teacher attrition of 2.5 percentage points.
Citation: Dan Goldhaber, Cyrus Grout, Nick Huntington-Klein (2014). Screen Twice, Cut Once: Assessing the Predictive Validity of Teacher Selection Tools. CALDER Working Paper No. 120
Teacher and principal evaluation systems now emerging in response to federal, state and/or local policy initiatives typically require that a component of teacher evaluation be based on multiple performance metrics, which must be combined to produce summative ratings of teacher effectiveness. Districts have utilized three common approaches to combine these multiple performance measures, all of which introduce bias and/or additional prediction error that was not present in the performance measures originally. This paper investigates whether the bias and error introduced by these approaches erodes the ability of evaluation systems to reliably identify high- and low-performing teachers. The analysis compares the expected differences in long-term teacher value-added among teachers identified as high- or low-performing under these three approaches, using simulated data based on estimated inter-correlations and reliability of measures in the Gates Foundation’s Measures of Effective Teaching project. Based on the results of our simulation exercise presented here, we conclude these approaches can undermine the evaluation system’s objectives in some contexts. Depending on the way these performance measures are actually combined to categorize teacher performance, the additional error and bias can be large enough to undermine the district’s objectives.
Citation: Michael Hansen, Mariann Lemke, Nicholas Sorensen (2014). Combining Multiple Performance Measures: Do Common Approaches Undermine Districts’ Personnel Evaluation Systems?. CALDER Working Paper No. 118
Most studies that have fueled alarm over the attrition and mobility rates of teachers have relied on proxy indicators of teacher quality, even though these proxies correlate only weakly with student performance. This paper examines the attrition and mobility of early-career teachers of varying quality using value-added measures of teacher performance. Unlike previous studies, this paper focuses on the variation in these effects across the effectiveness distribution. On average, more effective teachers tend to stay in their initial schools and in teaching. But the lowest performing teachers, who are generally the most likely to transfer between schools, appear to "churn" within the system, and teacher mobility appears significantly affected by student demographics and achievement levels.
Citation: Dan Goldhaber, Betheny Gross, Daniel Player (2014). Teacher Career Paths, Teacher Quality, and Persistence in the Classroom: Are Schools Keeping their Best?. CALDER Working Paper No. 29
Teacher pension systems target retirements within a narrow range of the career cycle by penalizing individuals who separate too soon or remain employed too long. The penalties result in the retention of some teachers who would otherwise choose to leave, and the premature exit of some teachers who would otherwise choose to stay. We examine how the effects of teachers' pension incentives on workforce composition influence teacher quality. Teachers who are held in by the "pull" incentives in the pension systems are not more effective, on average, than the typical teacher. Teachers who are encouraged to exit by the "push" incentives are more effective on average. We conclude that the net effect of teachers' pension incentives on workforce quality is small, but negative. Given the substantial and growing costs of current systems, and the lack of evidence regarding their efficacy, experimentation by traditional and charter schools with alternative retirement benefit structures would be useful.
Citation: Cory Koedel, Michael Podgursky (2014). Teacher Pension Systems, the Composition of the Teaching Workforce, and Teacher Quality. CALDER Working Paper No. 72
We examine the efficiency implications of imposing proportionality in teacher evaluation systems. Proportional evaluations force comparisons to be between equally-circumstanced teachers. We contrast proportional evaluations with global evaluations, which compare teachers to each other regardless of teaching circumstance. We consider a policy where administrators use the ratings from the evaluation system to help shape the teaching workforce, and define efficiency in terms of student achievement. Our analysis indicates that proportionality can be imposed in teacher evaluation systems without efficiency costs under a wide range of evaluation and estimation conditions. Proportionality is efficiency-enhancing in some cases. These findings are notable given that proportional teacher evaluations offer a number of other policy benefits.
Citation: Cory Koedel, Jiaxi Li (2014). The Efficiency Implications of Using Proportional Evaluations to Shape the Teaching Workforce. CALDER Working Paper No. 106
Measures of teachers’ “value added” to student achievement play an increasingly central role in k-12 teacher policy and practice, in part because they have been shown to predict teachers’ long-term impacts on students’ life outcomes. However, little research has examined variation in the long-term effects of teachers with similar value-added performance. In this study, we investigate variation in the persistence of teachers’ value-added effects on student achievement in New York City. We separate persistent effects into general effects that improve both the subject taught (math or English language arts (ELA)) and the other area of measured achievement and subject-specific effects which improve only the subject taught. Two findings emerge. First, a teacher’s value-added to ELA achievement has substantial crossover effects on long-term math performance. That is, having a better ELA teacher affects both math and ELA performance in a future year. Conversely, math teachers have only minimal long-term effects on ELA performance; their effects are far more subject-specific. Second, we identify substantial heterogeneity in the persistence of English Language Arts (ELA) teachers’ effects across observable student, teacher, and school characteristics. In particular, teachers in schools serving more poor, minority, and previously low-scoring students have less persistence than other teachers with the same value-added scores. Moreover, ELA teachers with stronger academic backgrounds have more persistent effects on student achievement, as do schools staffed with a higher proportion of such teachers. The results indicate that teachers’ effects on students’ long-term skills can vary as a function of instructional content and quality in ways that are not fully captured by value-added measures of teacher effectiveness.
Citation: Ben Master, Susanna Loeb, James Wyckoff (2014). Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge. CALDER Working Paper No. 104
Teachers in the United States are compensated largely on the basis of fixed schedules that reward experience and credentials. However, there is a growing interest in whether performance-based incentives based on rigorous teacher evaluations can improve teacher retention and performance. The evidence available to date has been mixed at best. This study presents novel evidence on this topic based on IMPACT, the controversial teacher-evaluation system introduced in the District of Columbia Public Schools by then-Chancellor Michelle Rhee. IMPACT implemented uniquely high-powered incentives linked to multiple measures of teacher performance (i.e., several structured observational measures as well as test performance). We present regression-discontinuity (RD) estimates that compare the retention and performance outcomes among low-performing teachers whose ratings placed them near the threshold that implied a strong dismissal threat. We also compare outcomes among high-performing teachers whose rating placed them near a threshold that implied an unusually large financial incentive. Our RD results indicate that dismissal threats increased the voluntary attrition of low-performing teachers by 11 percentage points (i.e., more than 50 percent) and improved the performance of teachers who remained by 0.27 of a teacher-level standard deviation. We also find evidence that financial incentives further improved the performance of high-performing teachers (effect size = 0.24).
Citation: Thomas Dee, James Wyckoff (2013). Incentives, Selection, and Teacher Performance: Evidence from IMPACT. CALDER Working Paper No. 102
This study explores whether teacher performance trajectory over time differs by school poverty settings. Focusing on elementary school mathematics teachers in North Carolina and Florida, we find no systematic relationship between school student poverty rates and teacher performance trajectories. In both high (>=60% FRL) and lower-poverty (<60% FRL) schools, teacher performance improves the fastest in the first five years and then flattens out in years five to ten. Teacher performance growth resumes between year ten and 15 in North Carolina but remains flat in Florida. In both school poverty settings, there is significant variation in teacher performance trajectories. At all career stages, the fastest-growing teachers (75th percentile) improve by .02-.04 standard deviations more in student gain scores annually than slower teachers (25th percentile). Our findings suggest that the lack of productivity “return” to experience in high-poverty schools reported in the literature is unlikely to be the result of differential teacher learning in high and lower-poverty schools.
Citation: Zeyu Xu, Umut Özek, Michael Hansen (2013). Teacher Performance Trajectories in High and Lower-Poverty Schools. CALDER Working Paper No. 101
The specifics of how growth models should be constructed and used to evaluate schools and teachers is a topic of lively policy debate in states and school districts nationwide. In this paper we take up the question of model choice and examine three competing approaches. The first approach, reflected in the popular student growth percentiles (SGPs) framework, eschews all controls for student covariates and schooling environments. The second approach, typically associated with value-added models (VAMs), controls for student background characteristics and under some conditions can be used to identify the causal effects of schools and teachers. The third approach, also VAM-based, fully levels the playing field so that the correlation between school- and teacher-level growth measures and student demographics is essentially zero. We argue that the third approach is the most desirable for use in educational evaluation systems. Our case rests on personnel economics, incentive-design theory, and the potential role that growth measures can play in improving instruction in K-12 schools.
Citation: Mark Ehlert, Cory Koedel, Eric Parsons, Michael Podgursky (2013). Selecting Growth Models for School and Teacher Evaluations: Should Proportionality Matter?. CALDER Working Paper No. 80
There is increasing agreement among researchers and policymakers that teachers vary widely in their ability to improve student achievement, and the difference between effective and ineffective teachers has substantial effects on standardized test outcomes as well as later life outcomes. However, there is not similar agreement about how to improve teacher effectiveness. Several research studies confirm that on average novice teachers show remarkable improvement in effectiveness over the first five years of their careers. In this paper we employ rich data from New York City to explore the variation among teachers in early career returns to experience. Our goal is to better understand the extent to which measures of teacher effectiveness during the first two years reliably predicts future performance. Our findings suggest that early career returns to experience may provide useful insights regarding future performance and offer opportunities to better understand how to improve teacher effectiveness. We present evidence not only about the predictive power of early value-added scores, but also on the limitations and imprecision of those predictions.
Citation: Allison Atteberry, Susanna Loeb, James Wyckoff (2013). Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness. CALDER Working Paper No. 90
We compare teacher preparation programs in Missouri based on the effectiveness of their graduates in the classroom. The differences in effectiveness between teachers from different preparation programs are very small. In fact, virtually all of the variation in teacher effectiveness comes from within-program differences between teachers. Prior research has overstated differences in teacher performance across preparation programs for several reasons, most notably because some sampling variability in the data has been incorrectly attributed to the preparation programs.
Citation: Cory Koedel, Eric Parsons, Michael Podgursky, Mark Ehlert (2012). Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs. CALDER Working Paper No. 79
Redistributing highly effective teachers from low- to high-need schools is an education policy tool that is at the center of several major current policy initiatives. The underlying assumption is that teacher productivity is portable across different schools settings. Using elementary and secondary school data from North Carolina and Florida, this paper investigates the validity of this assumption. Among teachers who switched between schools with substantially different poverty levels or academic performance levels, we find no change in those teachers’ measured effectiveness before and after a school change. This pattern holds regardless of the direction of the school change. We also find that high-performing teachers’ value-added dropped and low-performing teachers’ value-added gained in the post-move years, primarily as a result of regression to the within-teacher mean and unrelated to school setting changes. Despite such shrinkages, high-performing teachers in the pre-move years still outperformed low-performing teachers after moving to schools with different settings.
Citation: Zeyu Xu, Umut Özek, Matthew Corritore (2012). Portability of Teacher Effectiveness Across Schools. CALDER Working Paper No. 77
In this paper we report on work estimating the stability of value-added estimates of teacher effects, an important area of investigation given public interest in workforce policies that implicitly assume effectiveness is a stable attribute within teachers. The results strongly reject the hypothesis that teacher performance is completely stable within teachers over long periods of time, but estimates suggest that a component of performance appears to persist within teachers, even over a ten-year panel. We also find that little of the changes in teacher effectiveness estimates within teachers can be explained by observable characteristics.
Citation: Dan Goldhaber, Michael Hansen (2012). Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance. CALDER Working Paper No. 73
With teacher quality repeatedly cited as the most important schooling factor influencing student achievement, there has been increased interest in examining the efficacy of teacher training programs. This paper presents research examining the variation between and impact that individual teacher training institutions in Washington state have on the effectiveness of teachers they train. Using administrative data linking teachers' initial endorsements to student achievement on state reading and math tests, we find the majority of teacher training programs produce teachers who are no more or less effective than teachers who trained out-of-state. However, we do find a number of cases where there are statistically significant differences between estimates of training program effects for teachers who were credentialed at various in-state programs. These findings are robust to a variety of different model specifications.
Citation: Dan Goldhaber, Stephanie Liddle (2012). The Gateway to the Profession: Assessing Teacher Preparation Programs Based on Student Achievement. CALDER Working Paper No. 65
Although much has been written about the importance of leadership in the determination of organizational success, there is little quantitative evidence due to the difficulty of separating the impact of leaders from other organizational components – particularly in the public sector. Schools provide an especially rich environment for studying the impact of public sector management, not only because of the hypothesized importance of leadership but also because of the plentiful achievement data that provide information on institutional outcomes. Outcome-based estimates of principal value-added to student achievement reveal significant variation in principal quality that appears to be larger for high-poverty schools. Alternate lower-bound estimates based on direct estimation of the variance yield smaller estimates of the variation in principal productivity but ones that are still important, particularly for high poverty schools. Patterns of teacher exits by principal quality validate the notion that a primary channel for principal influence is the management of the teacher force. Finally, looking at principal transitions by quality reveals little systematic evidence that more effective leaders have a higher probability of exiting high poverty schools.
Citation: Gregory F. Branch, Eric Hanushek, Steven Rivkin (2012). Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals. CALDER Working Paper No. 66
This study seeks to identify the characteristics and training experiences of teachers who are differentially effective at promoting academic achievement among English language learners (ELLs). Our analyses indicate that general skills such as those reflected by scores on teacher certification exams and experience teaching non-ELL students are less predictive of achievement for ELL students than for other students. However, specific experience teaching ELL students is more important for predicting effectiveness with future ELL students than non-ELL students as is both in-service and pre-service training focused on ELL-specific instructional strategies.
Citation: Ben Master, Susanna Loeb, Camille Whitney, James Wyckoff (2012). Different Skills: Identifying Differentially Effective Teachers of English Language Learners. CALDER Working Paper No. 68