You are here
Research Areas
Educator preparation and teacher labor markets
Full Abstract
We examine the efficiency implications of imposing proportionality in teacher evaluation systems. Proportional evaluations force comparisons to be between equally-circumstanced teachers. We contrast proportional evaluations with global evaluations, which compare teachers to each other regardless of teaching circumstance. We consider a policy where administrators use the ratings from the evaluation system to help shape the teaching workforce, and define efficiency in terms of student achievement. Our analysis indicates that proportionality can be imposed in teacher evaluation systems without efficiency costs under a wide range of evaluation and estimation conditions. Proportionality is efficiency-enhancing in some cases. These findings are notable given that proportional teacher evaluations offer a number of other policy benefits.
Citation: Cory Koedel, Jiaxi Li (2014). The Efficiency Implications of Using Proportional Evaluations to Shape the Teaching Workforce. CALDER Working Paper No. 106
Full Abstract
Measures of teachers’ “value added” to student achievement play an increasingly central role in k-12 teacher policy and practice, in part because they have been shown to predict teachers’ long-term impacts on students’ life outcomes. However, little research has examined variation in the long-term effects of teachers with similar value-added performance. In this study, we investigate variation in the persistence of teachers’ value-added effects on student achievement in New York City. We separate persistent effects into general effects that improve both the subject taught (math or English language arts (ELA)) and the other area of measured achievement and subject-specific effects which improve only the subject taught. Two findings emerge. First, a teacher’s value-added to ELA achievement has substantial crossover effects on long-term math performance. That is, having a better ELA teacher affects both math and ELA performance in a future year. Conversely, math teachers have only minimal long-term effects on ELA performance; their effects are far more subject-specific. Second, we identify substantial heterogeneity in the persistence of English Language Arts (ELA) teachers’ effects across observable student, teacher, and school characteristics. In particular, teachers in schools serving more poor, minority, and previously low-scoring students have less persistence than other teachers with the same value-added scores. Moreover, ELA teachers with stronger academic backgrounds have more persistent effects on student achievement, as do schools staffed with a higher proportion of such teachers. The results indicate that teachers’ effects on students’ long-term skills can vary as a function of instructional content and quality in ways that are not fully captured by value-added measures of teacher effectiveness.
Citation: Ben Master, Susanna Loeb, James Wyckoff (2014). Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge. CALDER Working Paper No. 104
Full Abstract
We use a unique longitudinal sample of student teachers (“interns”) from six Washington state teacher training institutions to investigate patterns of entry into the teaching workforce. Specifically, we estimate split population models that simultaneously estimate the impact of individual characteristics and student teaching experiences on the timing and probability of initial hiring as a public school teacher. Not surprisingly, we find that interns endorsed to teach in “difficult-to-staff” areas are more likely to be hired as teachers than interns endorsed in other areas. Younger interns, white interns, and interns who did their student teaching in suburban schools are also more likely to find a teaching job. Prospective teachers who do their internships at schools that have more teacher turnover are more likely to find employment, often at those schools. Finally, interns with higher credential exam scores are more likely to be hired by the school where they did their student teaching. Contrary to expectations, few of the measures of the quality or the experience of an intern’s cooperating teacher are predictive of workforce entry in the expected direction.
Citation: Dan Goldhaber, John Krieg, Roddy Theobald (2013). Knocking on the Door to the Teaching Profession? Modeling the Entry of Prospective Teachers into the Workforce. CALDER Working Paper No. 105
Full Abstract
Teachers in the United States are compensated largely on the basis of fixed schedules that reward experience and credentials. However, there is a growing interest in whether performance-based incentives based on rigorous teacher evaluations can improve teacher retention and performance. The evidence available to date has been mixed at best. This study presents novel evidence on this topic based on IMPACT, the controversial teacher-evaluation system introduced in the District of Columbia Public Schools by then-Chancellor Michelle Rhee. IMPACT implemented uniquely high-powered incentives linked to multiple measures of teacher performance (i.e., several structured observational measures as well as test performance). We present regression-discontinuity (RD) estimates that compare the retention and performance outcomes among low-performing teachers whose ratings placed them near the threshold that implied a strong dismissal threat. We also compare outcomes among high-performing teachers whose rating placed them near a threshold that implied an unusually large financial incentive. Our RD results indicate that dismissal threats increased the voluntary attrition of low-performing teachers by 11 percentage points (i.e., more than 50 percent) and improved the performance of teachers who remained by 0.27 of a teacher-level standard deviation. We also find evidence that financial incentives further improved the performance of high-performing teachers (effect size = 0.24).
Citation: Thomas Dee, James Wyckoff (2013). Incentives, Selection, and Teacher Performance: Evidence from IMPACT. CALDER Working Paper No. 102
Full Abstract
This study explores whether teacher performance trajectory over time differs by school poverty settings. Focusing on elementary school mathematics teachers in North Carolina and Florida, we find no systematic relationship between school student poverty rates and teacher performance trajectories. In both high (>=60% FRL) and lower-poverty (<60% FRL) schools, teacher performance improves the fastest in the first five years and then flattens out in years five to ten. Teacher performance growth resumes between year ten and 15 in North Carolina but remains flat in Florida. In both school poverty settings, there is significant variation in teacher performance trajectories. At all career stages, the fastest-growing teachers (75th percentile) improve by .02-.04 standard deviations more in student gain scores annually than slower teachers (25th percentile). Our findings suggest that the lack of productivity “return” to experience in high-poverty schools reported in the literature is unlikely to be the result of differential teacher learning in high and lower-poverty schools.
Citation: Zeyu Xu, Umut Özek, Michael Hansen (2013). Teacher Performance Trajectories in High and Lower-Poverty Schools. CALDER Working Paper No. 101
Full Abstract
The specifics of how growth models should be constructed and used to evaluate schools and teachers is a topic of lively policy debate in states and school districts nationwide. In this paper we take up the question of model choice and examine three competing approaches. The first approach, reflected in the popular student growth percentiles (SGPs) framework, eschews all controls for student covariates and schooling environments. The second approach, typically associated with value-added models (VAMs), controls for student background characteristics and under some conditions can be used to identify the causal effects of schools and teachers. The third approach, also VAM-based, fully levels the playing field so that the correlation between school- and teacher-level growth measures and student demographics is essentially zero. We argue that the third approach is the most desirable for use in educational evaluation systems. Our case rests on personnel economics, incentive-design theory, and the potential role that growth measures can play in improving instruction in K-12 schools.
Citation: Mark Ehlert, Cory Koedel, Eric Parsons, Michael Podgursky (2013). Selecting Growth Models for School and Teacher Evaluations: Should Proportionality Matter?. CALDER Working Paper No. 80
Full Abstract
This paper studies the pension preferences of Washington State public school teachers by examining two periods of time during which teachers were able to choose between enrolling in a traditional defined benefit plan and a hybrid plan with defined benefit and defined contribution components. Our findings suggest that a large share of teachers are willing to transfer from a traditional DB plan to a hybrid pension plan, and that the probability that a teacher will choose to transfer is related to financial incentives and factors related to risk preferences. Among new hires, observable teacher and job characteristics explain little of the pension decision, but there is some evidence that more effective teachers are more likely to enroll in the hybrid pension plan. The general popularity of the hybrid plan suggests that states could reduce the financial risk associated with DB pensions without sacrificing the desirability of pension plans to employees.
Citation: Dan Goldhaber, Cyrus Grout (2013). Which Plan to Choose? The Determinants of Pension System Choice for Public School Teachers. CALDER Working Paper No. 98
Full Abstract
There is increasing agreement among researchers and policymakers that teachers vary widely in their ability to improve student achievement, and the difference between effective and ineffective teachers has substantial effects on standardized test outcomes as well as later life outcomes. However, there is not similar agreement about how to improve teacher effectiveness. Several research studies confirm that on average novice teachers show remarkable improvement in effectiveness over the first five years of their careers. In this paper we employ rich data from New York City to explore the variation among teachers in early career returns to experience. Our goal is to better understand the extent to which measures of teacher effectiveness during the first two years reliably predicts future performance. Our findings suggest that early career returns to experience may provide useful insights regarding future performance and offer opportunities to better understand how to improve teacher effectiveness. We present evidence not only about the predictive power of early value-added scores, but also on the limitations and imprecision of those predictions.
Citation: Allison Atteberry, Susanna Loeb, James Wyckoff (2013). Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness. CALDER Working Paper No. 90
Full Abstract
In this descriptive paper we detail the structure of two Washington State teacher retirement plans: a traditional defined benefit plan and a hybrid defined benefit-defined contribution plan. We provide preliminary evidence on how retirement plan structures may relate to the choices that teachers make. Our analysis of the financial incentives offered to Washington State teachers under the two different plans reveals several patterns that may influence teacher behavior. Teachers experience large gains in their pension wealth by crossing key age and experience thresholds. The relative magnitude of expected pension wealth differs sharply between the plans depending on when a teacher anticipates exiting the position, and the magnitude of anticipated returns to investment. We observe teacher choices between the traditional defined benefit plan and the hybrid plan during two time periods: 1996–1997 and 2008–2010. In 1996–1997 teachers were offered a financial inducement to switch into the newly created hybrid plan and defaulted into staying in the traditional plan if no action was taken. Teachers hired during 2008–2010 defaulted into the hybrid plan if no action was taken. Most of the teachers who were given a choice opted for the hybrid plan. This preference for the hybrid plan is more pronounced among the 1996–1997 cohort, who received a financial incentive in the form of a transfer payment for switching. The notable exception is among teachers who were over 55, and or teachers with relatively high experience levels, who were more likely to choose the traditional defined benefit plan.
Citation: Dan Goldhaber, Cyrus Grout, Annie Pennucci, Wesley Bignell (2012). Teacher Pension Choice: Surveying the Landscape in Washington State. CALDER Working Paper No. 81
Full Abstract
We compare teacher preparation programs in Missouri based on the effectiveness of their graduates in the classroom. The differences in effectiveness between teachers from different preparation programs are very small. In fact, virtually all of the variation in teacher effectiveness comes from within-program differences between teachers. Prior research has overstated differences in teacher performance across preparation programs for several reasons, most notably because some sampling variability in the data has been incorrectly attributed to the preparation programs.
Citation: Cory Koedel, Eric Parsons, Michael Podgursky, Mark Ehlert (2012). Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs. CALDER Working Paper No. 79
Full Abstract
During the late 1990's public pension funds across the United States accrued large actuarial surpluses. The seemingly flush conditions of the pension funds led legislators in most states to substantially improve retirement benefits for public workers, including teachers. In this study we examine the benefit enhancements to the teacher pension system in Missouri. These enhancements resulted in large windfall gains for teachers who were close to retirement when the legislation was enacted. By contrast, novice teachers and teachers who had not yet entered the labor force, were made worse off. The reason is that front-end contribution rates have been raised for current teachers to offset past liabilities accrued from the enhancements. Other things equal, the teaching profession in Missouri is now less appealing for young teachers than it was before the pension enhancements were enacted.
Citation: Cory Koedel, Shawn Ni, Michael Podgursky (2012). Who Benefits from Pension Enhancements?. CALDER Working Paper No. 76
Full Abstract
Redistributing highly effective teachers from low- to high-need schools is an education policy tool that is at the center of several major current policy initiatives. The underlying assumption is that teacher productivity is portable across different schools settings. Using elementary and secondary school data from North Carolina and Florida, this paper investigates the validity of this assumption. Among teachers who switched between schools with substantially different poverty levels or academic performance levels, we find no change in those teachers’ measured effectiveness before and after a school change. This pattern holds regardless of the direction of the school change. We also find that high-performing teachers’ value-added dropped and low-performing teachers’ value-added gained in the post-move years, primarily as a result of regression to the within-teacher mean and unrelated to school setting changes. Despite such shrinkages, high-performing teachers in the pre-move years still outperformed low-performing teachers after moving to schools with different settings.
Citation: Zeyu Xu, Umut Özek, Matthew Corritore (2012). Portability of Teacher Effectiveness Across Schools. CALDER Working Paper No. 77
Full Abstract
In this paper we report on work estimating the stability of value-added estimates of teacher effects, an important area of investigation given public interest in workforce policies that implicitly assume effectiveness is a stable attribute within teachers. The results strongly reject the hypothesis that teacher performance is completely stable within teachers over long periods of time, but estimates suggest that a component of performance appears to persist within teachers, even over a ten-year panel. We also find that little of the changes in teacher effectiveness estimates within teachers can be explained by observable characteristics.
Citation: Dan Goldhaber, Michael Hansen (2012). Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance. CALDER Working Paper No. 73
Full Abstract
Educators in public schools in the United States are typically enrolled in defined-benefit pension plans, which penalize across-plan mobility. We use administrative data from Missouri to examine how the mobility penalties affect the labor market for school leaders, and show that pension borders greatly reduce leadership flows across schools. Our most conservative estimates indicate that removing a pension border that divides two groups of schools will increase leadership flows between the groups by roughly 100 percent. We consider the implications of our findings for workforce quality in schools near pension borders in Missouri. Our results are of general interest given that thousands of public schools operate near pension boundaries nationwide. See Working Paper 67 for an updated version.
Citation: Cory Koedel, Jason Grissom, Shawn Ni, Michael Podgursky (2012). Pension-Induced Rigidities in the Labor Market for School Leaders. CALDER Working Paper No. 67
Full Abstract
In this paper we consider the challenges involved in evaluating teacher preparation programs when controlling for school contextual bias. Including school fixed effects in the achievement models used to estimate preparation program effects controls for school environment by relying on differences among student outcomes within the same schools to identify the program effects. However, identification of preparation program effects using school fixed effects requires teachers from different programs to teach in the same school. Even if program effects are identified, the precision of the estimated effects will depend on the degree to which graduates from different programs overlap across schools. In addition, if the connections between preparation programs result from the overlap of atypical graduates or from graduates teaching in atypical school environments, use of school effects could produce bias. Using statewide data from Florida, we show that teachers tend to teach in schools near the programs in which they received their training, but there is still sufficient overlap across schools to identify preparation program effects. We show that the ranking of preparation programs varies significantly depending on whether or not school environment is taken into account via school fixed effects. We find that schools and teachers that are integral to connecting preparation programs are atypical, with disproportionately high percentages of Hispanic teachers and students compared to the state averages. Finally, we find significant variance inflation in the estimated program effects when controlling for school fixed effects, and that the size of the variance inflation factor depends crucially on the length of the window used to compare graduates teaching in the same schools.
Citation: Kata Mihaly, Daniel McCaffery, Tim Sass, J.R. Lockwood (2012). Where You Come From or Where You Go? Distinguishing Between School Quality and the Effectiveness of Teacher Preparation Program Graduates. CALDER Working Paper No. 63
Full Abstract
With teacher quality repeatedly cited as the most important schooling factor influencing student achievement, there has been increased interest in examining the efficacy of teacher training programs. This paper presents research examining the variation between and impact that individual teacher training institutions in Washington state have on the effectiveness of teachers they train. Using administrative data linking teachers' initial endorsements to student achievement on state reading and math tests, we find the majority of teacher training programs produce teachers who are no more or less effective than teachers who trained out-of-state. However, we do find a number of cases where there are statistically significant differences between estimates of training program effects for teachers who were credentialed at various in-state programs. These findings are robust to a variety of different model specifications.
Citation: Dan Goldhaber, Stephanie Liddle (2012). The Gateway to the Profession: Assessing Teacher Preparation Programs Based on Student Achievement. CALDER Working Paper No. 65
Full Abstract
Although much has been written about the importance of leadership in the determination of organizational success, there is little quantitative evidence due to the difficulty of separating the impact of leaders from other organizational components – particularly in the public sector. Schools provide an especially rich environment for studying the impact of public sector management, not only because of the hypothesized importance of leadership but also because of the plentiful achievement data that provide information on institutional outcomes. Outcome-based estimates of principal value-added to student achievement reveal significant variation in principal quality that appears to be larger for high-poverty schools. Alternate lower-bound estimates based on direct estimation of the variance yield smaller estimates of the variation in principal productivity but ones that are still important, particularly for high poverty schools. Patterns of teacher exits by principal quality validate the notion that a primary channel for principal influence is the management of the teacher force. Finally, looking at principal transitions by quality reveals little systematic evidence that more effective leaders have a higher probability of exiting high poverty schools.
Citation: Gregory F. Branch, Eric Hanushek, Steven Rivkin (2012). Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals. CALDER Working Paper No. 66
Full Abstract
This study seeks to identify the characteristics and training experiences of teachers who are differentially effective at promoting academic achievement among English language learners (ELLs). Our analyses indicate that general skills such as those reflected by scores on teacher certification exams and experience teaching non-ELL students are less predictive of achievement for ELL students than for other students. However, specific experience teaching ELL students is more important for predicting effectiveness with future ELL students than non-ELL students as is both in-service and pre-service training focused on ELL-specific instructional strategies.
Citation: Ben Master, Susanna Loeb, Camille Whitney, James Wyckoff (2012). Different Skills: Identifying Differentially Effective Teachers of English Language Learners. CALDER Working Paper No. 68
Full Abstract
Researchers and policymakers often assume that teacher turnover harms student achievement, but recent evidence calls into question this assumption. Using a unique identification strategy that employs grade-level turnover and two classes of fixed-effects models, this study estimates the effects of teacher turnover on over 1.1 million New York City 4th grade student observations over 10 years. The results indicate that students in grade-levels with higher turnover score lower in both ELA and math and that this effect is particularly strong in schools with more low-performing and black students. Moreover, the results suggest that there is a disruptive effect of turnover beyond changing the composition in teacher quality.
Citation: Matthew Ronfeldt, Susanna Loeb, James Wyckoff (2012). How Teacher Turnover Harms Student Achievement. CALDER Working Paper No. 70
Full Abstract
In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM estimates of current teacher contributions to student learning. More precisely, the falsification test is designed to identify whether or not students are effectively randomly assigned conditional on the covariates included in the model. Rothstein's finding is significant because there is considerable interest in using VAM teacher effect estimates for high-stakes teacher personnel policies, and the results of the Rothstein test cast considerable doubt on the notion that VAMs can be used fairly for this purpose. However, in this paper, we illustrate—theoretically and through simulations—plausible conditions under which the Rothstein falsification test rejects VAMs even when students are randomly assigned, conditional on the covariates in the model, and even when there is no bias in estimated teacher effects.
Citation: Dan Goldhaber, Duncan Chaplin (2012). Assessing the “Rothstein Test”. Does it Really Show Teacher Value-Added Models are Biased?. CALDER Working Paper No. 71