You are here
Using administrative longitudinal data from five states, we study how value-added measures of teacher performance are affected by changes in state standards and assessments. We first document the stability of teachers’ value-added rankings during transitions to new standard and assessment regimes and compare our findings to stability during stable standard and assessment regimes. We also examine the predictive validity of value-added estimates during nontransition years over transition-year student achievement. In most cases we find that measures of teacher value added are similarly stable in transition years and nontransition years. Moreover, there is no evidence that the level of disadvantage of students taught disproportionately influences teacher rankings in transition years relative to stable years. In the states we study, student achievement in math can consistently be forecasted accurately—although not perfectly—using value-added estimates for teachers during stable standards and assessment regimes. There was somewhat less consistency in reading, because we find cases where test transitions significantly reduced forecasting accuracy.
Citation: Benjamin Backes, James Cowan, Dan Goldhaber, Cory Koedel, Luke Miller, Zeyu Xu (2016). The Common Core Conundrum: To What Extent Should We Worry That Changes to Assessments and Standards Will Affect Test-Based Measures of Teacher Performance?. CALDER Working Paper No. 152
Educators raise concerns about what happens to students when they are exposed to new teachers or teachers who are new to a school. These teachers face the challenge of preparing a year’s worth of new material, perhaps in an unfamiliar work environment. However, even when teachers remain in the same school they can switch assignments—teaching either a different grade or a different subject than they have taught before. While there exists some quasi-experimental literature on the effects for student achievement of being new to the profession (e.g., Rockoff, 2004) or to a school (Hanushek & Rivkin, 2010), to date there is little evidence about how much within-school churn typically happens and how it affects students. We use longitudinal panel data from New York City from 1974 to 2010 to document the phenomenon, and we tie assignment-switching behaviors to available student achievement in the period since 1999.
Citation: Allison Atteberry, Susanna Loeb, James Wyckoff (2016). High Rates of Within-School Teacher Reassignments and Implications for Student Achievement. CALDER Working Paper No. 151
We use data from six Washington State teacher education programs to investigate the relationship between teacher candidates’ student teaching experiences and their later teaching effectiveness and probability of attrition. We find that teachers who student taught in schools with lower teacher turnover are less likely to leave the state’s teaching workforce, and that teachers are more effective when the student demographics of their current school are similar to the student demographics of the school in which they did their student teaching. While descriptive, these findings suggest that the school context in which student teaching occurs has important implications for the later outcomes of teachers and their students.
Citation: Dan Goldhaber, John M. Krieg, Roddy Theobald (2016). Does the Match Matter? Exploring Whether Student Teaching Experiences Affect Teacher Effectiveness and Attrition. CALDER Working Paper No. 149
We use rich longitudinally matched administrative data on students and teachers in North Carolina to examine the patterns of differential effectiveness by teachers’ years of experience. The paper contributes to the literature by focusing on middle school teachers and by extending the analysis to student outcomes beyond test scores. Once we control statistically for the quality of individual teachers by the use of teacher fixed effects, we find large returns to experience for middle school teachers in the form both of higher test scores and improvements in student behavior, with the clearest behavioral effects emerging for reductions in student absenteeism. Moreover these returns extend well beyond the first few years of teaching. The paper contributes to policy debates by documenting that teachers can and do continue to learn on the job.
December 2015 Update
Citation: Helen Ladd, Lucy Sorensen (2015). Returns to Teacher Experience: Student Achievement and Motivation in Middle School. CALDER Working Paper No. 112
Rising costs of public employee pension plans are a source of fiscal stress in many cities and states and have led to calls for reform. To assess the economic consequences of plan changes it is important to have reliable statistical models of employee retirement behavior. The authors estimate a structural model of teacher retirement using administrative panel data. A Stock-Wise option value model provides a good fit to the data and predicts well out-of-sample on the effects of pension enhancements during the 1990s. The structural model is used to simulate the effect of alternatives to the current defined benefit plan.
Citation: Shawn Ni, Michael Podgursky (2015). How Teachers Respond to Pension System Incentives: New Estimates and Policy Applications. CALDER Working Paper No. 147
Recent evidence on teacher productivity suggests teachers meaningfully influence noncognitive student outcomes that are commonly overlooked by narrowly focusing on student test scores. These effects may show similar levels of variation across the teacher workforce and are not significantly correlated with value-added test score gains. Despite a large number of studies investigating the TFA effect on math and English achievement, little is known about nontested outcomes. Using administrative data from Miami-Dade County Public Schools, we investigate the relationship between being in a TFA classroom and non-test student outcomes. We validate our use of nontest student outcomes to assess differences in teacher productivity using the quasi-experimental teacher switching methods of Chetty, Friedman, and Rockoff (2014) and find multiple cases in which these tests reject the validity of candidate nontest outcomes. Among the cases deemed valid, we find suggestive evidence that students taught by TFA teachers in elementary and middle school were less likely to miss school due to unexcused absences and suspensions (compared to non-TFA teachers in the same school), although point estimates are very small. Other nontest outcomes were found to be valid but showed no evidence of a TFA effect.
Citation: Benjamin Backes, Michael Hansen (2015). Teach For America Impact Estimates on Nontested Student Outcomes. CALDER Working Paper No. 146
Teach For America (TFA) is an alternative certification program that intensively recruits and selects recent college graduates and midcareer professionals to teach in schools serving high-need students. Prior rigorous evaluations of the program have generally found positive effects of TFA teachers on students’ learning in math and science and no significant differences in reading or language arts, compared with non-TFA teachers’ effects in the same schools. No priorstudies, however, have specifically focused on TFA effects in the Atlanta region.
This report examines the efficacy of TFA teachers in the Atlanta region spanning the 2005-06 through 2013-14 school years. Using longitudinal administrative data from three major school districts with significant numbers of recent TFA placements, we generate TFA effect estimates based on two series of Georgia’s standardized tests—the end-of-grade Criterion-Referenced Competency Tests (CRCTs) and end-of-course tests (EOCTs).
We find evidence of a positive effect in student learning due to the hiring of TFA teachers in these three districts, compared with the performance of non-TFA colleagues in the same schools. Estimated TFA effects are positive and statistically significant in social studies and science on the state’s CRCTs, and in American literature on the state’s EOCTs. We find no significant differences in performance between TFA and non-TFA teachers in the other subjects we analyzed. Supplementary analyses show these results are not sensitive to the inclusion of data from a period of well-documented test score manipulation in Atlanta Public Schools.
Citation: Michael Hansen, Tim Sass (2015). Performance Estimates of Teach For America Teachers in Atlanta Metropolitan Area School Districts. CALDER Working Paper No. 145
We use data from workers in the largest public-sector occupation in the United States – teaching – to examine the effect of pension enhancements on employee retention. Specifically, we study a 1999 enhancement to the benefit formula for public school teachers in St. Louis that resulted in an immediate and dramatic increase in their incentives to remain in covered employment. To identify the effect of the enhancement on teacher retention, we leverage the fact that the strength of the incentive increase varied across the workforce depending on how far teachers were from retirement eligibility when it was enacted. Our results indicate that the St. Louis enhancement – which was structurally similar to enhancements that were enacted in other public pension plans across the United States in the late 1990s and early 2000s – was not a cost-effective way to increase employee retention.
Citation: Cory Koedel, P. Brett Xiang (2015). Pension Enhancements and the Retention of Public Employees: Evidence from Teaching. CALDER Working Paper No. 123
We use data from Washington state to examine two distinct stages of the teacher pipeline: the placement of prospective teachers in student teaching assignments and the hiring of prospective teachers into their first teaching positions. We find that prospective teachers are likely to complete their student teaching near their colleges and hometowns but prospective teachers’ student teaching positions are much more predictive of their first teaching positions than their hometowns. This suggests that the “draw of home” in new teacher hiring is driven by patterns in student teaching assignments. We also find that more qualified prospective teachers tend to student teach in more advantaged districts, suggesting that patterns in student teaching assignments may contribute to the inequitable distribution of teacher quality.
Citation: John Krieg, Roddy Theobald, Dan Goldhaber (2015). A Foot in the Door: Exploring the Role of Student Teaching Assignments in Teachers’ Initial Job Placements. CALDER Working Paper No. 144
Due to data limitations, very little is known about patterns of teacher cross-state mobility. The issue is important because barriers to cross-state mobility create labor market frictions that could lead both current and prospective teachers to opt out of the teaching profession. For this paper, we match state-level administrative data sets from Oregon and Washington and present evidence on patterns of in-service teacher mobility between these two states. We find levels of cross-state mobility that are drastically lower than levels of within-state mobility, even when accounting for proximity to the border. These findings are consistent with the hypothesis that there are significant penalties to cross-state mobility that may be attributable to state-specific licensure regulations, seniority rules and pension structures.
Citation: Dan Goldhaber, Cyrus Grout, Kristian Holden, Nate Brown (2015). Crossing the Border? Exploring the Cross-State Mobility of the Teacher Workforce. CALDER Working Paper No. 143
Public pension systems in many U.S. states face large funding shortfalls. Movement toward defined contribution (DC) pension structures may reduce the likelihood of future shortfalls. We address some limitations of the existing literature by studying public-sector employees who are enrolled in either a defined benefit (DB) plan or hybrid DB-DC plan, and who at some points have been able to choose between these plans. We find little evidence that the introduction of the hybrid plan increased employee turnover and that turnover is significantly lower among those who transferred from the DB plan to the hybrid plan.
Citation: Dan Goldhaber, Cyrus Grout, Kristian Holden (2015). Pension Structure and Employee Turnover: Evidence from a Large Public Pension System. CALDER Working Paper No. 142
We investigated the effects of a statewide program designed to increase the supply of teachers in “hard-to-staff” areas. The Florida Critical Teacher Shortage Program (FCTSP) had three elements: (a) it provided loan forgiveness to teachers who were certified and taught in designated shortage areas; (b) it compensated teachers for the tuition cost of taking courses to become certified in a designated shortage area; and (c) for a single year, it gave bonuses to high school teachers who were certified and taught in a designated subject area. Employing a difference-in-difference estimator, we find that the loan forgiveness program decreased attrition of teachers in shortage areas, although the effects varied by subject. Allowing for variation in the size of payments, we find that the effects were more pronounced when loan-forgiveness payments were more generous. A triple-difference estimate indicated the bonus program also substantially reduced the likelihood of teachers leaving the public school sector. A panel probit analysis reveals that the tuition-reimbursement program had modest positive effects on the likelihood a teacher would become certified in a designated shortage area. We also present qualitative evidence that loan-forgiveness recipients were of higher quality (as measured by value added) than nonrecipients who taught in the same subject but were not certified and thus ineligible.
Citation: Li Feng, Tim Sass (2015). The Impact of Incentives to Recruit and Retain Teachers in “Hard-to-Staff” Subjects: An Analysis of the Florida Critical Teacher Shortage Program. CALDER Working Paper No. 141
We examine the extent to which clustering large numbers of Teach For America (TFA) corps members in a limited number of low-performing schools was accompanied by changes in teacher mobility decisions. Using longitudinal data from Miami-Dade spanning six school years, augmented with survey responses from TFA’s own Alumni Survey for cohorts placed in the Miami region, we use Cox proportional hazards models and multinomial logit decision models in a modified difference-in-difference (DD) framework. Our results suggest that the increased concentration of TFA corps members in schools was associated with a reduction in TFA mobility across schools after the first year of service, but it did not affect the overall retention of corps members in the district after the two-year commitment. In addition, we find evidence suggesting non-TFA teachers in schools with a relatively high proportion of TFA corps members were significantly more likely to leave the district. We also find that TFA corps members retained beyond the two-year commitment performed substantially better in mathematics during their first two years of teaching: evidence of positive selection into postcommitment retention. Finally, we produce steady-state estimates of the minimum TFA effects necessary for the district to prefer hiring a TFA corps member relative to a non-TFA hire. TFA corps members in the district exceed this minimum value in both reading and mathematics.
Citation: Michael Hansen, Benjamin Backes, Victoria Brady (2015). Teacher Attrition and Mobility During the Teach For America Clustering Strategy in Miami-Dade County Public Schools. CALDER Working Paper No. 139
A sizeable body of evidence has documented the effectiveness of Teach For America (TFA) corps members at raising the mathematics test scores of their students, though little is known about the program’s impact at the school level. TFA’s recent placement strategy in the Miami-Dade County Public Schools, in which large numbers of TFA corps members are placed as clusters into a targeted set of disadvantaged schools, provides an opportunity to evaluate the impact of the TFA program on broader school performance. This study examines whether the influx of TFA corps members led to a spillover effect on other teachers’ performance. We find that many of the schools chosen to participate in the cluster strategy experienced large subsequent gains in mathematics achievement. These gains were driven in part by the composition effect of having larger numbers of effective TFA corps members. However, we do not find any evidence that the clustering strategy led to any spillover effect on schoolwide performance. In other words, our estimates suggest that the extra student gains for TFA corps members under the clustering strategy would be equivalent to gains resulting from an alternate placement strategy in which corps members were evenly distributed across schools.
Revised August 31, 2015
Citation: Michael Hansen, Benjamin Backes, Victoria Brady, Zeyu Xu (2015). Examining Spillover Effects from Teach For America Corps Members in Miami- Dade County Public Schools. CALDER Working Paper No. 113
There is increased policy interest in extending test-based evaluations in K-12 education to include student achievement in high school. High school achievement is typically measured by performance on end-of-course exams (EOCs), which test course-specific standards in a variety of subjects. However, unlike standardized tests in the early grades, students take EOCs at different points in their schooling careers. The timing of the test is a choice variable presumably determined by input from administrators, students and parents. Recent research indicates that school and district policies that determine when students take particular courses can have important consequences for achievement and subsequent outcomes like advanced course taking. We develop an approach for modeling EOC test performance that disentangles the influence of school and district policies regarding the timing of course taking from other factors. After separating out the timing issue, better measures of the quality of instruction provided by districts, schools and teachers can be obtained. Our approach also offers diagnostic value because it separates out the influence of school and district course-timing policies from other factors that determine student achievement.
Citation: Eric Parsons, Cory Koedel, Michael Podgursky, Mark Ehlert , P. Brett Xiang (2015). Incorporating End-of-Course Exam Timing into Educational Performance Evaluations. CALDER Working Paper No. 137
This study uses detailed administrative data on teachers and students from the state of North Carolina to revisit the empirical evidence on master’s degrees, with attention to teachers at the middle and high school levels. It provides descriptive information on which types of teachers obtain master’s degrees, for which subjects, at which institutions, and during what phase of their career. The study estimates returns to master’s degrees using teacher fixed effects to control for time-invariant characteristics of teachers, thus separating the effects of teacher decisions to get an advanced degree from the effects of having one. Even with this careful attention to selection bias, we confirm the findings of prior studies showing that teachers with master’s degrees are no more effective than those without. The only consistently positive effect of attaining a master’s degree emerging from this study relates not to student test scores but rather to lower student absentee rates in middle school.
Citation: Helen Ladd, Lucy C. Sorensen (2015). Do Master’s Degrees Matter? Advanced Degrees, Career Paths, and the Effectiveness of Teachers. CALDER Working Paper No. 136
We investigate patterns of teacher mobility in districts with different collective bargaining agreement (CBA) transfer provisions. We use detailed teacher-level longitudinal data from Washington State to estimate the probability that teachers of varying experience and effectiveness levels transfer out of their schools to other schools in the district, to other districts, or out of Washington kindergarten through 12th grade (K–12) public schools. We find consistent evidence that within-district transfer probabilities increase for veteran teachers with the proportion of disadvantaged students in a school but decrease for novice teachers with the proportion of disadvantaged students, and that the strength of these relationships is associated with the strength of seniority transfer provisions in CBAs. Specifically, the pattern of veteran teachers’ leaving disadvantaged schools and novice teachers’ staying in disadvantaged schools is more pronounced in districts with strong CBA seniority transfer protections. CBA transfer provisions do not, however, appear to be an important factor in teacher transfers out of school districts or the K–12 public school workforce in Washington. Finally, we find some evidence that more effective teachers are more likely to stay in advantaged schools when seniority is not a factor in transfer decisions.
Citation: Dan Goldhaber, Lesley Lavery, Roddy Theobald (2015). Inconvenient Truth? Do Collective Bargaining Agreements Help Explain the Mobility of Teachers Within School Districts?. CALDER Working Paper No. 135
A defining characteristic of charter schools is that they introduce a strong market element into public education. In this paper, we examine the evolution of the charter school sector in North Carolina between 1999 and 2012 through the lens of a market model. We examine trends in the mix of students enrolled in charter schools, the racial imbalance of charter schools, the quality of the match between parental preferences in charter schools relative to the quality of match in traditional public schools, and the distribution of test score performance across charter schools relative those in traditional public schools serving similar students over time. Taken together, our findings imply that the charter schools in North Carolina are increasingly serving the interests of relatively able white students in racially imbalanced schools.
Citation: Helen Ladd, Charles Clotfelter, John B. Holbein (2015). The Growing Segmentation of the Charter School Sector in North Carolina. CALDER Working Paper No. 133
One consequence of the Great Recession is that teacher layoffs occurred at a scale previously unseen. In this paper we assess the effects of receiving a layoff notice on teacher mobility using data from Los Angeles and Washington State. We find strong evidence that the receipt of a layoff notice increases the likelihood that teachers leave their schools, even in the absence of actually losing their position due to a layoff. Placebo tests suggest that it is the layoff process that induces “structural churn” rather than differential mobility of the teachers who are targeted by this process.
Citation: Dan Goldhaber, Katharine O. Strunk, Nate Brown, David S. Knight (2015). Lessons Learned from the Great Recession: Layoffs an the RIF-Induced Teacher Shuffle. CALDER Working Paper No. 129
This study provides a first look at how student college- and career-readiness have progressed in the early years of the Common Core State Standards (CCSS) implementation. It is motivated by concern that changes triggered by the standards transition might be disruptive to student learning in the short run, even when those changes may become beneficial once fully implemented. Using longitudinal administrative data from Kentucky, an early adopter of the CCSS, we followed three cohorts of students from the end of the 8th grade to the end of the 11th grade and found that students exposed to the CCSS—including students in both high- and low-poverty schools—made faster progress in learning than similar students who were not exposed to the standards. Although it is not conclusive whether cross-cohort improvement was entirely attributable to the standards reform, we found that students made large gains in proficiency in the years immediately before and after the transition. Additionally, we found student performance in subjects that adopted CCSS-aligned curriculum framework experienced larger, more immediate improvement than student performance in subjects that carried over last-generation curriculum framework.
Citation: Zeyu Xu, Kennan Cepa (2015). Getting College and Career Ready During State Transition Toward the Common Core State Standards. CALDER Working Paper No. 127