You are here
There is mounting evidence of substantial “teacher quality gaps” (TQGs) between advantaged and disadvantaged students, but practically no empirical evidence about their history. We use longitudinal data on public school students, teachers, and schools from two states—North Carolina and Washington—to provide a descriptive history of the evolution of TQGs in these states. We find that TQGs exist in every year in each state and for all measures we consider of student disadvantage and teacher quality. But there is variation in the magnitudes and sources of TQGs over time, between the two states, and depending on the measure of student disadvantage and teacher quality.
Citation: Dan Goldhaber, Vanessa Quince , Roddy Theobald (2016). Has It Always Been This Way? Tracing the Evolution of Teacher Quality Gaps in U.S. Public Schools. CALDER Working Paper No. 171
We use longitudinal data from Washington State to provide estimates of the extent to which performance on the edTPA, a performance-based, subject-specific assessment of teacher candidates, is predictive of the likelihood of employment in the teacher workforce and value-added measures of teacher effectiveness. While edTPA scores are highly predictive of employment in the state’s public teaching workforce, evidence on the relationship between edTPA scores and teaching effectiveness is more mixed. Specifically, continuous edTPA scores are a significant predictor of student mathematics achievement in some specifications, but when we consider that the edTPA is a binary screen of teaching effectiveness (i.e., pass/fail), we find that passing the edTPA is significantly predictive of teacher effectiveness in reading but not in mathematics. We also find that Hispanic candidates in Washington were more than three times more likely to fail the edTPA after it became consequential in the state than non-Hispanic White candidates.
Citation: Dan Goldhaber, James Cowan, Roddy Theobald (2016). Evaluating Prospective Teachers: Testing the Predictive Validity of the edTPA (Update). CALDER Working Paper No. 157
It is widely believed that teacher turnover adversely affects the quality of instruction in urban schools serving predominantly disadvantaged children, and a growing body of research investigates various components of turnover effects. The evidence at first seems contradictory, as the quality of instruction appears to decline following turnover despite the fact that most work shows higher attrition for less effective teachers. This raises concerns that confounding factors bias estimates of transition differences in teacher effectiveness, the adverse effects of turnover or both. After taking more extensive steps to account for nonrandom sorting of students into classrooms and endogenous teacher exits and grade-switching, we replicate existing findings of adverse selection out of schools and negative effects of turnover in lower-achievement schools. But we find that these turnover effects can be fully accounted for by the resulting loss in experience and productivity loss following the reallocation of some incumbent teachers to different grades.
Citation: Eric Hanushek, Steven Rivkin, Jeffrey Schiman (2016). Dynamic Effects of Teacher Turnover on the Quality of Instruction. CALDER Working Paper No. 170
Educational accountability policies are a popular tool to close the achievement gaps between advantaged and disadvantaged students. However, these policies may exacerbate inequality if families from advantaged backgrounds are better able to advocate for their children and thus circumvent policy. We investigate this possibility in the context of the early grade retention policy in Florida, which requires all students with reading skills below grade level to be retained in the third grade, yet grants exemptions under special circumstances. We find that Florida’s third-grade retention policy is in fact enforced differentially depending on children’s socioeconomic background, especially maternal education. Holding exemption eligibility constant, scoring right below the promotion cutoff results increases the retention probability 14 percent more for children whose mothers have less than a high school degree as compared to children whose mothers have a bachelor’s degree or more. We also find that the discrepancies in retention rates are mainly driven by the fact that students with well-educated mothers are more likely to be promoted based on subjective exemptions such as teacher portfolios.
Citation: Christina LiCalsi, Umut Özek, David Figlio (2016). The Uneven Implementation of Universal School Policies: Maternal Education and Florida’s Mandatory Grade Retention Policy. CALDER Working Paper No. 167
Most public school teachers in the United States are enrolled in defined benefit (DB) pension plans. Using administrative micro data from four states, combined with national pension funding data, we show these plans have accumulated substantial unfunded liabilities – effectively debt – owing to previous plan operations. On average across state plans, over 10 percent of current teachers’ earnings are being set aside to pay for previously-accrued pension liabilities. This amounts to a large reduction in real operating spending per student. Our findings make clear that a significant fraction of the resources allocated toward teacher compensation in current public education budgets is not being invested in resources to educate today’s students at all.
August 2016 Update; Originally posted November 2015
Citation: Benjamin Backes, Dan Goldhaber, Cyrus Grout, Cory Koedel, Shawn Ni, Michael Podgursky, P. Brett Xiang, Zeyu Xu (2016). Benefit or Burden? On the Intergenerational Inequity of Teacher Pension Plans. CALDER Working Paper No. 148
We make use of a new data source – matched birth records and longitudinal student records in Florida – to study the degree to which student outcomes differ across successive immigrant generations. Specifically, we investigate whether first, second, and third generation Asian and Hispanic immigrants in Florida perform differently on reading and mathematics tests, and whether they are differentially likely to get into serious trouble in school, to be truant from school, to graduate from high school, or to be ready for college upon high school graduation. We find evidence suggesting that early-arriving first generation immigrants perform better than do second generation immigrants, and second generation immigrants perform better than third generation immigrants. Among first generation immigrants, the earlier the arrival, the better the students tend to perform. These patterns of findings hold for both Asian and Hispanic students, and suggest a general pattern of successively reduced achievement – beyond a transitional period for recent immigrants – in the generations following the generation that immigrated to the United States.
Citation: Umut Özek, David Figlio (2016). Cross-Generational Differences in Educational Outcomes in the Second Great Wave of Immigration. CALDER Working Paper No. 162
U.S. women graduate high school at higher rates than U.S. men, but the female-male educational advantage is larger, and has increased by more, among black and low-SES students than among white and high-SES students. We explore why boys fare worse than girls—both behaviorally and educationally— by exploiting birth certificates matched to health, disciplinary, academic, and high school graduation records for over one million children born in Florida between 1992 and 2002. We account for unobserved family heterogeneity by contrasting the outcomes of opposite-sex siblings linked to birth mothers by administrative records. Relative to their sisters, boys born to low-education and unmarried mothers, raised in low-income neighborhoods, and enrolled at poor-quality public schools have a higher incidence of absences and behavioral problems throughout elementary and middle school, exhibit higher rates of behavioral and cognitive disability, perform worse on standardized tests, and are less likely to graduate from high school. We argue that the family disadvantage gradient in the gender gap is a causal effect of the post-natal environment: family disadvantage has no relationship with the sibling gender gap in neonatal health. Although family disadvantage is strongly correlated with school and neighborhood quality, the SES gradient in the sibling gender gap is almost as large within schools and neighborhoods as between them. A surprising implication of these findings is that, relative to white children, black boys fare worse than their sisters in significant part because black children— both boys and girls—are raised in more disadvantaged family environments.
Citation: David Figlio, Jeffrey Roth, Kryzsztof Karbownik (2016). Family Disadvantage and the Gender Gap in Behavioral and Educational Outcomes. CALDER Working Paper No. 161
Studies of the charter sector typically compare charters and traditional public schools at a point in time. These comparisons are potentially misleading because many charter-related reforms require time to generate results. We study quality dynamics among Texas charter schools from 2001-2011. School quality in the charter sector was initially highly variable and on average lower than traditional public schools. However, exits, improvement of existing charter schools, and expansion of higher-performing charter management organizations increased charter effectiveness relative to traditional public schools. We present evidence that reduced student mobility and an increased share of charters adhering to No Excuses- style curricula contribute to these improvements.
This paper examines the effect of school turnaround in North Carolina elementary and middle schools. Using a regression discontinuity design, we find that turnaround led to a drop in average school-level math and reading passing rates and an increased concentration of low-income students in treated schools. We use teacher survey data to examine how teacher activities changed. Treated schools brought in new principals and increased the time teachers devoted to professional development. The program also increased administrative burdens and distracted teachers, potentially reducing time available for instruction. Teacher turnover increased after the first full year of implementation. Overall, we find little success for North Carolina’s efforts to turn around low-performing schools under its federally funded Race to the Top grant.
Citation: Helen Ladd, Jennifer A. Heissel (2016). School Turnaround in North Carolina: A Regression Discontinuity Analysis. CALDER Working Paper No. 156
One of the concerns over high-stakes testing is the incentive for teachers to alter the scores of their students. We investigate the effects of teacher cheating on subsequent student achievement, attendance, behavior and educational attainment. We find that test scores drop below expected levels in the first year post-cheating year. These effects persist for reading and ELA, but not for math. The drop in later test scores appears to be due in part to a reduction in access to remediation services. We also find some evidence that cheated middle-school students may be more likely to drop out of high school.
Citation: Jarod Apperson, Carycruz Bueno, Tim Sass (2016). Do the Cheated Ever Prosper? The Long-Run Effects of Test-Score Manipulation by Teachers on Student Outcomes. CALDER Working Paper No. 155
In practice, teacher turnover appears to have negative effects on school quality as measured by student performance. However, some simulations suggest that turnover can instead have large, positive effects under a policy regime in which low-performing teachers can be accurately identified and replaced with more effective teachers. This study examines this question by evaluating the effects of teacher turnover on student achievement under IMPACT, the unique performance-assessment and incentive system in the District of Columbia Public Schools (DCPS). Employing a quasi-experimental design based on data from the first year years of IMPACT, we find that, on average, DCPS replaced teachers who left with teachers who increased student achievement by 0.08 SD in math. When we isolate the effects of lower-performing teachers who were induced to leave DCPS for poor performance, we find that student achievement improves by larger and statistically significant amounts (i.e., 0.14 SD in reading and 0.21 SD in math). In contrast, the effect of exits by teachers not sanctioned under IMPACT is typically negative but not statistically significant.
Citation: Melinda Adnot, Thomas Dee, Veronica Katz, James Wyckoff (2016). Teacher Turnover, Teacher Quality and Student Achievement in DCPS. CALDER Working Paper No. 153
Using administrative longitudinal data from five states, we study how value-added measures of teacher performance are affected by changes in state standards and assessments. We first document the stability of teachers’ value-added rankings during transitions to new standard and assessment regimes and compare our findings to stability during stable standard and assessment regimes. We also examine the predictive validity of value-added estimates during nontransition years over transition-year student achievement. In most cases we find that measures of teacher value added are similarly stable in transition years and nontransition years. Moreover, there is no evidence that the level of disadvantage of students taught disproportionately influences teacher rankings in transition years relative to stable years. In the states we study, student achievement in math can consistently be forecasted accurately—although not perfectly—using value-added estimates for teachers during stable standards and assessment regimes. There was somewhat less consistency in reading, because we find cases where test transitions significantly reduced forecasting accuracy.
Citation: Benjamin Backes, James Cowan, Dan Goldhaber, Cory Koedel, Luke Miller, Zeyu Xu (2016). The Common Core Conundrum: To What Extent Should We Worry That Changes to Assessments and Standards Will Affect Test-Based Measures of Teacher Performance?. CALDER Working Paper No. 152
Educators raise concerns about what happens to students when they are exposed to new teachers or teachers who are new to a school. These teachers face the challenge of preparing a year’s worth of new material, perhaps in an unfamiliar work environment. However, even when teachers remain in the same school they can switch assignments—teaching either a different grade or a different subject than they have taught before. While there exists some quasi-experimental literature on the effects for student achievement of being new to the profession (e.g., Rockoff, 2004) or to a school (Hanushek & Rivkin, 2010), to date there is little evidence about how much within-school churn typically happens and how it affects students. We use longitudinal panel data from New York City from 1974 to 2010 to document the phenomenon, and we tie assignment-switching behaviors to available student achievement in the period since 1999.
Citation: Allison Atteberry, Susanna Loeb, James Wyckoff (2016). High Rates of Within-School Teacher Reassignments and Implications for Student Achievement. CALDER Working Paper No. 151
We use data from six Washington State teacher education programs to investigate the relationship between teacher candidates’ student teaching experiences and their later teaching effectiveness and probability of attrition. We find that teachers who student taught in schools with lower teacher turnover are less likely to leave the state’s teaching workforce, and that teachers are more effective when the student demographics of their current school are similar to the student demographics of the school in which they did their student teaching. While descriptive, these findings suggest that the school context in which student teaching occurs has important implications for the later outcomes of teachers and their students.
Citation: Dan Goldhaber, John M. Krieg, Roddy Theobald (2016). Does the Match Matter? Exploring Whether Student Teaching Experiences Affect Teacher Effectiveness and Attrition. CALDER Working Paper No. 149
We use rich longitudinally matched administrative data on students and teachers in North Carolina to examine the patterns of differential effectiveness by teachers’ years of experience. The paper contributes to the literature by focusing on middle school teachers and by extending the analysis to student outcomes beyond test scores. Once we control statistically for the quality of individual teachers by the use of teacher fixed effects, we find large returns to experience for middle school teachers in the form both of higher test scores and improvements in student behavior, with the clearest behavioral effects emerging for reductions in student absenteeism. Moreover these returns extend well beyond the first few years of teaching. The paper contributes to policy debates by documenting that teachers can and do continue to learn on the job.
December 2015 Update
Citation: Helen Ladd, Lucy Sorensen (2015). Returns to Teacher Experience: Student Achievement and Motivation in Middle School. CALDER Working Paper No. 112
Rising costs of public employee pension plans are a source of fiscal stress in many cities and states and have led to calls for reform. To assess the economic consequences of plan changes it is important to have reliable statistical models of employee retirement behavior. The authors estimate a structural model of teacher retirement using administrative panel data. A Stock-Wise option value model provides a good fit to the data and predicts well out-of-sample on the effects of pension enhancements during the 1990s. The structural model is used to simulate the effect of alternatives to the current defined benefit plan.
Citation: Shawn Ni, Michael Podgursky (2015). How Teachers Respond to Pension System Incentives: New Estimates and Policy Applications. CALDER Working Paper No. 147
Recent evidence on teacher productivity suggests teachers meaningfully influence noncognitive student outcomes that are commonly overlooked by narrowly focusing on student test scores. These effects may show similar levels of variation across the teacher workforce and are not significantly correlated with value-added test score gains. Despite a large number of studies investigating the TFA effect on math and English achievement, little is known about nontested outcomes. Using administrative data from Miami-Dade County Public Schools, we investigate the relationship between being in a TFA classroom and non-test student outcomes. We validate our use of nontest student outcomes to assess differences in teacher productivity using the quasi-experimental teacher switching methods of Chetty, Friedman, and Rockoff (2014) and find multiple cases in which these tests reject the validity of candidate nontest outcomes. Among the cases deemed valid, we find suggestive evidence that students taught by TFA teachers in elementary and middle school were less likely to miss school due to unexcused absences and suspensions (compared to non-TFA teachers in the same school), although point estimates are very small. Other nontest outcomes were found to be valid but showed no evidence of a TFA effect.
Citation: Benjamin Backes, Michael Hansen (2015). Teach For America Impact Estimates on Nontested Student Outcomes. CALDER Working Paper No. 146
Teach For America (TFA) is an alternative certification program that intensively recruits and selects recent college graduates and midcareer professionals to teach in schools serving high-need students. Prior rigorous evaluations of the program have generally found positive effects of TFA teachers on students’ learning in math and science and no significant differences in reading or language arts, compared with non-TFA teachers’ effects in the same schools. No priorstudies, however, have specifically focused on TFA effects in the Atlanta region.
This report examines the efficacy of TFA teachers in the Atlanta region spanning the 2005-06 through 2013-14 school years. Using longitudinal administrative data from three major school districts with significant numbers of recent TFA placements, we generate TFA effect estimates based on two series of Georgia’s standardized tests—the end-of-grade Criterion-Referenced Competency Tests (CRCTs) and end-of-course tests (EOCTs).
We find evidence of a positive effect in student learning due to the hiring of TFA teachers in these three districts, compared with the performance of non-TFA colleagues in the same schools. Estimated TFA effects are positive and statistically significant in social studies and science on the state’s CRCTs, and in American literature on the state’s EOCTs. We find no significant differences in performance between TFA and non-TFA teachers in the other subjects we analyzed. Supplementary analyses show these results are not sensitive to the inclusion of data from a period of well-documented test score manipulation in Atlanta Public Schools.
Citation: Michael Hansen, Tim Sass (2015). Performance Estimates of Teach For America Teachers in Atlanta Metropolitan Area School Districts. CALDER Working Paper No. 145
We use data from workers in the largest public-sector occupation in the United States – teaching – to examine the effect of pension enhancements on employee retention. Specifically, we study a 1999 enhancement to the benefit formula for public school teachers in St. Louis that resulted in an immediate and dramatic increase in their incentives to remain in covered employment. To identify the effect of the enhancement on teacher retention, we leverage the fact that the strength of the incentive increase varied across the workforce depending on how far teachers were from retirement eligibility when it was enacted. Our results indicate that the St. Louis enhancement – which was structurally similar to enhancements that were enacted in other public pension plans across the United States in the late 1990s and early 2000s – was not a cost-effective way to increase employee retention.
Citation: Cory Koedel, P. Brett Xiang (2015). Pension Enhancements and the Retention of Public Employees: Evidence from Teaching. CALDER Working Paper No. 123
We use data from Washington state to examine two distinct stages of the teacher pipeline: the placement of prospective teachers in student teaching assignments and the hiring of prospective teachers into their first teaching positions. We find that prospective teachers are likely to complete their student teaching near their colleges and hometowns but prospective teachers’ student teaching positions are much more predictive of their first teaching positions than their hometowns. This suggests that the “draw of home” in new teacher hiring is driven by patterns in student teaching assignments. We also find that more qualified prospective teachers tend to student teach in more advantaged districts, suggesting that patterns in student teaching assignments may contribute to the inequitable distribution of teacher quality.
Citation: John Krieg, Roddy Theobald, Dan Goldhaber (2015). A Foot in the Door: Exploring the Role of Student Teaching Assignments in Teachers’ Initial Job Placements. CALDER Working Paper No. 144