CALDER working papers have not undergone final formal review and should be cited as working papers. They are intended to encourage discussion and suggestions for revision before final publication. Any opinions, findings, and conclusions expressed in these papers are those of the authors and do not necessarily reflect the views of our funders.
To make up for pandemic-related learning losses, many U.S. public school districts have increased enrollment in their summer school programs. We assess summer school as a strategy for COVID-19 learning recovery by tracking the academic progress of students who attended summer school in 2022 across eight districts serving 400,000 students. Based on students’ spring to fall progress, we find a positive impact for summer school on math test achievement (0.03 standard deviation, SD), but not on reading tests. These effects are predominantly driven by students in upper elementary grades. To put the results into perspective, if we assume that these districts have losses similar to those present at the end of the 2022–23 school year (i.e., approximately -0.2 SD), we estimate summer programming closed approximately 2% to 3% of the districts’ total learning losses in math, but none in reading.
Citation: Ian Callen, Maria V. Carbonari, Michael DeArmond, Daniel Dewey, Elise Dizon-Ross, Dan Goldhaber, Jazmin Isaacs, Thomas J. Kane, Megan Kuhfeld, Anna McDonald, Andrew McEachin, Emily Morton, Atsuko Muroga, Douglas O. Staiger (2023). Summer School as a Learning Loss Recovery Strategy After COVID-19: Evidence From Summer 2022. CALDER Working Paper No. 291-0823
We study the impact of a bonus policy implemented by Hawai‘i Public Schools starting in fall 2020 that raised the salaries of all special education teachers in the state by $10,000. We estimate that the introduction of this policy reduced the proportion of vacant special education teaching positions by 32%, or 1.2 percentage points, and the proportion of special education positions that were vacant or filled by an unlicensed teacher by 35%, or 4.0 percentage points. The bonus policy did not have significant impacts on special education teacher retention; instead, the impacts of the policy were driven almost entirely by an increase in the number of general education teachers in the state who moved into open special education teaching positions. The effects of the bonus policy were also largest in historically hard-to-staff schools in which all teachers also received “tiered school” bonuses of up to $8,000. Hawai‘i therefore represents a unique but instructive case of how strategic financial incentives can help address special education teacher shortages.
Citation: Roddy Theobald, Zeyu Xu, Allison Gilmour, Lisa Lachlan-Hache, Liz Bettini, Nathan Jones (2023). The Impact of a $10,000 Bonus on Special Education Teacher Shortages in Hawai‘i. CALDER Working Paper No. 290-0823
This report provides data about the special education teacher pipeline in Pennsylvania and projects associated with the Pennsylvania Bureau of Special Education’s Attract, Prepare, and Retain (APR) efforts. We first used administrative data from the Pennsylvania Department of Education to provide a historical portrait of the special education teacher pipeline in the state. These data provide a baseline picture of special education teachers in the state and the motivation for the APR projects. To provide formative feedback on the APR projects, we also surveyed students and educators participating in three such projects: Developing Future Special Educators Grants, Mentoring, and Learning Communities/Networking. These data provide some early evidence about how participants view their experiences with these projects.
Citation: Roddy Theobald, Equia Aniagyei-Cobbold, Marcy Stein (2023). The Special Education Teacher Pipeline in Pennsylvania: Year 1 Report. CALDER Working Paper No. 289-0723
Nationally, more than 75% of individuals who are credentialed to teach are prepared in traditional college- or university-based teacher education programs (TEPs). But the college and employment pathways that prospective teachers take to TEP enrollment and completion have not been comprehensively examined. A better understanding of how credentialed individuals find their way into TEPs helps us understand the sources of new teacher supply early in the prospective teacher pipeline. With that in mind, we analyze pathways into and through TEPs using historical postsecondary and unemployment insurance data from Washington state. We find that the pathways are quite varied with around 40% of bachelor’s-level TEP completers spending at least some time in community colleges and less than 40% enrolling and finishing at the same university directly after high school. Pathways to master’s TEP completion are even more varied, with almost half of the completers having prior employment experience. For researchers, this varied landscape raises important questions about the relationship between pathways, candidate persistence, and eventual job performance. For policymakers, the results suggest that efforts to recruit the next generation of teachers need to look beyond the pool of students already enrolled at a 4-year university to include students at 2-year colleges or in the labor force who might be interested in entering a TEP.
Citation: Dan Goldhaber, John Krieg, Stephanie Liddle, Roddy Theobald (2023). The Long and Winding Road: Mapping the College and Employment Pathways to Teacher Education Program Completion in Washington State. CALDER Working Paper No. 288-0723
Tutoring contracts cannot address all the take-up and implementation challenges associated with academic interventions. They also cannot ensure that all students catch up from COVID losses. Districts and vendors both need to find ways to build momentum, capacity, and, in some cases, urgency among teachers, students, and families to make these programs work. But contracts can create incentives for vendors to focus on session attendance, tutoring dosage, and results; on the margin, such incentives might get vendors to pay more attention to the student outcomes that districts care about most.
If school systems are going to deliver the academic support necessary for recovery, they need clear-eyed assessments of their current catch-up efforts. And if they are working with an external partner, they need agreements with incentives that put everyone’s attention and effort on what matters most: delivering academic support to help students catch up from the chaos of the last three years. Although there is not much evidence yet about the impact of different contract provisions, we should all be paying attention to whether contract language focuses vendors on ensuring that students receive the services that districts are paying for.
Citation: Michael DeArmond, Dan Goldhaber, Thomas J. Kane, Andrew McEachin, Emily Morton (2023). Paying for Access: Are Tutoring Contracts Focused on the Right Things?. CALDER Policy Brief No. 34
This study investigated the underexplored topic of teacher preparation program admissions processes by interviewing faculty and analyzing program documents. We investigated how 31 K-12 mathematics and science teacher preparation programs (MSTPPs) and faculty attend to diversity, equity, inclusion, and social and racial justice (DEIJ). Specific foci included applicant recruitment and selection, components of applications (e.g., forms, essays, interviews), and how applicants’ DEIJ-related information and orientations factor into admissions. We found that all MSTPPs participating in the study collected information related to DEIJ (e.g., applicants’ ethnoracial backgrounds, citizenship), and all interviewed faculty expressed an interest in increasing the diversity of applicants and admitted students. Faculty expressed preferences for applicants who evidenced positive DEIJ orientations, such as recognizing social and ethnoracial injustices, but at the same time, differences were evident in how MSTPPs and faculty attended to DEIJ. Considerations, implication, and dilemmas for teacher preparation programs and faculty are discussed.
Citation: Amy Roth McDuffie, David Slavit, Dan Goldhaber, Roddy Theobald, Nicole Griggs (2023). Attention to Equity in Teacher Education Admissions Processes. CALDER Working Paper No. 287-0623
The onset of the pandemic in spring 2020 substantially disrupted routes into teaching and offered a unique opportunity to study this process with different requirements for initial entry into the classroom. We examine the impacts of the Temporary Certificate of Eligibility (Temporary CE), which allowed teacher candidates in New Jersey to enter the workforce before completing assessment and performance requirements. Relative to the novice teacher workforce before the pandemic, Temporary CE teachers were substantially more diverse without any significant effects on teacher performance or student test scores. However, Temporary CE holders were less likely to remain in the same school or in the New Jersey teaching workforce between 2020–21 and 2021–22. Although Temporary CE holders disproportionately entered through alternate routes into teaching, these patterns hold for both traditional- and alternate-route entrants.
Citation: Benjamin Backes, Dan Goldhaber (2023). The Relationship Between Pandemic-Era Teacher Licensure Waivers and Teacher Demographics, Retention, and Effectiveness in New Jersey. CALDER Working Paper No. 286-0623
This qualitative study examines the information collected about applicants to mathematics or science teacher preparation programs (MSTPPs) and how university faculty perceive and value this information in admissions decisions. Based on document review and interviews with MSTPP faculty and admissions directors, we found that broad measures of mathematics and science content background (e.g., achievement test scores, past mathematics and science courses taken) were used more frequently than information on applicants’ specific mathematics and science content knowledge and dispositions. In many cases, application components (such as interviews and personal essay statements) were perceived by faculty to be conducive to surfacing applicants’ content knowledge and dispositions; however, they were not constructed or employed in a way that afforded the obtainment of this information. We highlight salient examples of MSTPPs’ collection and use of information related to mathematics and science and discuss implications for TPP admissions processes.
Citation: David Slavit, Amy Roth McDuffie, Nicole Griggs, Dan Goldhaber, Roddy Theobald (2023). Faculty Perspectives and Values Toward Mathematics and Science Content Information Used in Teacher Preparation Admissions Processes. CALDER Working Paper No. 285-0623
Prior research has connected characteristics of cooperating teachers who supervise student teaching to performance measures of the teacher candidates they host, suggesting more effective teachers may also be better mentors. The specific measures of cooperating teacher effectiveness considered in this prior literature (value added and performance evaluations), however, are infrequently observable to individuals responsible for student teaching placements. In this paper, we consider a more easily observed proxy for mentor effectiveness: National Board (NB) Certification. We find that NB teachers are considerably more likely to host candidates than other teachers, candidates supervised by NB teachers are slightly more likely to be hired within three years, and these candidates have slightly lower value added in English language arts than their peers, all else being equal. We find no significant relationship between cooperating teacher NB certification and candidates’ later attrition and value added in math. We conclude that individuals and policies seeking to leverage student teaching placements to improve student and teacher outcomes may need to focus on less easily observable proxies of cooperating teacher quality than NB certification status.
Citation: Dan Goldhaber, John Krieg, Roddy Theobald, Grace Falken (2023). National Board Certification as a Signal of Cooperating Teacher Quality. CALDER Working Paper No. 284-0523
This paper examines how different measures of teacher quality are related to students’ long-run educational trajectories. We estimate teachers’ test-based and nontest value-added (the latter based on contributions to student absences, suspensions, grade progression, and grades) and assess how these predict various student postsecondary outcomes. We find that both types of value-added have positive effects on student outcomes. Test-based teacher quality measures have more explanatory power for outcomes relevant for students at the top of the achievement distribution, such as attending a more selective college, while nontest measures have more explanatory power for whether students enroll in college at all.
This is an updated version of the paper originally titled "Teachers and Students’ Postsecondary Outcomes: Testing the Predictive Power of Test and Nontest Teacher Quality Measures" (WP 270-1022), released in October 2022.
Citation: Benjamin Backes, James Cowan, Dan Goldhaber, Roddy Theobald (2023). How to Measure a Teacher: The Influence of Test and Nontest Value-Added on Long-Run Student Outcomes. CALDER Working Paper No. 270-0423-2
We use over 25 years of longitudinal data from Washington state to provide a descriptive portrait of the paraeducator workforce in the state. Paraeducators are more racially and ethnically diverse than special education teachers, particularly in the last decade, and tend to be less experienced. They also have full-time salaries that are about half of the average special education teacher salary. Paraeducator-to-student ratios have decreased over time in the state, but they are higher in schools serving more students of color. Finally, and perhaps most importantly, paraeducator attrition rates from the state’s public school workforce have increased dramatically over time; for example, the paraeducator attrition rate after the 2021-22 school year (23%) was over twice as high as the attrition rate after the 2008-09 school year (8%). These findings have implications for how policymakers and school leaders should approach decision-making related to the paraeducator workforce, as well as how researchers might approach further research with this group of educators.
Citation: Roddy Theobald, Lindsey Kaler, Elizabeth Bettini, Nathan Jones (2023). A Descriptive Portrait of the Paraeducator Workforce in Washington State. CALDER Working Paper No. 283-0423
Several decades of research using school administrative data show that teacher quality is inequitably distributed across schools. But these estimates may understate teacher-related inequities if they do not account for how teacher vacancies or late hires are distributed across schools. We investigate these hiring issues using data on a direct proxy of school hiring needs: teacher job postings collected from public school district websites. These data allow us to document how, over the course of the school year, hiring needs vary across districts, schools, and subject areas. We find that schools serving more students of color have greater hiring needs throughout the hiring cycle. We also find that hiring needs for special education and STEM positions are consistently higher than hiring needs for elementary positions. Schools with growing enrollments, as well as schools and subjects with higher prior attrition rates, also tend to have more job postings. Postings for schools in towns and rural areas tend to stay open longer than for schools in suburban and urban areas. Finally, we validate that job postings, which can be obtained quickly and inexpensively, are a good indicator of school and district needs in that they closely line up with eventual teacher hires.
Citation: Dan Goldhaber, Grace Falken, Roddy Theobald (2023). What Do Teacher Job Postings Tell Us about School Hiring Needs and Equity?. CALDER Working Paper No. 282-0323
A fundamental question for education policy is whether outcomes-based accountability, including comprehensive educator evaluations and a closer relationship between effectiveness and compensation, improves the quality of instruction and raises achievement. We use synthetic control methods to study the comprehensive teacher and principal evaluation and compensation systems introduced in the Dallas Independent School District (Dallas ISD) in 2013 for principals and 2015 for teachers. Under this far-reaching reform, educator evaluations that are used to support teacher growth and determine salary depend on a combination of supervisor evaluations, student achievement, and student or family survey responses. The reform replaced salary scales based on experience and educational attainment with those based on evaluation scores, a radical departure from decades of rigid salary schedules. The synthetic control estimates reveal positive and significant effects of the reforms on math and reading achievement that increase over time. From 2015 through 2019, the average achievement for the synthetic control district fluctuates narrowly between -0.27 s.d. and -0.3 s.d., while the Dallas ISD average increases steadily from -0.28 s.d. in 2015 to -0.08 s.d. in 2019, the final year of the sample. Though the increase for reading is roughly half as large, it is also highly significant.
Citation: Eric Hanushek, Jin Luo, Andrew Morgan, Minh Nguyen, Ben Ost, Steven Rivkin, Ayman Shakeel (2023). The Effects of Comprehensive Educator Evaluation and Pay Reform on Achievement. CALDER Working Paper No. 281-0323
Efforts to attract and retain effective educators in high-poverty public schools have had limited success. Dallas ISD addressed this challenge with information produced by its evaluation system to offer large, compensating differentials to highly effective teachers willing to work in its lowest-achievement schools. The Accelerating Campus Excellence (ACE) program resulted in immediate and sustained achievement increases. The improvements were dramatic, bringing average achievement in the previously lowest-performing schools close to the district average. When ACE stipends are largely eliminated, a substantial fraction of highly effective teachers leave, and test scores fall. This highlights the central importance of performance-based incentives.
Citation: Andrew Morgan, Minh Nguyen, Eric Hanushek, Ben Ost, Steven Rivkin (2023). Attracting and Retaining Highly Effective Educators in Hard-to-Staff Schools. CALDER Working Paper No. 280-0323
Anecdotal evidence points to the importance of school principals, but the limited existing research has neither provided consistent results nor indicated any set of essential characteristics of effective principals. This paper exploits extensive student-level panel data across six states to investigate both variations in principal performance and the relationship between effectiveness and key certification factors. While principal effectiveness varies widely across states, there is little indication that regulation of the background and training of principals yields consistently effective performance. Having prior teaching or management experience is not related to our estimates of principal value-added.
This is an updated version of the paper originally titled "Path to the Principalship and Value Added: A Cross-state Comparison of Elementary and Middle School Principals", released in January 2019.
Citation: Wes Austin, Bingjie Chen, Dan Goldhaber , Eric Hanushek, Kristian Holden, Cory Koedel, Helen Ladd, Jin Luo, Eric Parsons, Gregory Phelan, Steven Rivkin, Tim Sass, Mavzuna Turaeva (2023). Does Regulating Entry Requirements Lead to More Effective Principals?. CALDER Working Paper No. 213-0323-2
The decision to close schools in March 2020 was a massive disruption to public education. But pandemic-related closures did more than put in-person instruction on hold. School closures also meant that teacher candidates could not complete their student teaching experiences (Choate et al., 2021). And when licensure test centers closed, prospective teachers could not sit for the exams they needed to get credentialed. As COVID wreaked havoc on the school system—and worries over staffing shortages grew—most states responded to these disruptions by modifying and relaxing their requirements for becoming a teacher during the pandemic.
Three years later, the negative consequences of school closures for students are well known: learning declined and pre-existing inequities grew (Goldhaber et al., 2022). But how (if at all) changes to licensure requirements affected teachers and students is less clear. To provide an initial picture of what happened, this research brief describes the nature of pandemic-era licensure modifications, how many teachers they might have affected, and why we should care. Our rough estimate suggests that around 100,000 graduates of traditional preparation programs might have entered the profession under changed licensure rules in 2020-2021.
Citation: Michael DeArmond, Dan Goldhaber, Sydney Payne (2023). COVID's Under-the-Radar Experiment with Teacher Licensure. CALDER Policy Brief No. 33
We use administrative panel data from seven states covering nearly 3 million students to document and explore variation in “academic mobility,” a term we use to describe the extent to which students’ ranks in the distribution of academic performance change during their public schooling careers. On average, we show that student ranks are highly persistent during elementary and secondary education—that is, academic mobility is limited in U.S. schools as a whole. Still, there is non-negligible variation in the degree of upward mobility across some student subgroups as well as individual school districts. On average, districts that exhibit the greatest upward academic mobility serve more socioeconomically advantaged populations and have higher value-added to student achievement.
This is an update of the August 2021 version of this paper, which was originally published February 2020 with the title "Where are Initially Low-performing Students the Most Likely to Succeed? A Multi-state Analysis of Academic Mobility".
Citation: Wes Austin, David Figlio, Dan Goldhaber, Eric Hanushek, Tara Kilbride, Cory Koedel, Jaeseok Sean Lee, Jin Luo, Umut Özek, Eric Parsons, Steven Rivkin, Tim Sass, Katharine O. Strunk (2023). Academic Mobility in U.S. Public Schools: Evidence from Nearly 3 Million Students. CALDER Working Paper No. 227-0323-3
Reading has been at the forefront of early-grade educational interventions, but addressing the educational needs of students in math early on is also critical given that early gaps in math skills widen further over the course of schooling. In this study, we examine the effects of Kentucky’s Math Achievement Fund – a unique state-level program that combines targeted interventions, peer-coaching, and close collaboration among teachers to improve math achievement in grades K-3 – on student outcomes and the costs associated with this policy. We find significant positive effects of the program not only on math achievement, but also on test scores in reading and non-test outcomes including student attendance and disciplinary incidents. The benefits exist across racial/ethnic groups and students from different socioeconomic statuses, and they are slightly higher for racial minorities. These findings, along with the cost estimate of the program, suggest that this program could provide a cost-effective blueprint to address the educational needs of students in math in early grades.
Citation: Zeyu Xu, Umut Özek, Jesse Levin, Dong Hoon Lee (2023). Effects of Large-Scale Early Math Interventions on Student Outcomes: Evidence from Kentucky’s Math Achievement Fund. CALDER Working Paper No. 279-0323
This study examines the extent to which students obtain postsecondary credentials in the CTE fields of focus they choose in high school. Using school fixed effects models, we find that focusing on a particular CTE field in high school is associated with an increased probability of enrolling and obtaining a postsecondary credential in that field. The secondary-postsecondary relationship varies across focus areas, and it is strongest in health (increase of 12.5 percentage points), which is disproportionately chosen by females. Across all fields of focus, however, most students enroll and obtain a postsecondary credential in fields that are different from what they focused on in high school.
This is an updated version of the paper originally released in July 2022.
Citation: Zeyu Xu, Benjamin Backes (2023). Linkage Between Fields of Concentration in High School Career-Technical Education and College Majors. CALDER Working Paper No. 269-0323-2
We examine the potential to expand and diversify the production of university STEM degrees by shifting the margin of initial enrollment between community colleges and 4-year universities. Our analysis is based on statewide administrative microdata from the Missouri Department of Higher Education and Workforce Development covering enrollees in all public postsecondary institutions statewide. We find that the potential for shifting the enrollment margin to expand degree production in STEM fields is modest, even at an upper bound, because most community college students are not academically prepared for bachelor’s degree programs in STEM fields. We also find that shifting the enrollment margin is unlikely to improve racial/ethnic diversity among university STEM degree recipients. This is because community college students at the enrollment margin are less diverse than their peers who enter universities directly.
Citation: Cheng Qian, Cory Koedel (2023). Could Shifting the Margin between Community College and University Enrollment Expand and Diversify University Degree Production in STEM Fields?. CALDER Working Paper No. 244-0323-2
