You are here
Tutoring contracts cannot address all the take-up and implementation challenges associated with academic interventions. They also cannot ensure that all students catch up from COVID losses. Districts and vendors both need to find ways to build momentum, capacity, and, in some cases, urgency among teachers, students, and families to make these programs work. But contracts can create incentives for vendors to focus on session attendance, tutoring dosage, and results; on the margin, such incentives might get vendors to pay more attention to the student outcomes that districts care about most.
If school systems are going to deliver the academic support necessary for recovery, they need clear-eyed assessments of their current catch-up efforts. And if they are working with an external partner, they need agreements with incentives that put everyone’s attention and effort on what matters most: delivering academic support to help students catch up from the chaos of the last three years. Although there is not much evidence yet about the impact of different contract provisions, we should all be paying attention to whether contract language focuses vendors on ensuring that students receive the services that districts are paying for.
Citation: Michael DeArmond, Dan Goldhaber, Thomas J. Kane, Andrew McEachin, Emily Morton (2023). Paying for Access: Are Tutoring Contracts Focused on the Right Things?. CALDER Policy Brief No. 34
We use over 25 years of longitudinal data from Washington state to provide a descriptive portrait of the paraeducator workforce in the state. Paraeducators are more racially and ethnically diverse than special education teachers, particularly in the last decade, and tend to be less experienced. They also have full-time salaries that are about half of the average special education teacher salary. Paraeducator-to-student ratios have decreased over time in the state, but they are higher in schools serving more students of color. Finally, and perhaps most importantly, paraeducator attrition rates from the state’s public school workforce have increased dramatically over time; for example, the paraeducator attrition rate after the 2021-22 school year (23%) was over twice as high as the attrition rate after the 2008-09 school year (8%). These findings have implications for how policymakers and school leaders should approach decision-making related to the paraeducator workforce, as well as how researchers might approach further research with this group of educators.
Citation: Roddy Theobald, Lindsey Kaler, Elizabeth Bettini, Nathan Jones (2023). A Descriptive Portrait of the Paraeducator Workforce in Washington State. CALDER Working Paper No. 283-0423
Anecdotal evidence points to the importance of school principals, but the limited existing research has neither provided consistent results nor indicated any set of essential characteristics of effective principals. This paper exploits extensive student-level panel data across six states to investigate both variations in principal performance and the relationship between effectiveness and key certification factors. While principal effectiveness varies widely across states, there is little indication that regulation of the background and training of principals yields consistently effective performance. Having prior teaching or management experience is not related to our estimates of principal value-added.
This is an updated version of the paper originally titled "Path to the Principalship and Value Added: A Cross-state Comparison of Elementary and Middle School Principals", released in January 2019.
Citation: Wes Austin, Bingjie Chen, Dan Goldhaber , Eric Hanushek, Kristian Holden, Cory Koedel, Helen Ladd, Jin Luo, Eric Parsons, Gregory Phelan, Steven Rivkin, Tim Sass, Mavzuna Turaeva (2023). Does Regulating Entry Requirements Lead to More Effective Principals?. CALDER Working Paper No. 213-0323-2
We use administrative panel data from seven states covering nearly 3 million students to document and explore variation in “academic mobility,” a term we use to describe the extent to which students’ ranks in the distribution of academic performance change during their public schooling careers. On average, we show that student ranks are highly persistent during elementary and secondary education—that is, academic mobility is limited in U.S. schools as a whole. Still, there is non-negligible variation in the degree of upward mobility across some student subgroups as well as individual school districts. On average, districts that exhibit the greatest upward academic mobility serve more socioeconomically advantaged populations and have higher value-added to student achievement.
This is an update of the August 2021 version of this paper, which was originally published February 2020 with the title "Where are Initially Low-performing Students the Most Likely to Succeed? A Multi-state Analysis of Academic Mobility".
Citation: Wes Austin, David Figlio, Dan Goldhaber, Eric Hanushek, Tara Kilbride, Cory Koedel, Jaeseok Sean Lee, Jin Luo, Umut Özek, Eric Parsons, Steven Rivkin, Tim Sass, Katharine O. Strunk (2023). Academic Mobility in U.S. Public Schools: Evidence from Nearly 3 Million Students. CALDER Working Paper No. 227-0323-3
CALDER Policy Brief No. 25-0421
We use comprehensive administrative data from three states to document the relationships between geographic mobility and student outcomes during K-12 schooling. We focus specifically on nonstructural mobility events—which we define as school or district changes that do not occur naturally as the result of planned transitions between schools—and on longitudinal measures that capture these events cumulatively for students. We show that the number of nonstructural moves experienced by a student is a powerful indicator of low academic performance and graduation rates. Longitudinal information on student mobility is unlikely to be readily available to local practitioners—i.e., individual districts, schools, or teachers. However, due to recent investments in longitudinal data systems in most states, this information can be made available at low cost by state education agencies.
This paper has been published in AERA Open and can be found here, January 2022.
Citation: Dan Goldhaber, Cory Koedel, Umut Özek, Eric Parsons (2021). Using Longitudinal Student Mobility Across Schools and Districts to Identify At-Risk Students. CALDER Policy Brief No. 25
This study examines the effects of internal migration driven by severe natural disasters on host communities, and the mechanisms behind these effects, using the large influx of migrants into Florida public schools after Hurricane Maria. I find adverse effects of the influx in the first year on existing student test scores, disciplinary problems, and student mobility among high-performing students in middle and high school that also persist in the second year. I also find evidence that compensatory resource allocation within schools is an important factor driving the adverse effects of large, unexpected migrant flows on incumbent students in the short-run.
This paper was published in The Journal of Human Resources in January 2021 and can be found here.
WP 233-0320 was originally released in March 2020. This updated version, WP 233-0320-2, was released in January 2021.
Citation: Umut Özek (2020). Examining the Educational Spillover Effects of Severe Natural Disasters: The Case of Hurricane Maria . CALDER Working Paper No. 233-0320-2
We study the effects of exposure to non-resident students on the outcomes of undergraduate in-state students during a period of high non-resident enrollment growth at the University of Missouri-Columbia. Our models leverage within-major, cross-time variation in non-resident exposure for identification. We find no evidence that increased exposure to domestic non-residents affects in-state student outcomes and our null results are precisely estimated. We find evidence of modest negative impacts on in-state students when their exposure to foreign students increases using our preferred specification. However, the identifying variation in exposure to foreign students in our data is limited and this result is not robust in all of our models.
This paper has been published in Contemporary Economic Policy and can be found here, January 2020.
Citation: Diyi Li, Cheng Qian, Cory Koedel (2020). Non-Resident Postsecondary Enrollment Growth and the Outcomes of In-State Students. CALDER Working Paper No. 225-0120
We leverage nationally representative data and statewide data from Washington to investigate trends in occupational career and technical education (CTE) participation for students with and without disabilities. Consistent with prior work, we document declines in occupational CTE participation since the early 2000s, and provide the first empirical evidence that students with disabilities disproportionately contributed to this decline. But we also show that occupational CTE participation has stabilized for all students in the past decade in Washington, and that participation by students with disabilities in applied science, technology, engineering, mathematics, and medicine (applied STEMM) CTE courses has increased since the early 2000s. These trends are encouraging given prior evidence linking applied STEMM-CTE participation to better long-term outcomes for students with disabilities.
Citation: Roddy Theobald, Jay Plasman, Michael Gottfried, Trevor Gratz, Kristian Holden, Dan Goldhaber (2019). Sometimes Less, Sometimes More: Trends in Career and Technical Education Participation for Students With Disabilities. CALDER Working Paper No. 220-0819
Accurate understanding of environmental moderation of genetic influences is vital to advancing the science of cognitive development as well as for designing interventions. One widely- reported idea is increasing genetic influence on cognition for children raised in higher socioeconomic status families, including recent proposals that the pattern is a particularly US phenomenon. We use matched birth and school records from Florida siblings and twins born in 1994-2002 to provide the largest, most population-diverse consideration of this hypothesis to date. We find no evidence of SES moderation of genetic influence on test scores, suggesting that articulating gene-environment interactions for cognition is more complex and elusive than previously supposed.
Citation: David Figlio, Jeremy Freese, Krzysztof Karbownik, Jeffrey Roth (2018). Socioeconomic Status and Genetic Influences on Cognitive Development. CALDER Working Paper No. 193
It is notoriously difficult to identify peer effects within the family because of the common shocks and reflection problems. We make use of a novel identification strategy and unique data in order to gain some purchase on this problem. We employ data from the universe of children born in Florida between 1994 and 2002 and in Denmark between 1990 and 2001, which we match to school and medical records. To address the identification problem, we examine the effects of having a sibling with a disability. Utilizing three-plus-child families, we employ a differences-in- differences research design which makes use of the fact that birth order influences the amount of time that a child spends in early childhood with their siblings, disabled or not. We observe consistent evidence in both locations that the second child in a family is differentially affected when the third child is disabled.
Citation: Sandra E. Black, Sanni Breining, David Figlio, Jonathan Guryan, Krzysztof Karbownik, Helena Skyt Nielsen, Jeffrey Roth, Marianne Simonsen (2018). Sibling Spillovers. CALDER Working Paper No. 192
This study examines the community-wide effects of two statewide early childhood policy initiatives in North Carolina. One initiative provides funding to improve the quality of child care services at the county level for all children between the ages of 0 to 5, and the other provides funding for preschool slots for disadvantaged four-year-olds. Differences across counties in the timing of the rollout and in the magnitude of the state financial investments per child provide the variation in programs needed to estimate their effects on schooling outcomes in third grade. We find robust positive effects of each program on third-grade test scores in both reading and math. These effects can best be explained by a combination of direct benefits for participants and spillover benefits for others. Our preferred models suggest that the combined average effects on test scores of investments in both programs at 2009 funding levels are equivalent to two to four months of instruction in grade 3.
Citation: Helen Ladd, Clara Muschkin, Kenneth A. Dodge (2015). From Birth to School: Early Childhood Initiatives and Third-Grade Outcomes in North Carolina. CALDER Working Paper No. 134
In this paper, we present a closer look at the student achievement trends in the District of Columbia between 2006-07 and 2012-13. We have three main conclusions. First, we find that overall, math scores in the District have improved. The improvements in reading scores during this time frame, however, were primarily limited to the first year after the PERAA implementation. While almost all student subgroups have experienced test score gains in math, these improvements were higher among the more affluent black and Hispanic students. Second, we find that these observed trends in math scores persist even after controlling for the cross-cohort differences in observed student characteristics. In particular, the estimates indicate that less than 10 percent of the year-to-year improvements in test scores can be attributed to the changing student composition in the District over this time frame. Finally, we show that existing students have also experienced gains in math even though the students who are new to the District’s public school system score at higher levels on standardized tests when compared to existing students.
Citation: Umut Özek (2014). A Closer Look at the Student Achievement Trends in the District of Columbia between 2006-07 and 2012-13. CALDER Working Paper No. 119
Over 40% of full time four-year college students fail to earn a bachelor’s degree within six years, and many never complete their education. This paper describes this sizeable fraction of the U.S. higher education market and estimates counterfactual predicted probabilities of degree completion, had students made different initial postsecondary enrollment choices. Using data from the NLSY97, a rich nationally representative data set, we make several observations. First, policies aimed at increasing postsecondary degree attainment by encouraging college enrollment are likely to be unproductive, given that students who are currently not enrolling in postsecondary education have very low predicted probabilities of completion, due to their low academic preparedness. This holds true for enrollment in both two-year and four-year colleges. Second, we find that students who drop-out of four-year colleges generally also have very low predicted probabilities of completion, although this varies across student groups. Finally, we conclude that had four-year college drop-outs begun their postsecondary careers at a two-year college, their predicted probabilities of postsecondary degree completion would be significantly higher. While most of this increase in degree completion comes through increased associate’s degree attainment, about a third of four-year college drop-outs would have a higher chance of bachelor’s degree completion, had they begun college at a two-year institution. While our results are only a descriptive analysis, and should not be interpreted as causal findings, until more is understood about the types of students who drop-out of college and potential reasons why, there will likely be little progress in reducing the college failure rate in the U.S.
Citation: Erin Dunlop Velez (2014). America’s College Drop-Out Epidemic: Understanding the College Drop-Out Population. CALDER Working Paper No. 109
Current federal education policies promote learning in Science, Technology, Engineering, and Math (STEM) and the participation of minority students in these fields. Using longitudinal data on students in Florida and North Carolina, value-added estimates in math and science are generated to categorize schools into performance levels and identify differences in school STEM measures by performance levels. Several STEM-relevant variables show a significant association with effectiveness in math and science, including STEM teacher turnover, calculus and early algebra participation, and math and science instructional indices created from survey items in the data. Surprisingly, a negative association between students’ STEM course participation and success in STEM is consistently documented across both states, in addition to low participation of underrepresented minority students in successful schools in STEM.
Citation: Michael Hansen (2013). Characteristics of Schools Successful in STEM: Evidence from Two States’ Longitudinal Data. CALDER Working Paper No. 97
Students who do not have access to credit may not be able to complete their optimal level of post-secondary education. More than one out of every ten community college students nationwide attends a community college that does not allow access to federal college loans. This paper takes advantage of plausibly exogenous variation in whether a student's community college offers student loans to evaluate the effect of access to Stafford loans on student outcomes, including educational attainment, employment, and finances. Using the Beginning Postsecondary Student Study of 2004, I show that access to federal Stafford loans does not affect the decision to attend community college. However, I find that Stafford loan access increases overall borrowing among community college students by $262 a year and increases the likelihood of transferring to a four-year school by 5.6 percentage points. Additionally, for high-need and female students, loan access increases their total months of enrollment and dependent students' bachelor's degree attainment as well. These sizable effects of loan access on student behavior indicate that federal loans relax credit constraints for some community college students.
Citation: Erin Dunlop Velez (2013). What Do Stafford Loans Actually Buy You? - The Effect of Stafford Loan Access on Community College Students. CALDER Working Paper No. 94
This paper contributes to the empirical literature on remediation in community colleges by using policy variation across North Carolina’s community colleges to examine how remediation affects various outcomes for traditional-age college students. We find that being required to take a remedial course (as we define it in this paper) either in math or in English significantly reduces a student’s probability of success in college and also the probability that a student ever passes a college-level math or English course. Among students who are required to take a remedial course in their first semester, however, we find no adverse effects on the probability of returning for another semester. We also find differential effects by a student’s prior achievement level, family income and gender. Despite the difference in the methodologies, our main findings are generally consistent with, albeit somewhat more negative, than those from prior studies based on regression discontinuity designs.
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor, Clara Muschkin (2013). Developmental Education in North Carolina Community Colleges. CALDER Working Paper No. 88
There has been a resurgence in research that investigates the efficacy of early investments as a means of reducing gaps in academic performance. However, the strongest evidence for these effects comes from experimental evaluations of small, highly enriched programs. We add to this literature by assessing the extent to which a large-scale public program, Texas's targeted pre-Kindergarten (pre-K), affects scores on math and reading achievement tests, the likelihood of being retained in grade, and the probability that a student receives special education services. We find that having participated in Texas's targeted pre-K program is associated with increased scores on the math and reading sections of the Texas Assessment of Academic Skills (TAAS), reductions in the likelihood of being retained in grade, and reductions in the probability of receiving special education services. We also find that participating pre-K increases mathematics scores for students who take the Spanish version of the TAAS tests. These results show that even modest, public pre-K program implemented at scale can have important effects on students’ educational achievement.
Citation: Rodney J. Andrews, Paul Jargowsky, Kristin Kuhne (2012). The Effects of Texas’s Targeted Pre-Kindergarten Program on Academic Performance. CALDER Working Paper No. 84
This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools — which comprise three levels of clustering (students in classrooms in schools) — to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises frequently in practice because many available data sets identify the schools that students attend but not the classrooms in which they are taught. Do studies conducted under these circumstances yield results that are substantially different from what they would have been if this information had been available? The paper first considers this problem in the context of planning a school randomized study based on preexisting two-level information about how academic outcomes for students vary across schools and across students within schools (but not across classrooms in schools). The paper next considers this issue in the context of estimating intervention effects from school-randomized studies. Findings are based on empirical analyses of four multisite data sets using academic outcomes for students within classrooms within schools. The results indicate that in almost all situations one will obtain nearly identical results whether or not the classroom or middle level is omitted when designing or analyzing studies.
Citation: Pei Zhu, Robin Jacob, Howard Bloom, Zeyu Xu (2011). Designing and Analyzing Studies that Randomize Schools To Estimate Intervention Effects on Student Academic Outcomes Without Classroom-Level Information. CALDER Working Paper No. 61
Does differential access to computer technology at home compound the educational disparities between the rich and the poor? Would a program of government provision of computers to early secondary school students reduce these disparities? This study covers years 2000 to 2005, a period when home computers and high-speed Internet access expanded dramatically. Using administrative data on North Carolina public school students to corroborate earlier surveys that document broad racial and socioeconomic gaps in home computer access and use, the authors compared the children's reading and math scores before and after they acquired a home computer, and compared these scores to those of peers who had a home computer by fifth grade and to test scores of students who never acquired a home computer. The introduction of home computer technology is associated with modest but statistically significant and persistent negative impacts on student math and reading test scores. The authors also conclude that home computers are put to more productive use in households where parental monitoring is more effective. Further evidence suggests that providing universal access to home computers and high-speed internet access would broaden, rather than narrow, math and reading achievement gaps.
Citation: Jacob Vigdor, Helen Ladd (2010). Scaling the Digital Divide: Home Computer Technology and Student Achievement. CALDER Working Paper No. 48
The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms.
Citation: Zeyu Xu, Austin Nichols (2010). New Estimates of Design Parameters for Clustered Randomization Studies: Findings from North Carolina and Florida. CALDER Working Paper No. 43