You are here
There is empirical evidence of substantial heterogeneity in economic mobility across geographic areas and the efficacy of schools has been suggested as an explanatory factor. Using administrative microdata from seven states covering nearly 3 million students, we explore the potential role of schools in promoting economic mobility by estimating cross-district variation in “academic mobility”—a term we use to describe the extent to which students’ ranks in the distribution of academic performance change during their schooling careers. We show that there exists considerable heterogeneity in academic mobility across school districts. However, after aggregating our district-level measures of academic mobility to the commuting-zone level and merging them with geographically matched external estimates of economic mobility, we find little scope for geographic differences in academic mobility to meaningfully account for differences in economic mobility.
This is an updated version of the paper originally titled "Where are Initially Low-performing Students the Most Likely to Succeed? A Multi-state Analysis of Academic Mobility (Preliminary Draft)", released in February 2020.
Citation: Wes Austin, David Figlio, Dan Goldhaber, Eric Hanushek, Tara Kilbride, Cory Koedel, Jaeseok Sean Lee, Jin Lou, Umut Özek, Eric Parsons, Steven Rivkin, Tim Sass, Katharine Strunk (2021). Academic Mobility in U.S. Public Schools: Evidence from Nearly 3 Million Students. CALDER Working Paper No. 227-0821-2
CALDER Policy Brief No. 25-0421
We use comprehensive administrative data from three states to document the relationships between geographic mobility and student outcomes during K-12 schooling. We focus specifically on nonstructural mobility events—which we define as school or district changes that do not occur naturally as the result of planned transitions between schools—and on longitudinal measures that capture these events cumulatively for students. We show that the number of nonstructural moves experienced by a student is a powerful indicator of low academic performance and graduation rates. Longitudinal information on student mobility is unlikely to be readily available to local practitioners—i.e., individual districts, schools, or teachers. However, due to recent investments in longitudinal data systems in most states, this information can be made available at low cost by state education agencies.
Citation: Dan Goldhaber, Cory Koedel, Umut Özek, Eric Parsons (2021). Using Longitudinal Student Mobility Across Schools and Districts to Identify At-Risk Students. CALDER Working Paper No.
This study examines the effects of internal migration driven by severe natural disasters on host communities, and the mechanisms behind these effects, using the large influx of migrants into Florida public schools after Hurricane Maria. I find adverse effects of the influx in the first year on existing student test scores, disciplinary problems, and student mobility among high-performing students in middle and high school that also persist in the second year. I also find evidence that compensatory resource allocation within schools is an important factor driving the adverse effects of large, unexpected migrant flows on incumbent students in the short-run.
This paper was published in The Journal of Human Resources in January 2021 and can be found here.
WP 233-0320 was originally released in March 2020. This updated version, WP 233-0320-2, was released in January 2021.
Citation: Umut Özek (2020). Examining the Educational Spillover Effects of Severe Natural Disasters: The Case of Hurricane Maria . CALDER Working Paper No. 233-0320-2
We study the effects of exposure to non-resident students on the outcomes of undergraduate in-state students during a period of high non-resident enrollment growth at the University of Missouri-Columbia. Our models leverage within-major, cross-time variation in non-resident exposure for identification. We find no evidence that increased exposure to domestic non-residents affects in-state student outcomes and our null results are precisely estimated. We find evidence of modest negative impacts on in-state students when their exposure to foreign students increases using our preferred specification. However, the identifying variation in exposure to foreign students in our data is limited and this result is not robust in all of our models.
This paper has been published in Contemporary Economic Policy and can be found here, January 2020.
Citation: Diyi Li, Cheng Qian , Cory Koedel (2020). Non-Resident Postsecondary Enrollment Growth and the Outcomes of In-State Students. CALDER Working Paper No. 225-0120
We leverage nationally representative data and statewide data from Washington to investigate trends in occupational career and technical education (CTE) participation for students with and without disabilities. Consistent with prior work, we document declines in occupational CTE participation since the early 2000s, and provide the first empirical evidence that students with disabilities disproportionately contributed to this decline. But we also show that occupational CTE participation has stabilized for all students in the past decade in Washington, and that participation by students with disabilities in applied science, technology, engineering, mathematics, and medicine (applied STEMM) CTE courses has increased since the early 2000s. These trends are encouraging given prior evidence linking applied STEMM-CTE participation to better long-term outcomes for students with disabilities.
Citation: Roddy Theobald, Jay Plasman, Michael Gottfried, Trevor Gratz, Kristian Holden, Dan Goldhaber (2019). Sometimes Less, Sometimes More: Trends in Career and Technical Education Participation for Students With Disabilities. CALDER Working Paper No. 220-0819
An increasing emphasis on principals as key to school improvement has contributed to efforts to elevate principal effectiveness that have taken various forms across the US. The primacy of the state as the focal point of educational reform elevates the value of understanding commonalities and differences among states in characteristics of principals, the distribution of principals among schools and ultimately the policies associated with more effective school leadership, particularly for disadvantaged children. This paper describes major state policies, the distribution of elementary school principals among schools along a several dimensions, and pathways to the principalship to illustrate similarities and differences among six states in the tenure and experience distributions and how these vary by student demographic characteristics and district size. Measurement of principal effectiveness and its relationship with principal characteristics and state policies would be ideal, but complications introduced by the dynamics of principal influences and confounding effects of other factors inhibit this effort. Nonetheless, school value added to achievement provides information on differences in principal effectiveness, and we report within-school variation value added across principal regimes and the associations between value added and principal characteristics. The analysis reveals many similarities and some differences among the states, some of which are related to differences in governance structures. Perhaps the most striking differences relate to the pathways to the principalship including the fraction of principals with experiences as assistant principals and teachers.
Citation: Wes Austin, Bingjie Chen, Dan Goldhaber , Eric Hanushek, Kristian Holden, Cory Koedel, Helen Ladd, Jin Luo, Eric Parsons, Gregory Phelan, Steven Rivkin, Tim Sass, Mavzuna Turaeva (2019). Path to the Principalship and Value Added: A Cross-state Comparison of Elementary and Middle School Principals. CALDER Working Paper No. 213-0119-1
Accurate understanding of environmental moderation of genetic influences is vital to advancing the science of cognitive development as well as for designing interventions. One widely- reported idea is increasing genetic influence on cognition for children raised in higher socioeconomic status families, including recent proposals that the pattern is a particularly US phenomenon. We use matched birth and school records from Florida siblings and twins born in 1994-2002 to provide the largest, most population-diverse consideration of this hypothesis to date. We find no evidence of SES moderation of genetic influence on test scores, suggesting that articulating gene-environment interactions for cognition is more complex and elusive than previously supposed.
Citation: David Figlio, Jeremy Freese, Krzysztof Karbownik, Jeffrey Roth (2018). Socioeconomic Status and Genetic Influences on Cognitive Development. CALDER Working Paper No. 193
It is notoriously difficult to identify peer effects within the family because of the common shocks and reflection problems. We make use of a novel identification strategy and unique data in order to gain some purchase on this problem. We employ data from the universe of children born in Florida between 1994 and 2002 and in Denmark between 1990 and 2001, which we match to school and medical records. To address the identification problem, we examine the effects of having a sibling with a disability. Utilizing three-plus-child families, we employ a differences-in- differences research design which makes use of the fact that birth order influences the amount of time that a child spends in early childhood with their siblings, disabled or not. We observe consistent evidence in both locations that the second child in a family is differentially affected when the third child is disabled.
Citation: Sandra E. Black, Sanni Breining, David Figlio, Jonathan Guryan, Krzysztof Karbownik, Helena Skyt Nielsen, Jeffrey Roth, Marianne Simonsen (2018). Sibling Spillovers. CALDER Working Paper No. 192
This study examines the community-wide effects of two statewide early childhood policy initiatives in North Carolina. One initiative provides funding to improve the quality of child care services at the county level for all children between the ages of 0 to 5, and the other provides funding for preschool slots for disadvantaged four-year-olds. Differences across counties in the timing of the rollout and in the magnitude of the state financial investments per child provide the variation in programs needed to estimate their effects on schooling outcomes in third grade. We find robust positive effects of each program on third-grade test scores in both reading and math. These effects can best be explained by a combination of direct benefits for participants and spillover benefits for others. Our preferred models suggest that the combined average effects on test scores of investments in both programs at 2009 funding levels are equivalent to two to four months of instruction in grade 3.
Citation: Helen Ladd, Clara G. Muschkin, Kenneth A. Dodge (2015). From Birth to School: Early Childhood Initiatives and Third-Grade Outcomes in North Carolina. CALDER Working Paper No. 134
In this paper, we present a closer look at the student achievement trends in the District of Columbia between 2006-07 and 2012-13. We have three main conclusions. First, we find that overall, math scores in the District have improved. The improvements in reading scores during this time frame, however, were primarily limited to the first year after the PERAA implementation. While almost all student subgroups have experienced test score gains in math, these improvements were higher among the more affluent black and Hispanic students. Second, we find that these observed trends in math scores persist even after controlling for the cross-cohort differences in observed student characteristics. In particular, the estimates indicate that less than 10 percent of the year-to-year improvements in test scores can be attributed to the changing student composition in the District over this time frame. Finally, we show that existing students have also experienced gains in math even though the students who are new to the District’s public school system score at higher levels on standardized tests when compared to existing students.
Citation: Umut Özek (2014). A Closer Look at the Student Achievement Trends in the District of Columbia between 2006-07 and 2012-13. CALDER Working Paper No. 119
Over 40% of full time four-year college students fail to earn a bachelor’s degree within six years, and many never complete their education. This paper describes this sizeable fraction of the U.S. higher education market and estimates counterfactual predicted probabilities of degree completion, had students made different initial postsecondary enrollment choices. Using data from the NLSY97, a rich nationally representative data set, we make several observations. First, policies aimed at increasing postsecondary degree attainment by encouraging college enrollment are likely to be unproductive, given that students who are currently not enrolling in postsecondary education have very low predicted probabilities of completion, due to their low academic preparedness. This holds true for enrollment in both two-year and four-year colleges. Second, we find that students who drop-out of four-year colleges generally also have very low predicted probabilities of completion, although this varies across student groups. Finally, we conclude that had four-year college drop-outs begun their postsecondary careers at a two-year college, their predicted probabilities of postsecondary degree completion would be significantly higher. While most of this increase in degree completion comes through increased associate’s degree attainment, about a third of four-year college drop-outs would have a higher chance of bachelor’s degree completion, had they begun college at a two-year institution. While our results are only a descriptive analysis, and should not be interpreted as causal findings, until more is understood about the types of students who drop-out of college and potential reasons why, there will likely be little progress in reducing the college failure rate in the U.S.
Citation: Erin Dunlop Velez (2014). America’s College Drop-Out Epidemic: Understanding the College Drop-Out Population. CALDER Working Paper No. 109
Current federal education policies promote learning in Science, Technology, Engineering, and Math (STEM) and the participation of minority students in these fields. Using longitudinal data on students in Florida and North Carolina, value-added estimates in math and science are generated to categorize schools into performance levels and identify differences in school STEM measures by performance levels. Several STEM-relevant variables show a significant association with effectiveness in math and science, including STEM teacher turnover, calculus and early algebra participation, and math and science instructional indices created from survey items in the data. Surprisingly, a negative association between students’ STEM course participation and success in STEM is consistently documented across both states, in addition to low participation of underrepresented minority students in successful schools in STEM.
Citation: Michael Hansen (2013). Characteristics of Schools Successful in STEM: Evidence from Two States’ Longitudinal Data. CALDER Working Paper No. 97
Students who do not have access to credit may not be able to complete their optimal level of post-secondary education. More than one out of every ten community college students nationwide attends a community college that does not allow access to federal college loans. This paper takes advantage of plausibly exogenous variation in whether a student's community college offers student loans to evaluate the effect of access to Stafford loans on student outcomes, including educational attainment, employment, and finances. Using the Beginning Postsecondary Student Study of 2004, I show that access to federal Stafford loans does not affect the decision to attend community college. However, I find that Stafford loan access increases overall borrowing among community college students by $262 a year and increases the likelihood of transferring to a four-year school by 5.6 percentage points. Additionally, for high-need and female students, loan access increases their total months of enrollment and dependent students' bachelor's degree attainment as well. These sizable effects of loan access on student behavior indicate that federal loans relax credit constraints for some community college students.
Citation: Erin Dunlop Velez (2013). What Do Stafford Loans Actually Buy You? - The Effect of Stafford Loan Access on Community College Students. CALDER Working Paper No. 94
This paper contributes to the empirical literature on remediation in community colleges by using policy variation across North Carolina’s community colleges to examine how remediation affects various outcomes for traditional-age college students. We find that being required to take a remedial course (as we define it in this paper) either in math or in English significantly reduces a student’s probability of success in college and also the probability that a student ever passes a college-level math or English course. Among students who are required to take a remedial course in their first semester, however, we find no adverse effects on the probability of returning for another semester. We also find differential effects by a student’s prior achievement level, family income and gender. Despite the difference in the methodologies, our main findings are generally consistent with, albeit somewhat more negative, than those from prior studies based on regression discontinuity designs.
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor, Clara Muschkin (2013). Developmental Education in North Carolina Community Colleges. CALDER Working Paper No. 88
There has been a resurgence in research that investigates the efficacy of early investments as a means of reducing gaps in academic performance. However, the strongest evidence for these effects comes from experimental evaluations of small, highly enriched programs. We add to this literature by assessing the extent to which a large-scale public program, Texas's targeted pre-Kindergarten (pre-K), affects scores on math and reading achievement tests, the likelihood of being retained in grade, and the probability that a student receives special education services. We find that having participated in Texas's targeted pre-K program is associated with increased scores on the math and reading sections of the Texas Assessment of Academic Skills (TAAS), reductions in the likelihood of being retained in grade, and reductions in the probability of receiving special education services. We also find that participating pre-K increases mathematics scores for students who take the Spanish version of the TAAS tests. These results show that even modest, public pre-K program implemented at scale can have important effects on students’ educational achievement.
Citation: Rodney J. Andrews, Paul Jargowsky, Kristin Kuhne (2012). The Effects of Texas’s Targeted Pre-Kindergarten Program on Academic Performance. CALDER Working Paper No. 84
This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools — which comprise three levels of clustering (students in classrooms in schools) — to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises frequently in practice because many available data sets identify the schools that students attend but not the classrooms in which they are taught. Do studies conducted under these circumstances yield results that are substantially different from what they would have been if this information had been available? The paper first considers this problem in the context of planning a school randomized study based on preexisting two-level information about how academic outcomes for students vary across schools and across students within schools (but not across classrooms in schools). The paper next considers this issue in the context of estimating intervention effects from school-randomized studies. Findings are based on empirical analyses of four multisite data sets using academic outcomes for students within classrooms within schools. The results indicate that in almost all situations one will obtain nearly identical results whether or not the classroom or middle level is omitted when designing or analyzing studies.
Citation: Pei Zhu, Robin Jacob, Howard Bloom, Zeyu Xu (2011). Designing and Analyzing Studies that Randomize Schools To Estimate Intervention Effects on Student Academic Outcomes Without Classroom-Level Information. CALDER Working Paper No. 61
Does differential access to computer technology at home compound the educational disparities between the rich and the poor? Would a program of government provision of computers to early secondary school students reduce these disparities? This study covers years 2000 to 2005, a period when home computers and high-speed Internet access expanded dramatically. Using administrative data on North Carolina public school students to corroborate earlier surveys that document broad racial and socioeconomic gaps in home computer access and use, the authors compared the children's reading and math scores before and after they acquired a home computer, and compared these scores to those of peers who had a home computer by fifth grade and to test scores of students who never acquired a home computer. The introduction of home computer technology is associated with modest but statistically significant and persistent negative impacts on student math and reading test scores. The authors also conclude that home computers are put to more productive use in households where parental monitoring is more effective. Further evidence suggests that providing universal access to home computers and high-speed internet access would broaden, rather than narrow, math and reading achievement gaps.
Citation: Jacob Vigdor, Helen Ladd (2010). Scaling the Digital Divide: Home Computer Technology and Student Achievement. CALDER Working Paper No. 48
The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms.
Citation: Zeyu Xu, Austin Nichols (2010). New Estimates of Design Parameters for Clustered Randomization Studies: Findings from North Carolina and Florida. CALDER Working Paper No. 43
This paper describes the school mobility rates for elementary and middle school students in North Carolina and attempts to estimate the effect of school mobility on the performance of different groups of students using student fixed effects models. School mobility is defined as changing schools at times that are non-promotional (e.g., moving from middle to high school). We used detailed administrative data on North Carolina students and schools from 1996 to 2005 and followed four cohorts of 3rd graders for six years each. School mobility rates were highest for minority and disadvantaged students. School mobility rates for Hispanic students declined across successive cohorts, but increased for Black students. Findings on effects were most pronounced in math. School mobility hurt the math performance of Black and Hispanic students, but not the math performance of white students. School mobility improved the reading performance of white and more advantaged students, but had no effect on the reading performance of minority students. "Strategic" school moves (cross-district) benefitted or had no effect on student performance, but "reactive" moves (within district) hurt all groups of students. White and Hispanic students were more likely to move to a higher quality school while Blacks were more likely to move to a lower quality school. The negative effects of school mobility increased with the number of school moves.
Citation: Zeyu Xu, Jane Hannaway, Stephanie D'Souza (2009). Student Transience in North Carolina:The Effect of School Mobility on Student Outcomes Using Longitudinal Data. CALDER Working Paper No. 22
This research brief estimates the overall extent of test measurement error and how this varies across students using New York City student- level longitudinal data across grades 3-8 from 1999- 2007. Results reinforce the importance of accounting for measurement error, as it meaningfully increases effect size estimates associated with teacher attributes. There are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important in the formulation and implementation of personnel policies. Also, effect sizes as traditionally measured have led analysts to understate the magnitudes of effects because the standard deviation of observed scores overstates the dispersion of true achievement in the student population.
Citation: Donald Boyd, Pamela Grossman, Hamilton Lankford, Susanna Loeb, James Wyckoff (2008). Overview of Measuring Effect Sizes: The Effect of Measurement Error. CALDER Working Paper No.