You are here
Social policy and program impact
The COVID-19 pandemic initially resulted in an unanticipated and near-universal shift from in-person to virtual instruction in spring 2020. During the 2020-21 school year, schools began to re-open and families were faced with decisions regarding the instructional mode for their children. We leverage administrative, survey, and virtual-learning data to examine the determinants of family learning-mode choice and the effects of virtual education on student engagement and academic achievement. Family preference for virtual (versus face-to-face) instruction was most highly associated with school-level infection rates and appeared relatively uniform within schools. We find that students who were assigned a higher proportion of instructional days in virtual mode experienced higher rates of attendance, but also negative student achievement growth compared to students who were assigned a higher proportion of instructional days in face-to-face mode. Students belonging to marginalized groups experienced more positive associations with attendance but were also more likely to experience lower student achievement growth when assigned a greater proportion of instructional days in virtual mode. Insights from this study can be used to better understand family preference as well as to target and refine virtual learning in a post-COVID-19 society.
Citation: Jennifer Darling-Aduana, Henry T. Woodyard, Tim Sass, Sarah S. Barry (2022). Learning-Mode Choice, Student Engagement, and Achievement Growth During the COVID-19 Pandemic. CALDER Working Paper No. 260-0122
CALDER Policy Brief No. 27-0921
Citation: Heather Boughton, Jessica de Barros, Dan Goldhaber, Sydney Payne, Nathaniel Schwartz (2021). The Once-in-a-Generation Opportunity: What States and Districts Can Do Now to Learn From American Rescue Plan ESSER Interventions. CALDER Policy Brief No. 27
What does it mean for students to be in a gifted program? While about 7% of students nationally participate in gifted programs, relatively little is known about the experiences of students in these programs or how they vary across districts. Combining administrative and survey data, we describe the structure of gifted programs across nearly 300 school districts in Washington State. Using covariate adjustments and student fixed effects, we find that participation in gifted programs increases access to advanced courses, high-achieving peers, smaller classrooms, and more qualified teachers. These effects are largely concentrated in larger urban and suburban school districts that frequently run large, self-contained gifted programs. Effects of participation are much smaller for small school districts, rural or town school districts, and districts with small gifted programs. While gifted participation changes the educational environment for the average student in the state, the median school district program effect is near zero across the measures of educational environments we consider. This divergence is driven by a pattern of large school districts, high-income school districts, and urban and suburban school districts having programs with significantly larger effects on learning environments. Finally, we find that gifted program effects are larger for some student subgroups, but this is entirely due to district treatment effect heterogeneity, not differential effects on subgroups within districts.
Citation: Benjamin Backes, James Cowan, Dan Goldhaber (2021). What Makes for a "Gifted" Education? Exploring How Participation in Gifted Programs Affects Students' Learning Environments. CALDER Working Paper No. 256-0821
In this paper we use data from two states—Michigan and Washington—on COVID case rates at the county level linked to information on the type of instructional modality offered by local public school districts to assess the relationship between modality and COVID outcomes. We focus primarily on COVID case rates, but also provide estimates for hospitalizations (in Washington only) and deaths. Our preferred district and month fixed effects models exploit within district (over time) variation in instructional modality and account for time-invariant district factors. In both states, we find evidence that instructional modality does lead to increases in COVID spread in communities with moderate to high levels of pre-existing COVID cases, although the causal effect is small in magnitude.
Working Paper No. 247-1220 was originally released in December 2020 and has since been updated to Working Paper No. 247-1220-3, released in July 2021.
Citation: Dan Goldhaber, Scott A. Imberman, Katharine O. Strunk, Bryant Hopkins, Nate Brown, Erica Harbatkin, Tara Kilbride (2021). To What Extent Does In-Person Schooling Contribute to the Spread of COVID-19? Evidence from Michigan and Washington. CALDER Working Paper No. 247-0721-3
CALDER Policy Brief No. 26-0721
Students’ performance on standardized tests is clearly predictive of their later outcomes (Goldhaber & Özek, 2019) but whether the costs of administering tests are justified by the value of the tests for improving students’ outcomes is controversial. This controversy fuels heated debates over federal testing requirements, such as those instituted under No Child Left Behind (NCLB) and continued under the Every Student Succeeds Act (ESSA). All ESSA-required tests were canceled in the 2019-20 school year due to COVID-19, and there was significant political debate about whether they ought to be required in 2020-21. Ultimately the U.S. Department of Education (USDOE) allowed for some testing flexibility in the form of ESSA waivers.
In this policy brief, we outline some of the common ways states might use tests to improve student outcomes and the implications of requested and approved waivers on these uses. In particular, we highlight features of waiver requests that are especially important if tests are to be used by specific actors (e.g., families) to benefit students in specific ways. We conclude with a discussion of how testing policy needs to be designed if statewide tests are going to both be useful and maintain political support.
Citation: Paul Bruno, Dan Goldhaber (2021). Reflections on What Pandemic-Related State Test Waiver Requests Suggest About the Priorities for the Use of Tests. CALDER Policy Brief No. 26
We study the effect of exposure to immigrants on the educational outcomes of US-born students, using a unique dataset combining population-level birth and school records from Florida. This research question is complicated by substantial school selection of US-born students, especially among White and comparatively affluent students, in response to the presence of immigrant students in the school. We propose a new identification strategy to partial out the unobserved non-random selection into schools, and find that the presence of immigrant students has a positive effect on the academic achievement of US-born students, especially for students from disadvantaged backgrounds. Moreover, the presence of immigrants does not affect negatively the performance of affluent US-born students, who typically show a higher academic achievement compared to immigrant students. We provide suggestive evidence on potential channels.
Citation: David Figlio, Paola Giuliano, Riccardo Marchingiglio, Umut Özek, Paola Sapienza (2021). Diversity in Schools: Immigrants and the Educational Performance of U.S. Born Students. CALDER Working Paper No. 250-0321
We evaluate the feasibility of estimating test-score growth for schools and districts with a gap year in test data. Our research design uses a simulated gap year in testing when a true test gap did not occur, which facilitates comparisons of district- and school-level growth estimates with and without a gap year. We find that growth estimates based on the full data and gap year data are generally similar, establishing that useful growth measures can be constructed with a gap year in test data. Our findings apply most directly to testing disruptions that occur in the absence of other disruptions to the school system. They also provide insights about the test stoppage induced by COVID-19, although our work is just a first step toward producing informative school- and district-level growth measures from the pandemic period.
This is an updated version of the paper originally titled "Estimating Test-Score Growth with a Gap Year in the Data", released in January 2021.
This paper has been published in AERA Open and can be viewed here, August 2021.
Citation: Ishtiaque Fazlul, Cory Koedel, Eric Parsons, Cheng Qian (2021). Estimating Test-Score Growth for Schools and Districts with a Gap Year in the Data. CALDER Working Paper No. 248-0121-2
Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students’ future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties.
Citation: Benjamin Backes, James Cowan (2020). Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. CALDER Working Paper No. 241-0820
The Community Eligibility Provision (CEP) is a policy change to the federally-administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data.
This paper was published in Educatinal Evaluation and Policy Analysis in November 2020 and can be found here.
Note: Portions of this paper were previously circulated under the title “Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision.” We have since split the original paper into two parts. This is the first part.
WP 234-0320 was originally released in March 2020. This is the third update, WP 234-0320-3, released in September 2020.
Citation: Cory Koedel, Eric Parsons (2020). The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students. CALDER Working Paper No. 234-0320-3
Using detailed administrative data for public schools, we document racial and ethnic segregation at the classroom level in North Carolina, a state that has experienced a sharp increase in Hispanic enrollment. We decompose classroom-level segregation in counties into within-school and between-school components. We find that the within-school component accounted for a sizable share of total segregation in middle schools and high schools. Recognizing its importance could temper the praise for school assignment policies that reduce racial disparities between schools but allow large disparities within them. More generally, we observe between the two components a complementary relationship, with one component tending to be large when the other one is small. Comparing the degree of segregation for the state’s two largest racial/ethnic minority groups, we find that White/Hispanic segregation was more severe than White/Black segregation, particularly within schools. Finally, we examine enrollment patterns by course and show that school segregation brings with it differences by race and ethnicity in the courses that students take, with White students more likely to be enrolled in advanced classes.
WP 230-0220 was originally released in February 2020. This is the third update, WP 230-0220-3, released in January 2021.
This paper has been published in Race and Social Problems and can be found here, January 2021.
Citation: Charles Clotfelter, Helen Ladd, Calen R. Clifton, Mavzuna Turaeva (2020). School Segregation at the Classroom Level in a Southern ‘New Destination’ State. CALDER Working Paper No. 230-0220-3
Using value-added models, we find that high schools impact students’ self-reported socio-emotional development (SED) by enhancing social well-being and promoting hard work. Conditional on schools’ test score impacts, schools that improve SED, reduce school-based arrests, and increase high-school completion, college-going, and college persistence. Schools that improve social well-being have larger effects on attendance and behavioral infractions in high school, while those that promote hard work have larger effects on GPA. Results suggest that adolescence can be a formative period for socio-emotional growth, high-school impacts on SED can be captured using self-report surveys, and SED can be fostered by schools to improve longer-run outcomes. These findings are robust to tests for plausible forms of selection.
This paper has been published in American Economic Review and can be found here, December 2020.
Citation: Kirabo Jackson, Shanette C. Porter, John Q. Easton, Alyssa Blanchard, Sebastián Kiguel (2020). School Effects on Socio-emotional Development, School-Based Arrests, and Educational Attainment. CALDER Working Paper No. 226-0220
CALDER Policy Brief No. 18-0919
Over the past two decades, charter schools have become the most popular form of school choice, especially in urban school districts. As such, a great deal of empirical research has focused on charter schools. Looking at the literature on the student achievement effects of charter school attendance, the weight of the evidence suggests a moderately positive effect with significant heterogeneity in effectiveness across different types of charter schools and across different states/school districts. For example, “No Excuses” charters such as Knowledge is Power Program (KIPP) charter schools have been shown to outperform other charters and traditional public schools in raising student achievement. Educational attainment effects of charter schools have been more positive, with significant effects on high school graduation, college enrollment, and persistence in college. That said, there is still need for more research on (1) the effects of charter schools on later life outcomes including earnings and risky behavior; (2) whether effective charter providers will remain effective at a larger scale; (3) whether the policies and practices of effective charter schools can be successfully implemented in the traditional public school sector; and (4) whether certain state/school district policies better facilitate the growth of an effective charter school sector.
Citation: Umut Özek, Tim Sass (2019). Charter Schools and Student Outcomes: What Have We Learned Over Two Decades?. CALDER Policy Brief No. 18
Traditionally, teacher salaries have been determined solely by experience and educational attainment. This has led to chronic shortages of teachers in particular subject areas, such as math, science and special education. We study the first long-running statewide program to differentiate teacher pay based on subject area, Georgia’s bonus system for math and science teachers. Using a difference-in-differences strategy, we find the bonuses reduce teacher attrition by 18 to 28 percent. However, we find no evidence the program increases the probability that education majors become secondary math or science teachers upon graduation or alters specific major choices within the education field.
Citation: Carycruz Bueno, Tim Sass (2019). The Effects of Differential Pay on Teacher Recruitment and Retention . CALDER Working Paper No. 219-0519
CALDER Policy Brief No. 14-0319-1
Citation: Carrie Conaway (2019). Maximizing Research Use in the World We Actually Live In: Relationships, Organizations, and Interpretation. CALDER Policy Brief No. 14
Using a regression discontinuity design generated by school-entry cutoffs and school records from an anonymous district in Florida, we identify externalities in human capital production function arising from sibling spillovers. We find positive spillover effects from an older to a younger child in less affluent families and negative spillover effects from a younger to an older child in more affluent families. These results provide empirical evidence that educational policies could create both positive and negative within-family externalities depending on the characteristics of the affected households.
CALDER Working Paper 217-0219 was originally published in February 2019, this is the second version, published September 2021.
This paper has been published in The Journal of Human Resources and can be found here, September 2021.
Citation: Krzysztof Karbownik, Umut Özek (2019). Setting a Good Example? Examining Sibling Spillovers in Educational Achievement Using a Regression Discontinuity Design. CALDER Working Paper No. 217-0219-2
We study the effect of preferences for boys on the performance in mathematics of girls, using evidence from two different data sources. In our first set of results, we identify families with a preference for boys by using fertility stopping rules in a large population of households whose children attend public schools in Florida. Girls growing up in a boy-biased family score on average 3 percentage points lower on math exams when compared to girls raised in other types of families. In our second set of results we find similar strong effects when we study the correlations between girls’ performance in mathematics and maternal gender role attitudes, using evidence from the National Longitudinal Survey of Youth. We conclude that socialization at home can explain a non-trivial part of the observed gender disparities in mathematics performance and document that maternal gender attitudes correlate with those of their children.
This paper has been published in The Journal of Economic Behavior and Organization and can be found here, March 2021.
Citation: Gaia Dossi, David Figlio, Paola Giuliano, Paola Sapienza (2019). Born in the Family: Preferences for Boys and the Gender Gap in Math . CALDER Working Paper No. 216-0219-1
Free and reduced-price meal (FRM) data are used ubiquitously to proxy for student disadvantage in education research and policy applications. The Community Eligibility Provision (CEP)—a recently-implemented policy change to the federally-administered National School Lunch Program—allows schools serving low-income populations to identify all students as FRM-eligible regardless of individual circumstances. We study the CEP’s effect on FRM eligibility as a proxy for student disadvantage, and relatedly, we examine the viability of direct certification (DC) status as an alternative disadvantage measure. Our findings on whether the CEP degrades the informational content of FRM data are mixed. At the individual level there is essentially no effect, but the CEP does meaningfully change the information conveyed by the FRM-eligible share of students in a school. Our comparison of FRM and DC data in the post-CEP era shows that these measures are similarly informative as proxies for disadvantage, despite the CEP-induced information loss in FRM data. Using both measures together can improve the identification of disadvantaged students, but only marginally.
WP 214-0119-1 is no longer available. It has been replaced by two different papers, the first of which is available here: https://caldercenter.org/publications/effect-community-eligibility-provision-ability-free-and-reduced-price-meal-data
Citation: Cory Koedel, Eric Parsons (2019). Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision. CALDER Working Paper No. 214-0119-1
Novice teachers’ professional contexts may have important implications for their effectiveness, development, and retention. However, descriptions of these contexts suffer from data limitations, resulting in unidimensional or vague characterizations. Using 10 years of administrative data from the Los Angeles Unified School District, we describe patterns of new teacher sorting using 27 context measures organized along three distinct dimensions - intensity of instructional responsibilities, homophily, and colleague qualifications – and use school-level survey data to measure a fourth dimension (professional culture). Relative to more experienced teachers, novice teachers have placements that are more challenging along the first three dimensions, and composite measures are differentially predictive of teachers’ outcomes. This suggests that policymakers should consider placements to better retain and develop novice teachers.
This paper has been published in American Educational Research Journal and can be found here, October 2019.
Citation: Paul Bruno, Sarah Rabovsky, Katharine O. Strunk (2019). Taking their First Steps: The Distribution of New Teachers into School and Classroom Contexts and Implications for Teacher Effectiveness and Growth. CALDER Working Paper No. 212-0119-1
We present evidence of a positive relationship between school starting age and children’s cognitive development from age 6 to 18 using a fuzzy regression discontinuity design and large-scale population-level birth and school data from the state of Florida. We estimate effects of being old for grade (being born in September versus August) that are remarkably stable – always around 0.2 SD difference in test scores – across a wide range of heterogeneous groups, based on maternal education, poverty at birth, race/ethnicity, birth weight, gestational age, and school quality. While the September-August difference in kindergarten readiness is dramatically different by subgroup, by the time students take their first exams, the heterogeneity in estimated effects on test scores effectively disappears. We do, however, find significant heterogeneity in other outcome measures such as disability status and middle and high school course selections. We also document substantial variation in compensatory behaviors targeted towards young for grade children. While the more affluent families tend to redshirt their children, young for grade children from less affluent families are more likely to be retained in grades prior to testing. School district practices regarding retention and redshirting are correlated with improved out- comes for the groups less likely to use those remediation approaches (i.e., retention in the case of more-affluent families and redshirting in the case of less-affluent families.) Finally, we find that very few school policies or practices mitigate the test score advantage of September born children.
The world is experiencing the second largest refugee crisis in a century, and one of the major points of contention involves the possible adverse effects of incoming refugees on host communities. We examine the effects of a large refugee influx into Florida public schools following the Haitian earthquake of 2010 using unique matched birth and schooling records. We find precise zero estimated effects of refugees on the educational outcomes of incumbent students in the year of the earthquake or in the two years that follow, regardless of the socioeconomic status, grade level, ethnicity, or birthplace of incumbent students.
Citation: David Figlio, Umut Özek (2017). Unwelcome Guests? The Effects of Refugees on the Educational Outcomes of Incumbent Students. CALDER Working Paper No. 180