You are here
Schools, communities, and social policy
We evaluate the feasibility of estimating test-score growth with a gap year in testing data, informing the scenario when state testing resumes after the 2020 COVID-19 induced test stoppage. Our research design is to simulate a gap year in testing using pre-COVID-19 data—when a true test gap did not occur—which facilitates comparisons of district- and school-level growth measures that are estimated with and without a gap year. We find that growth estimates based on the full data and gap-year data are generally similar, and our results highlight an advantage of using comprehensive growth models with rich controls for student and schooling circumstances. With the caveat that there is looming uncertainty about which students will be tested in public schools when testing resumes, and how they will be tested, our findings establish the potential for estimating useful growth measures with a gap year in testing.
Citation: Ishtiaque Fazlul, Cory Koedel, Eric Parsons, Cheng Qian (2021). Estimating Test-Score Growth with a Gap Year in the Data. CALDER Working Paper No. 248-0121
The decision about how and when to open schools to in-person instruction has been a key question for policymakers throughout the COVID-19 pandemic. The instructional modality of schools has implications not only for the health and safety of students and staff, but also student learning and the degree to which parents can engage in job activities. We consider the role of instructional modality (inperson, hybrid, or remote instruction) in disease spread among the wider community. Using a variety of regression modeling strategies to address unobserved heterogeneity, we find that simple correlations show in-person modalities are correlated with increased COVID cases, but accounting for both pre-existing cases and a richer set of covariates brings estimates close to zero on average. In Ordinary Least Squares (OLS) specifications, in-person modality options are not associated with increased spread of COVID at low levels of pre-existing COVID cases, but cases do increase at moderate to high pre-existing COVID rates. A bounding exercise suggests that the OLS findings for in-person modality are likely to represent an upper bound on the true relationship. These findings are robust to the inclusion of county and district fixed effects in terms of the insignificance of the findings, but the models with fixed effects are also somewhat imprecisely estimated.
Working Paper No. 247-1220 was originally released in December 2020 and has since been updated to Working Paper No. 247-1220-2, released in January 2021.
Citation: Dan Goldhaber, Scott A. Imberman, Katharine Strunk, Bryant Hopkins, Nate Brown, Erica Harbatkin, Tara Kilbride (2020). To What Extent Does In-Person Schooling Contribute to the Spread of COVID-19? Evidence from Michigan and Washington. CALDER Working Paper No. 247-1220-2
Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students’ future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties.
Citation: Benjamin Backes, James Cowan (2020). Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. CALDER Working Paper No. 241-0820
The Community Eligibility Provision (CEP) is a policy change to the federally-administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data.
Note: Portions of this paper were previously circulated under the title “Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision.” We have since split the original paper into two parts. This is the first part.
WP 234-0320 was originally released in March 2020. This is the third update, WP 234-0320-3, which was released in September 2020.
Citation: Cory Koedel, Eric Parsons (2020). The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students. CALDER Working Paper No. 234-0320-3
Using detailed administrative data for public schools, we document racial and ethnic segregation at the classroom level in North Carolina, a state that has experienced a sharp increase in Hispanic enrollment. We decompose classroom-level segregation in counties into within-school and between-school components. We find that the within-school component accounted for a sizable share of total segregation in middle schools and high schools. Recognizing its importance could temper the praise for school assignment policies that reduce racial disparities between schools but allow large disparities within them. More generally, we observe between the two components a complementary relationship, with one component tending to be large when the other one is small. Comparing the degree of segregation for the state’s two largest racial/ethnic minority groups, we find that White/Hispanic segregation was more severe than White/Black segregation, particularly within schools. Finally, we examine enrollment patterns by course and show that school segregation brings with it differences by race and ethnicity in the courses that students take, with White students more likely to be enrolled in advanced classes.
WP 230-0220 was originally released in February 2020. This is the third update, WP 230-0220-3, released in January 2021.
Citation: Charles Clotfelter, Helen Ladd, Calen R. Clifton, Mavzuna R. Turaeva (2020). School Segregation at the Classroom Level in a Southern ‘New Destination’ State. CALDER Working Paper No. 230-0220-3
Using value-added models, we find that high schools impact students’ self-reported socio-emotional development (SED) by enhancing social well-being and promoting hard work. Conditional on schools’ test score impacts, schools that improve SED, reduce school-based arrests, and increase high-school completion, college-going, and college persistence. Schools that improve social well-being have larger effects on attendance and behavioral infractions in high school, while those that promote hard work have larger effects on GPA. Results suggest that adolescence can be a formative period for socio-emotional growth, high-school impacts on SED can be captured using self-report surveys, and SED can be fostered by schools to improve longer-run outcomes. These findings are robust to tests for plausible forms of selection.
Citation: Kirabo Jackson, Shanette C. Porter, John Q. Easton, Alyssa Blanchard , Sebastián Kiguel (2020). School Effects on Socio-emotional Development, School-Based Arrests, and Educational Attainment. CALDER Working Paper No. 226-0220
Traditionally, teacher salaries have been determined solely by experience and educational attainment. This has led to chronic shortages of teachers in particular subject areas, such as math, science and special education. We study the first long-running statewide program to differentiate teacher pay based on subject area, Georgia’s bonus system for math and science teachers. Using a difference-in-differences strategy, we find the bonuses reduce teacher attrition by 18 to 28 percent. However, we find no evidence the program increases the probability that education majors become secondary math or science teachers upon graduation or alters specific major choices within the education field.
Citation: Carycruz Bueno, Tim Sass (2019). The Effects of Differential Pay on Teacher Recruitment and Retention . CALDER Working Paper No. 219-0519
We identify externalities in human capital production function arising from sibling spillovers. Using regression discontinuity design generated by school-entry cutoffs and school records from one district in Florida, we find positive spillover effects from an older to a younger child in less affluent families and negative spillover effects from a younger to an older child in more affluent families. These results are consistent with direct spillovers dominating in economically disadvantaged families and with parental reinforcement in more affluent families.
Citation: Krzysztof Karbownik, Umut Özek (2019). Setting a Good Example? Examining Sibling Spillovers in Educational Achievement Using a Regression Discontinuity Design. CALDER Working Paper No. 217-0219-1
We study the effect of preferences for boys on the performance in mathematics of girls, using evidence from two different data sources. In our first set of results, we identify families with a preference for boys by using fertility stopping rules in a large population of households whose children attend public schools in Florida. Girls growing up in a boy-biased family score on average 3 percentage points lower on math exams when compared to girls raised in other types of families. In our second set of results we find similar strong effects when we study the correlations between girls’ performance in mathematics and maternal gender role attitudes, using evidence from the National Longitudinal Survey of Youth. We conclude that socialization at home can explain a non-trivial part of the observed gender disparities in mathematics performance and document that maternal gender attitudes correlate with those of their children.
Citation: Gaia Dossi, David Figlio, Paola Giuliano, Paola Sapienza (2019). Born in the Family: Preferences for Boys and the Gender Gap in Math . CALDER Working Paper No. 216-0219-1
Free and reduced-price meal (FRM) data are used ubiquitously to proxy for student disadvantage in education research and policy applications. The Community Eligibility Provision (CEP)—a recently-implemented policy change to the federally-administered National School Lunch Program—allows schools serving low-income populations to identify all students as FRM-eligible regardless of individual circumstances. We study the CEP’s effect on FRM eligibility as a proxy for student disadvantage, and relatedly, we examine the viability of direct certification (DC) status as an alternative disadvantage measure. Our findings on whether the CEP degrades the informational content of FRM data are mixed. At the individual level there is essentially no effect, but the CEP does meaningfully change the information conveyed by the FRM-eligible share of students in a school. Our comparison of FRM and DC data in the post-CEP era shows that these measures are similarly informative as proxies for disadvantage, despite the CEP-induced information loss in FRM data. Using both measures together can improve the identification of disadvantaged students, but only marginally.
WP 214-0119-1 is no longer available. It has been replaced by two different papers, the first of which is available here: https://caldercenter.org/publications/effect-community-eligibility-provision-ability-free-and-reduced-price-meal-data
Citation: Cory Koedel, Eric Parsons (2019). Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision. CALDER Working Paper No. 214-0119-1
Novice teachers’ professional contexts may have important implications for their effectiveness, development, and retention. However, descriptions of these contexts suffer from data limitations, resulting in unidimensional or vague characterizations. Using 10 years of administrative data from the Los Angeles Unified School District, we describe patterns of new teacher sorting using 27 context measures organized along three distinct dimensions - intensity of instructional responsibilities, homophily, and colleague qualifications – and use school-level survey data to measure a fourth dimension (professional culture). Relative to more experienced teachers, novice teachers have placements that are more challenging along the first three dimensions, and composite measures are differentially predictive of teachers’ outcomes. This suggests that policymakers should consider placements to better retain and develop novice teachers.
Citation: Paul Bruno, Sarah Rabovsky, Katharine Strunk (2019). Taking their First Steps: The Distribution of New Teachers into School and Classroom Contexts and Implications for Teacher Effectiveness and Growth. CALDER Working Paper No. 212-0119-1
We present evidence of a positive relationship between school starting age and children’s cognitive development from age 6 to 18 using a fuzzy regression discontinuity design and large-scale population-level birth and school data from the state of Florida. We estimate effects of being old for grade (being born in September versus August) that are remarkably stable – always around 0.2 SD difference in test scores – across a wide range of heterogeneous groups, based on maternal education, poverty at birth, race/ethnicity, birth weight, gestational age, and school quality. While the September-August difference in kindergarten readiness is dramatically different by subgroup, by the time students take their first exams, the heterogeneity in estimated effects on test scores effectively disappears. We do, however, find significant heterogeneity in other outcome measures such as disability status and middle and high school course selections. We also document substantial variation in compensatory behaviors targeted towards young for grade children. While the more affluent families tend to redshirt their children, young for grade children from less affluent families are more likely to be retained in grades prior to testing. School district practices regarding retention and redshirting are correlated with improved out- comes for the groups less likely to use those remediation approaches (i.e., retention in the case of more-affluent families and redshirting in the case of less-affluent families.) Finally, we find that very few school policies or practices mitigate the test score advantage of September born children.
The world is experiencing the second largest refugee crisis in a century, and one of the major points of contention involves the possible adverse effects of incoming refugees on host communities. We examine the effects of a large refugee influx into Florida public schools following the Haitian earthquake of 2010 using unique matched birth and schooling records. We find precise zero estimated effects of refugees on the educational outcomes of incumbent students in the year of the earthquake or in the two years that follow, regardless of the socioeconomic status, grade level, ethnicity, or birthplace of incumbent students.
Citation: David Figlio, Umut Özek (2017). Unwelcome Guests? The Effects of Refugees on the Educational Outcomes of Incumbent Students. CALDER Working Paper No. 180
We use remarkable population-level administrative education and birth records from Florida to study the role of Long-Term Orientation on the educational attainment of immigrant students living in the US. Controlling for the quality of schools and individual characteristics, students from countries with long term oriented attitudes perform better than students from cultures that do not emphasize the importance of delayed gratification. These students perform better in third grade reading and math tests, have larger test score gains over time, have fewer absences and disciplinary incidents, are less likely to repeat grades, and are more likely to graduate from high school in four years. Also, they are more likely to enroll in advanced high school courses, especially in scientific subjects. Parents from long term oriented cultures are more likely to secure better educational opportunities for their children. A larger fraction of immigrants speaking the same language in the school amplifies the effect of Long-Term Orientation on educational performance. We validate these results using a sample of immigrant students living in 37 different countries.
We make use of a new data source – matched birth records and longitudinal student records in Florida – to study the degree to which student outcomes differ across successive immigrant generations. Specifically, we investigate whether first, second, and third generation Asian and Hispanic immigrants in Florida perform differently on reading and mathematics tests, and whether they are differentially likely to get into serious trouble in school, to be truant from school, to graduate from high school, or to be ready for college upon high school graduation. We find evidence suggesting that early-arriving first generation immigrants perform better than do second generation immigrants, and second generation immigrants perform better than third generation immigrants. Among first generation immigrants, the earlier the arrival, the better the students tend to perform. These patterns of findings hold for both Asian and Hispanic students, and suggest a general pattern of successively reduced achievement – beyond a transitional period for recent immigrants – in the generations following the generation that immigrated to the United States.
Citation: Umut Özek, David Figlio (2016). Cross-Generational Differences in Educational Outcomes in the Second Great Wave of Immigration. CALDER Working Paper No. 162
U.S. women graduate high school at higher rates than U.S. men, but the female-male educational advantage is larger, and has increased by more, among black and low-SES students than among white and high-SES students. We explore why boys fare worse than girls—both behaviorally and educationally— by exploiting birth certificates matched to health, disciplinary, academic, and high school graduation records for over one million children born in Florida between 1992 and 2002. We account for unobserved family heterogeneity by contrasting the outcomes of opposite-sex siblings linked to birth mothers by administrative records. Relative to their sisters, boys born to low-education and unmarried mothers, raised in low-income neighborhoods, and enrolled at poor-quality public schools have a higher incidence of absences and behavioral problems throughout elementary and middle school, exhibit higher rates of behavioral and cognitive disability, perform worse on standardized tests, and are less likely to graduate from high school. We argue that the family disadvantage gradient in the gender gap is a causal effect of the post-natal environment: family disadvantage has no relationship with the sibling gender gap in neonatal health. Although family disadvantage is strongly correlated with school and neighborhood quality, the SES gradient in the sibling gender gap is almost as large within schools and neighborhoods as between them. A surprising implication of these findings is that, relative to white children, black boys fare worse than their sisters in significant part because black children— both boys and girls—are raised in more disadvantaged family environments.
Citation: David Figlio, Jeffrey Roth, Kryzsztof Karbownik (2016). Family Disadvantage and the Gender Gap in Behavioral and Educational Outcomes. CALDER Working Paper No. 161
Millions of tons of hazardous wastes have been produced in the United States in the last 60 years which have been dispersed into the air, into water, and on and under the ground. Using new population-level data that follows cohorts of children born in the state of Florida between 1994 and 2002, this paper examines the short and long-term effects of prenatal exposure to environmental toxicants on children living within two miles of a Superfund site, toxic waste sites identified by the Environmental Protection Agency as being particularly severe. We compare siblings living within two miles from a Superfund site at birth where at least one sibling was conceived before or during cleanup of the site, and the other(s) was conceived after the site cleanup was completed using a family fixed effects model. Children conceived to mothers living within 2 miles of a Superfund site before it was cleaned are 7.4 percentage points more likely to repeat a grade, have 0.06 of a standard deviation lower test scores, and are 6.6 percentage points more likely to be suspended from school than their siblings who were conceived after the site was cleaned. Children conceived to mothers living within one mile of a Superfund site before it was cleaned are 10 percentage points more likely to be diagnosed with a cognitive disability than their later born siblings as well. These results tend to be larger and are more statistically significant than the estimated effects of proximity to a Superfund site on birth outcomes. This study suggests that the cleanup of severe toxic waste sites has significant positive effects on a variety of long-term cognitive and developmental outcomes for children.
Citation: Claudia Persico, David Figlio, Jeffrey Roth (2016). Inequality Before Birth: The Developmental Consequences of Environmental Toxicants. CALDER Working Paper No. 160
We make use of a new data resource, merged birth and school records for all children born in Florida from 1992 to 2002, to study the effects of birth weight on cognitive development from kindergarten through schooling. Using twin fixed effects models, we find that the effects of birth weight on cognitive development are essentially constant through the school career; that these effects are very similar across a wide range of family backgrounds; and that they are invariant to measures of school quality. We conclude that the effects of poor neonatal health on adult outcomes are therefore set very early.
Citation: David Figlio, Jonathan Guryan, Kryzsztof Karbownik, Jeffrey Roth (2013). The Effects of Poor Neonatal Health on Children’s Cognitive Development. CALDER Working Paper No. 95
This paper explores the relative effects of school and neighborhood characteristics on student achievement in Texas. Previous empirical studies have estimated one of these effects in the absence of controls for the other, leading to potentially misleading results. School variables are more robust and explain a greater degree of the variance in test scores than neighborhood characteristics. Neighborhood level variables, as a group, are statistically significant even in the presence of school variables. The particular pattern of effects varies by the manner in which the school context was controlled, by poverty status, move status, and location in the conditional achievement distribution. But neighborhood always mattered. Even if neighborhood conditions are less robust than school context effects, concern about neighborhood conditions is still justified. Reducing the concentration of poverty and economic segregation may be the easiest way to decrease the "savage inequalities" that exist between schools.
Citation: Paul Jargowsky, Mohamed El Komi (2009). Before or After the Bell?: School Context and Neighborhood Effects on Student Achievement. CALDER Working Paper No. 28