You are here
We evaluate the feasibility of estimating test-score growth with a gap year in testing data, informing the scenario when state testing resumes after the 2020 COVID-19 induced test stoppage. Our research design is to simulate a gap year in testing using pre-COVID-19 data—when a true test gap did not occur—which facilitates comparisons of district- and school-level growth measures that are estimated with and without a gap year. We find that growth estimates based on the full data and gap-year data are generally similar, and our results highlight an advantage of using comprehensive growth models with rich controls for student and schooling circumstances. With the caveat that there is looming uncertainty about which students will be tested in public schools when testing resumes, and how they will be tested, our findings establish the potential for estimating useful growth measures with a gap year in testing.
Citation: Ishtiaque Fazlul, Cory Koedel, Eric Parsons, Cheng Qian (2021). Estimating Test-Score Growth with a Gap Year in the Data. CALDER Working Paper No. 248-0121
Prior work has documented a substantial penalty associated with taking the Partnership for Assessment of Readiness for College and Careers (PARCC) online relative to on paper (Backes & Cowan, 2019). However, this penalty does not necessarily make online tests less useful. For example, it could be the case that computer literacy skills are correlated with students’ future ability to navigate high school coursework, and thus more predictive of later outcomes. Using a statewide implementation of PARCC in Massachusetts, we test the relative predictive validity of online and paper tests. We are unable to detect a difference between the two and in most cases can rule out even modest differences. Finally, we estimate mode effects for the new Massachusetts statewide assessment. In contrast to the first years of PARCC implementation, we find very small mode effects, showing that it is possible to implement online assessments at scale without large online penalties.
Citation: Benjamin Backes, James Cowan (2020). Is Online a Better Baseline? Comparing the Predictive Validity of Computer- and Paper-Based Tests. CALDER Working Paper No. 241-0820
The Community Eligibility Provision (CEP) is a policy change to the federally-administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data.
Note: Portions of this paper were previously circulated under the title “Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision.” We have since split the original paper into two parts. This is the first part.
WP 234-0320 was originally released in March 2020. This is the third update, WP 234-0320-3, which was released in September 2020.
Citation: Cory Koedel, Eric Parsons (2020). The Effect of the Community Eligibility Provision on the Ability of Free and Reduced-Price Meal Data to Identify Disadvantaged Students. CALDER Working Paper No. 234-0320-3
This study examines the effects of internal migration driven by severe natural disasters on host communities, and the mechanisms behind these effects, using the large influx of migrants into Florida public schools after Hurricane Maria. I find adverse effects of the influx in the first year on existing student test scores, disciplinary problems, and student mobility among high-performing students in middle and high school that also persist in the second year. I also find evidence that compensatory resource allocation within schools is an important factor driving the adverse effects of large, unexpected migrant flows on incumbent students in the short-run.
WP 233-0320 was originally released in March 2020. This updated version, WP 233-0320-2, was released in January 2021.
Citation: Umut Özek (2020). Examining the Educational Spillover Effects of Severe Natural Disasters: The Case of Hurricane Maria . CALDER Working Paper No. 233-0320-2
This study adds to the currently limited evidence base on the efficacy of interventions targeting non-college-ready high school students by examining the impact of Kentucky’s Targeted Interventions (TI) program. We focus on interventions that students received under TI in the senior year of high school based on their 11th grade ACT test scores. Using difference-in-regression discontinuity and difference-in-difference designs with seven cohorts of 11th grade students, we find that, for an average per-student cost of about $600, TI significantly reduces the likelihood that students enroll in remedial course in both 2- and 4-year postsecondary institutions by 5–10 percentage points in math and 3–4 percentage points in English. These effects are similar among students who are eligible for free-or reduced-price lunch, Black and Hispanic students, students with remediation needs in multiple subjects, and students in lower-performing schools. Evidence also shows that TI increases the likelihood that students enroll in and pass college math before the end of the first year by four percentage points in 4-year universities. However, little evidence exists for TI affecting credit accumulation or persistence.
Citation: Zeyu Xu, Benjamin Backes, Amanda Oliveira, Dan Goldhaber (2020). Targeted Interventions in High School: Preparing Students for College. CALDER Working Paper No. 232-0220
Using detailed administrative data for public schools, we document racial and ethnic segregation at the classroom level in North Carolina, a state that has experienced a sharp increase in Hispanic enrollment. We decompose classroom-level segregation in counties into within-school and between-school components. We find that the within-school component accounted for a sizable share of total segregation in middle schools and high schools. Recognizing its importance could temper the praise for school assignment policies that reduce racial disparities between schools but allow large disparities within them. More generally, we observe between the two components a complementary relationship, with one component tending to be large when the other one is small. Comparing the degree of segregation for the state’s two largest racial/ethnic minority groups, we find that White/Hispanic segregation was more severe than White/Black segregation, particularly within schools. Finally, we examine enrollment patterns by course and show that school segregation brings with it differences by race and ethnicity in the courses that students take, with White students more likely to be enrolled in advanced classes.
WP 230-0220 was originally released in February 2020. This is the third update, WP 230-0220-3, released in January 2021.
Citation: Charles Clotfelter, Helen Ladd, Calen R. Clifton, Mavzuna R. Turaeva (2020). School Segregation at the Classroom Level in a Southern ‘New Destination’ State. CALDER Working Paper No. 230-0220-3
We use administrative microdata from six states to study academic mobility in K-12 education, by which we mean the extent to which students’ ranks in the distribution of academic performance change during their schooling careers. We find that there is substantial heterogeneity across districts in the academic mobility of students. The heterogeneity across districts is largest in terms of absolute mobility—i.e., in some districts, students throughout the distribution, including initially low performers, gain on other students in the statewide distribution, and vice-versa—whereas there is less cross-district variation in relative mobility. The most prominent correlates of high-mobility districts include value-added to achievement and the socioeconomic status of the student population.
Citation: Wes Austin, David Figlio, Dan Goldhaber, Eric Hanushek, Tara Kilbride, Cory Koedel, Jaeseok Sean Lee, Jin Lou, Umut Özek, Eric Parsons, Steven Rivkin, Tim Sass, Katharine Strunk (2020). Where are Initially Low-performing Students the Most Likely to Succeed? A Multi-state Analysis of Academic Mobility (Preliminary Draft). CALDER Working Paper No. 227-0220
Using value-added models, we find that high schools impact students’ self-reported socio-emotional development (SED) by enhancing social well-being and promoting hard work. Conditional on schools’ test score impacts, schools that improve SED, reduce school-based arrests, and increase high-school completion, college-going, and college persistence. Schools that improve social well-being have larger effects on attendance and behavioral infractions in high school, while those that promote hard work have larger effects on GPA. Results suggest that adolescence can be a formative period for socio-emotional growth, high-school impacts on SED can be captured using self-report surveys, and SED can be fostered by schools to improve longer-run outcomes. These findings are robust to tests for plausible forms of selection.
Citation: Kirabo Jackson, Shanette C. Porter, John Q. Easton, Alyssa Blanchard , Sebastián Kiguel (2020). School Effects on Socio-emotional Development, School-Based Arrests, and Educational Attainment. CALDER Working Paper No. 226-0220
We study the effects of exposure to non-resident students on the outcomes of undergraduate in-state students during a period of high non-resident enrollment growth at the University of Missouri-Columbia. Our models leverage within-major, cross-time variation in non-resident exposure for identification. We find no evidence that increased exposure to domestic non-residents affects in-state student outcomes and our null results are precisely estimated. We find evidence of modest negative impacts on in-state students when their exposure to foreign students increases using our preferred specification. However, the identifying variation in exposure to foreign students in our data is limited and this result is not robust in all of our models.
Citation: Diyi Li, Cheng Qian , Cory Koedel (2020). Non-Resident Postsecondary Enrollment Growth and the Outcomes of In-State Students. CALDER Working Paper No. 225-0120
Indiana, Oklahoma, and Washington have programs designed to address college enrollment and completion gaps by offering a promise of state-based college financial aid to low-income middle school students in exchange for making a pledge to do well in high school, be a good citizen, not be convicted of a felony, and apply for financial aid to college. Using a triple-difference specification, we find that Washington’s College Bound Scholarship shifted enrollment from out-of-state to in-state colleges at which the scholarship could be used. While we find suggestive evidence that the program increased the likelihood of attending a postsecondary institution and attaining a bachelor’s degree within five years of high school, we discuss why the program might be more successful if it did not require students to sign a pledge.
Citation: Mark C. Long, Dan Goldhaber, Trevor Gratz (2019). Washington’s College Bound Scholarship Program and its Effect on College Entry, Persistence, and Completion. CALDER Working Paper No. 221-0919
We leverage nationally representative data and statewide data from Washington to investigate trends in occupational career and technical education (CTE) participation for students with and without disabilities. Consistent with prior work, we document declines in occupational CTE participation since the early 2000s, and provide the first empirical evidence that students with disabilities disproportionately contributed to this decline. But we also show that occupational CTE participation has stabilized for all students in the past decade in Washington, and that participation by students with disabilities in applied science, technology, engineering, mathematics, and medicine (applied STEMM) CTE courses has increased since the early 2000s. These trends are encouraging given prior evidence linking applied STEMM-CTE participation to better long-term outcomes for students with disabilities.
Citation: Roddy Theobald, Jay Plasman, Michael Gottfried, Trevor Gratz, Kristian Holden, Dan Goldhaber (2019). Sometimes Less, Sometimes More: Trends in Career and Technical Education Participation for Students With Disabilities. CALDER Working Paper No. 220-0819
In this paper we estimate the impacts of the “pathways” chosen by community college students—in terms of desired credentials and fields of study, as well as other choices and outcomes along the paths—on the attainment of credentials with labor market value. We focus on the extent to which there are recorded changes in students’ choices over time, whether students make choices informed by their chances of success and by labor market value of credentials, and the impacts of choices on outcomes. We find that several characteristics of chosen pathways, such as field of study and desired credential as well as early “momentum,” affect outcomes. Student choices of pathways are not always driven by information about later chances of success, in terms of probabilities of completing programs and attaining strong earnings. Students also change pathways quite frequently, making it harder to accumulate the credits needed in their fields. Attainment of credentials with greater market value could thus likely be improved by appropriate guidance and supports for students along the way, and perhaps by broader institutional changes as well.
This paper was published in Community College Review in April 2021 and can be found here.
Citation: Harry Holzer, Zeyu Xu (2019). Community College Pathways for Disadvantaged Students . CALDER Working Paper No. 218-0519
We study the effect of preferences for boys on the performance in mathematics of girls, using evidence from two different data sources. In our first set of results, we identify families with a preference for boys by using fertility stopping rules in a large population of households whose children attend public schools in Florida. Girls growing up in a boy-biased family score on average 3 percentage points lower on math exams when compared to girls raised in other types of families. In our second set of results we find similar strong effects when we study the correlations between girls’ performance in mathematics and maternal gender role attitudes, using evidence from the National Longitudinal Survey of Youth. We conclude that socialization at home can explain a non-trivial part of the observed gender disparities in mathematics performance and document that maternal gender attitudes correlate with those of their children.
Citation: Gaia Dossi, David Figlio, Paola Giuliano, Paola Sapienza (2019). Born in the Family: Preferences for Boys and the Gender Gap in Math . CALDER Working Paper No. 216-0219-1
Free and reduced-price meal (FRM) data are used ubiquitously to proxy for student disadvantage in education research and policy applications. The Community Eligibility Provision (CEP)—a recently-implemented policy change to the federally-administered National School Lunch Program—allows schools serving low-income populations to identify all students as FRM-eligible regardless of individual circumstances. We study the CEP’s effect on FRM eligibility as a proxy for student disadvantage, and relatedly, we examine the viability of direct certification (DC) status as an alternative disadvantage measure. Our findings on whether the CEP degrades the informational content of FRM data are mixed. At the individual level there is essentially no effect, but the CEP does meaningfully change the information conveyed by the FRM-eligible share of students in a school. Our comparison of FRM and DC data in the post-CEP era shows that these measures are similarly informative as proxies for disadvantage, despite the CEP-induced information loss in FRM data. Using both measures together can improve the identification of disadvantaged students, but only marginally.
WP 214-0119-1 is no longer available. It has been replaced by two different papers, the first of which is available here: https://caldercenter.org/publications/effect-community-eligibility-provision-ability-free-and-reduced-price-meal-data
Citation: Cory Koedel, Eric Parsons (2019). Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision. CALDER Working Paper No. 214-0119-1
We study a teacher incentive policy in Washington State that awards a financial bonus to National Board certified teachers in high poverty schools. Using a regression discontinuity design, we find that the bonus policy increased the proportion of certified teachers in bonus-eligible schools by improving hiring, increasing certification rates of incumbent teachers, and reducing turnover. Depending on the method, we estimate that the proportion of NBCTs in treated schools increased by about four to eight percentage points over the first five years of eligibility. However, the improvement in certification rates corresponds to a change of about 0.2 to 0.3 percent of a standard deviation in teacher quality per year and we do not find evidence that the bonus resulted in detectible effects on student test achievement.
Citation: James Cowan, Dan Goldhaber (2018). Do Bonuses Affect Teacher Staffing and Student Achievement in High Poverty Schools? Evidence from an Incentive for National Board Certified Teachers in Washington State. CALDER Working Paper No. 194-0618-1
We study the effects of access to high school math and science courses on postsecondary STEM enrollment and degree attainment using administrative microdata from Missouri. Our data panel includes over 140,000 students from 14 cohorts entering the 4-year public university system. The effects of high school course access are identified by exploiting plausibly exogenous variation in course offerings within high schools over time. We find that differential access to high school courses does not affect postsecondary STEM enrollment or degree attainment. Our null results are estimated precisely enough to rule out moderate impacts.
This paper was revised February 2019. It was originally released in February 2018.
Citation: Rajeev Darolia, Cory Koedel, Joyce B. Main, Felix Ndashimye, Junpeng Yan (2018). High School Course Access and Postsecondary STEM Enrollment and Attainment. CALDER Working Paper No. 186
This study examines the community-wide effects of two statewide early childhood policy initiatives in North Carolina. One initiative provides funding to improve the quality of child care services at the county level for all children between the ages of 0 to 5, and the other provides funding for preschool slots for disadvantaged four-year-olds. Differences across counties in the timing of the rollout and in the magnitude of the state financial investments per child provide the variation in programs needed to estimate their effects on schooling outcomes in third grade. We find robust positive effects of each program on third-grade test scores in both reading and math. These effects can best be explained by a combination of direct benefits for participants and spillover benefits for others. Our preferred models suggest that the combined average effects on test scores of investments in both programs at 2009 funding levels are equivalent to two to four months of instruction in grade 3.
Citation: Helen Ladd, Clara G. Muschkin, Kenneth A. Dodge (2015). From Birth to School: Early Childhood Initiatives and Third-Grade Outcomes in North Carolina. CALDER Working Paper No. 134
In this paper, we present a closer look at the student achievement trends in the District of Columbia between 2006-07 and 2012-13. We have three main conclusions. First, we find that overall, math scores in the District have improved. The improvements in reading scores during this time frame, however, were primarily limited to the first year after the PERAA implementation. While almost all student subgroups have experienced test score gains in math, these improvements were higher among the more affluent black and Hispanic students. Second, we find that these observed trends in math scores persist even after controlling for the cross-cohort differences in observed student characteristics. In particular, the estimates indicate that less than 10 percent of the year-to-year improvements in test scores can be attributed to the changing student composition in the District over this time frame. Finally, we show that existing students have also experienced gains in math even though the students who are new to the District’s public school system score at higher levels on standardized tests when compared to existing students.
Citation: Umut Özek (2014). A Closer Look at the Student Achievement Trends in the District of Columbia between 2006-07 and 2012-13. CALDER Working Paper No. 119
In this paper we will briefly review recent trends in employment outcomes for disadvantaged youth, focusing specifically on those who have become "disconnected" from school and the labor market, and why these trends have occurred. We then review a range of policy prescriptions that might improve those outcomes. These policies include: 1) Efforts to enhance education and employment outcomes, both among in-school youth who are at risk of dropping out and becoming disconnected as well as out-of-school youth who have already done so; 2) Policies to increase earnings and incent more labor force participation among youth, such as expanding the eligibility of childless adults (and especially non-custodial parents) for the Earned Income Tax Credit (EITC); and 3) Specific policies to reduce barriers to employment faced by ex-offenders and non-custodial parents (NCPs). We also consider policies that target the demand side of the labor market, in efforts to spur the willingness of employers to hire these young people and perhaps to improve the quality of jobs available to them.
Citation: Peter B. Edelman, Harry Holzer (2013). Connecting the Disconnected: Improving Education and Employment Outcomes Among Disadvantaged Youth. CALDER Working Paper No. 96
Current federal education policies promote learning in Science, Technology, Engineering, and Math (STEM) and the participation of minority students in these fields. Using longitudinal data on students in Florida and North Carolina, value-added estimates in math and science are generated to categorize schools into performance levels and identify differences in school STEM measures by performance levels. Several STEM-relevant variables show a significant association with effectiveness in math and science, including STEM teacher turnover, calculus and early algebra participation, and math and science instructional indices created from survey items in the data. Surprisingly, a negative association between students’ STEM course participation and success in STEM is consistently documented across both states, in addition to low participation of underrepresented minority students in successful schools in STEM.
Citation: Michael Hansen (2013). Characteristics of Schools Successful in STEM: Evidence from Two States’ Longitudinal Data. CALDER Working Paper No. 97