Research Areas

Testing students and using test information to hold schools and, in some cases, teachers accountable for student achievement has arguably been the primary national strategy for school improvement over the past decade and a half. Tests are also intended to be used as a diagnostic tool to identify individual student needs, so that students can be set on a trajectory for long-term academic success. We use panel data from three states – North Carolina, Massachusetts and Washington State – to investigate how accurate early measures of achievement are in predicting later high school outcomes. We contribute to the literature in three distinct ways. First, the long panels we employ allow us to quantify the accuracy of models predicting how early (3rd and 4th grade) measures of student background and achievement predict several later schooling outcomes: 8th grade test achievement, high school course-taking, and high school graduation. Second, we test the extent to which predictions based on distinct segments of student data (e.g. grades 3 to 8, then 8 to 12) sacrifice forecast accuracy; this is of particular policy relevance for states or localities that do not yet have long administrative data panels. Finally, we test the degree to which the use of parameter estimates from models predicting schooling outcomes derived from one state diminish the accuracy of predicting outcomes in other states.

Citation: Dan Goldhaber, Malcolm Wolff, Timothy Daly (2020). Assessing the Accuracy of Elementary School Test Scores as Predictors of Students’ High School Outcomes. CALDER Working Paper No. 235-0520

We describe the postsecondary transitions of students taking CTE courses in high school using administrative data on one cohort of high school graduates in Washington State. Conditional on observable characteristics, CTE concentrators—high school graduates who complete at least four CTE credits—are about 4 percentage points less likely to enroll in college than other high school graduates. However, CTE students are significantly more likely to enroll in and complete vocational programs, especially in certificate programs in applied STEM and public safety fields. Among students not enrolled in college, CTE students are also more likely to obtain full-time employment—and to work more intensively—within the first three years following high school graduation. Although the improvements in employment outcomes do not offset reductions in college enrollment, the higher completion rates of vocational credentials among CTE concentrators indicate some important positive outcomes for this population.

Citation: James Cowan, Dan Goldhaber, Harry Holzer, Natsumi Naito, Zeyu Xu (2019). Career and Technical Education in High School and Postsecondary Pathways in Washington State. CALDER Working Paper No. 224-1119

Citation: Rajeev Darolia, Cory Koedel (2018). High Schools and Students’ Initial Colleges and Majors. CALDER Working Paper No. 165

Indiana, Oklahoma, and Washington each have programs designed to address college enrollment gaps by offering a promise of state-based college financial aid to low-income middle school students in exchange for making a pledge to do well in high school, be a good citizen, not be convicted of a felony, and apply for financial aid to college. Using a triple-difference specification, we estimate the effects of Washington’s College Bound Scholarship program on students’ high school grades, high school graduation, juvenile detention and rehabilitation, and incarceration in state prison during high school or early adulthood. We find insignificant and substantively small or negative effects on these outcomes. These results call into question the rationale for such early commitment programs.

WP 178 was revised in February 2019. It was originally released in June 2017.”

Citation: Dan Goldhaber, Mark C. Long, Trevor Gratz, Jordan Rooklyn (2017). Pledging to Do "Good”: An Early Commitment Pledge Program, College Scholarships, and High School Outcomes in Washington State. CALDER Working Paper No. 178

The purpose of this paper is to assess the effects of this increase in the mandated minimum number of math courses. This assessment entails two separate questions. One is whether the policy affected actual course-taking among high school students. In exploring this question, we are attentive to the likelihood that the new standard might have a bigger effect on some groups of students than on others. Another question is whether any such changes in high school course-taking, together with the threat of being denied admission, affected college enrollment patterns or students’ choices or performance once enrolled.

Our findings fall into three groups. First, the evidence is consistent with the expectation that the increased requirements would influence the number of high school math courses taken by at least some students. Throughout our analysis we characterize students by their math aptitude as measured by their performance on the required 8^th grade math end of grade test, with performance divided into deciles from low to high.  Many students, particularly those at the higher deciles, were already taking four math courses by the time the minimum number was increased, so the new requirement presumably had no direct effect on them. But in eight of the 10 deciles we observed greater-than-expected increases in the share of students who, using the proxy we had (whether a student had taken Algebra II by 11^th grade), were in a position to meet the new four-course standard. We cannot prove that these increases were due to the policy, but it is reasonable to think that at least most of them were.

Second are findings related to whether the increase in math courses affected whether students enrolled in one of the state’s public university campuses and, if so, where. Because the increases in math courses were greatest for students with 8^th grade math scores in the middle deciles, one might have expected that the branch campuses whose students traditionally come from those deciles would have experienced the biggest increases in enrollment due to the changes in math course taking in high school. Surprisingly, we did not find that. Instead, we find increases in predicted enrollment due to changes in math course taking across all campuses, distributed differently across math achievement deciles. Each branch experienced increases in predicted enrollment, but those increases tended to be for students in the deciles that were already most common at those branches. For the branches that have traditionally drawn from deciles below the median, the newly stimulated enrollments came from those deciles. For the two branches with the highest shares of students from the top deciles before the policy change, the new policy stimulated new enrollment, and it was mainly in those same top deciles. Despite the general tendency before the change for top-decile students to have taken four math courses, many top decile students apparently had not been doing so, especially in school districts that had not pushed such students to do so in the past. Once the policy change was enacted, such districts beefed up their math pathways, causing more top students to take more math. Conceivably, the new requirement caused these top students to consider attending the leading research universities at Chapel Hill or NC State instead of one of the branches closer to their homes.

The third set of findings relate to whether the minimum course requirement affected the behavior of students once they enrolled in one of the branches. Here the results are less broad-based than for the other analyses.  We find some evidence that the policy change increased the likelihood that high decile students would major in a STEM field, but reduced the likelihood of low decile students of doing so.  Further, we find that the program raised the GPA of students in deciles 8 and 9, but had at most limited effects on four-year graduation rates. 

Citation: Charles Clotfelter, Helen Ladd (2016). Raising the Bar for College Admission: North Carolina’s Increase in Minimum Math Course Requirements. CALDER Working Paper No. 163

Students are typically given a large amount of freedom to choose the level of “curricular dispersion:” the tight focus or lack thereof in the courses they elect to take while in college. There is little evidence about what predicts students’ curricular dispersion, whether it affects later college or labor force outcomes, or, in fact, how to measure curricular dispersion. In this paper we develop a measure of curricular dispersion and use data from Washington State to explore its predictors and associated outcomes. We find that prior dispersion predicts future dispersion but not subsequent changes in college major. We report mixed findings on the associations between curricular dispersion and overall college GPA, the probability of graduation, and early career wages.

Citation: Dan Goldhaber, James Cowan, Mark C. Long, Nick Huntington-Klein (2015). College Curricular Dispersion: More Well Rounded or Less Well Trained?. CALDER Working Paper No. 130

I investigate the determinants of high school completion and college attendance, the likelihood of taking science, technology, engineering or math (STEM) courses in the first year of college and the probability of earning a degree in a STEM field. The focus is on women and minorities, who tend to be underrepresented in STEM fields. Tracking four cohorts of students throughout Florida, I find that large differences in math achievement across racial lines exist as early as elementary school and persist through high school. These achievement differences lead to higher drop-out rates in high school and a reduced probability of attending college for black students. However, conditional on immediately attending a four-year college after high school, black and Hispanic students are more likely than whites to take STEM courses during their first year in college. Increased exposure to Hispanic math and science teachers in middle and high school tends to increase the likelihood that Hispanic students take STEM courses during their first year in college, though pairing black students and black math/science teachers does not have the same positive effect. For all students, having high school math and science teachers with a degree in biology, chemistry or math (as opposed to education) is associated with a higher likelihood of taking STEM courses as college freshmen. When pre-college differences in income and math achievement are taken into account, black and Hispanic students are at least as likely as white students to successfully complete a STEM major. Racial/ethnic pairing of students and college instructors in first-year STEM courses does not increase the likelihood of majoring in a STEM field. In contrast to underrepresented minorities, women perform nearly as well as men on math achievement tests through high school and are more likely to finish high school and attend college than males. Among college students, however, women are less likely than men to take courses in the physical sciences in their first year and are less likely to earn a degree in physics or engineering, even after adjusting for pre-college test scores. Gender matching of students and math/science teachers in middle and high school tends to increase the likelihood that female college freshman will take at least one STEM course, However, conditional on first-year coursework, neither gender matching at the secondary or college levels appears to have any effect on the likelihood of completing a major in a STEM field.

Citation: Tim Sass (2015). Understanding the STEM Pipeline. CALDER Working Paper No. 125

This paper reports results from a resume-based field experiment designed to examine employer preferences for job applicants who attended for-profit colleges. For-profit colleges have seen sharp increases in enrollment in recent years despite alternatives such as public community colleges being much cheaper. We sent almost 9,000 fictitious resumes of young job applicants who recently completed their schooling to online job postings in six occupational categories and tracked employer callback rates. We find no evidence that employers prefer applicants with resumes listing a for-profit college relative to those whose resumes list either a community college or no college at all. 

Citation: Rajeev Darolia, Cory Koedel, Paco Martorell, Katie Wilson, Francisco Perez-Arce (2014). Do Employers Prefer Workers Who Attend For-Profit Colleges? Evidence from a Field Experiment. CALDER Working Paper No. 116

We evaluate whether there is a causal connection between changes in wages by occupation and subsequent changes in the number of college majors completed in associated fields. Using aggregate national data and individual-level data from Washington State, we find statistically significant, although modest, relationships between wages and majors. College majors are most strongly related to wages observed three years earlier, when students were college freshmen. Majors with a tight connection to particular occupations show a stronger response to wages. The overall modest relationship suggests that policies which inform students about labor market outcomes are unlikely to greatly change student behavior.

Citation: Mark Long, Dan Goldhaber, Nick Huntington-Klein (2014). Do Students’ College Major Choice Respond to Changes in Wages?. CALDER Working Paper No. 107