You are here
There has been a resurgence in research that investigates the efficacy of early investments as a means of reducing gaps in academic performance. However, the strongest evidence for these effects comes from experimental evaluations of small, highly enriched programs. We add to this literature by assessing the extent to which a large-scale public program, Texas's targeted pre-Kindergarten (pre-K), affects scores on math and reading achievement tests, the likelihood of being retained in grade, and the probability that a student receives special education services. We find that having participated in Texas's targeted pre-K program is associated with increased scores on the math and reading sections of the Texas Assessment of Academic Skills (TAAS), reductions in the likelihood of being retained in grade, and reductions in the probability of receiving special education services. We also find that participating pre-K increases mathematics scores for students who take the Spanish version of the TAAS tests. These results show that even modest, public pre-K program implemented at scale can have important effects on students’ educational achievement.
Citation: Rodney J. Andrews, Paul Jargowsky, Kristin Kuhne (2012). The Effects of Texas’s Targeted Pre-Kindergarten Program on Academic Performance. CALDER Working Paper No. 84
This paper provides practical guidance for researchers who are designing and analyzing studies that randomize schools — which comprise three levels of clustering (students in classrooms in schools) — to measure intervention effects on student academic outcomes when information on the middle level (classrooms) is missing. This situation arises frequently in practice because many available data sets identify the schools that students attend but not the classrooms in which they are taught. Do studies conducted under these circumstances yield results that are substantially different from what they would have been if this information had been available? The paper first considers this problem in the context of planning a school randomized study based on preexisting two-level information about how academic outcomes for students vary across schools and across students within schools (but not across classrooms in schools). The paper next considers this issue in the context of estimating intervention effects from school-randomized studies. Findings are based on empirical analyses of four multisite data sets using academic outcomes for students within classrooms within schools. The results indicate that in almost all situations one will obtain nearly identical results whether or not the classroom or middle level is omitted when designing or analyzing studies.
Citation: Pei Zhu, Robin Jacob, Howard Bloom, Zeyu Xu (2011). Designing and Analyzing Studies that Randomize Schools To Estimate Intervention Effects on Student Academic Outcomes Without Classroom-Level Information. CALDER Working Paper No. 61
Teacher experience is a cornerstone of traditional single–salary schedules; it drives teacher transfer policies that prioritize seniority; and it is commonly considered a major source of inequity across schools and, therefore, a target for redistribution.The underlying assumption is that experience promotes effectiveness. But is this really the case? Do students attain higher levels of achievement when taught by more experienced teachers? Recent evidence from CALDER studies provides new insight into the effects of teacher experience.
Citation: Jennifer King Rice (2010). The Impact of Teacher Experience: Examining the Evidence and Policy Implications. CALDER Working Paper No.
Does differential access to computer technology at home compound the educational disparities between the rich and the poor? Would a program of government provision of computers to early secondary school students reduce these disparities? This study covers years 2000 to 2005, a period when home computers and high-speed Internet access expanded dramatically. Using administrative data on North Carolina public school students to corroborate earlier surveys that document broad racial and socioeconomic gaps in home computer access and use, the authors compared the children's reading and math scores before and after they acquired a home computer, and compared these scores to those of peers who had a home computer by fifth grade and to test scores of students who never acquired a home computer. The introduction of home computer technology is associated with modest but statistically significant and persistent negative impacts on student math and reading test scores. The authors also conclude that home computers are put to more productive use in households where parental monitoring is more effective. Further evidence suggests that providing universal access to home computers and high-speed internet access would broaden, rather than narrow, math and reading achievement gaps.
Citation: Jacob Vigdor, Helen Ladd (2010). Scaling the Digital Divide: Home Computer Technology and Student Achievement. CALDER Working Paper No. 48
The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms.
Citation: Zeyu Xu, Austin Nichols (2010). New Estimates of Design Parameters for Clustered Randomization Studies: Findings from North Carolina and Florida. CALDER Working Paper No. 43
This paper describes the school mobility rates for elementary and middle school students in North Carolina and attempts to estimate the effect of school mobility on the performance of different groups of students using student fixed effects models. School mobility is defined as changing schools at times that are non-promotional (e.g., moving from middle to high school). We used detailed administrative data on North Carolina students and schools from 1996 to 2005 and followed four cohorts of 3rd graders for six years each. School mobility rates were highest for minority and disadvantaged students. School mobility rates for Hispanic students declined across successive cohorts, but increased for Black students. Findings on effects were most pronounced in math. School mobility hurt the math performance of Black and Hispanic students, but not the math performance of white students. School mobility improved the reading performance of white and more advantaged students, but had no effect on the reading performance of minority students. "Strategic" school moves (cross-district) benefitted or had no effect on student performance, but "reactive" moves (within district) hurt all groups of students. White and Hispanic students were more likely to move to a higher quality school while Blacks were more likely to move to a lower quality school. The negative effects of school mobility increased with the number of school moves.
Citation: Zeyu Xu, Jane Hannaway, Stephanie D'Souza (2009). Student Transience in North Carolina:The Effect of School Mobility on Student Outcomes Using Longitudinal Data. CALDER Working Paper No. 22
This research brief estimates the overall extent of test measurement error and how this varies across students using New York City student- level longitudinal data across grades 3-8 from 1999- 2007. Results reinforce the importance of accounting for measurement error, as it meaningfully increases effect size estimates associated with teacher attributes. There are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important in the formulation and implementation of personnel policies. Also, effect sizes as traditionally measured have led analysts to understate the magnitudes of effects because the standard deviation of observed scores overstates the dispersion of true achievement in the student population.
Citation: Donald Boyd, Pamela Grossman, Hamilton Lankford, Susanna Loeb, James Wyckoff (2008). Overview of Measuring Effect Sizes: The Effect of Measurement Error. CALDER Working Paper No.
This research brief describes the legal and operational structure of the Texas longitudinal data system related to recent changes in the Family Educational Rights and Privacy Act of 1974 (FERPA)—which establishes the rights of parents to access their children's educational records and protects the confidentiality of student information—that more closely align law and practice. The U.S. Department of Education's FERPA Final Regulations Amendments took effect January 8, 2009.
Citation: Daniel M. O'Brien (2008). The Texas FERPA Story. CALDER Working Paper No.
This brief calculates graduation rates for the state of Florida using longitudinal data. We describe our measurement strategies and compare them with the state’s official measurement procedures. We calculate the diploma and GED attainment rates of six separate cohorts of Florida 9th graders who began high school between 1995/96 and 2000/01. We then present rates of both diploma receipt and GED receipt at four years and in later years. The results show an increasing trend in graduation rates in the state over the period studied and a substantial bump at five years, with growth flattening out after that time.
Using a unique longitudinal dataset covering all Florida public school students in grades 3–10 over a five-year period, we analyze the impact of classroom peers on individual student performance. Unlike many previous data sets used to study peer effects in education, our data allow us to identify each member of a given student's classroom peer group in elementary, middle and high school as well as the classroom teacher responsible for instruction. As a result, we can control for individual student fixed effects simultaneously with individual teacher fixed effects, thereby alleviating biases due to endogenous assignment of both peers and teachers, including some dynamic aspects of such assignments. We find some sizable, significant peer effects within nonlinear models, but not with linear specifications. We find peer effects depend on a student's own ability and on the ability of the peers under consideration. Peer effects tend to be smaller when teacher fixed effects are included, a result that suggests co-movement of peer and teacher quality within a student over time. We also find that peer effects tend to be stronger at the classroom level than the grade level.
This paper uses administrative data for the public K-12 schools of North Carolina to measure racial segregation in the public schools of North Carolina. Using data for the 2005/06 school year, the authors update previous calculations that measure segregation in terms of unevenness in racial enrollment patterns both between schools and within schools. They find that classroom segregation generally increased between 2000/01 and 2005/06, continuing, albeit at a slightly slower rate, the trend observed over the preceding six years. Segregation increased sharply in Charlotte-Mecklenburg, which introduced a new choice plan in 2002. Over the same period, racial and economic disparities in teacher quality widened in that district.
Citation: Charles Clotfelter, Helen Ladd, Jacob Vigdor (2008). School Segregation under Color-Blind Jurisprudence: The Case of North Carolina. CALDER Working Paper No. 16