“Evidence-based” policymaking is the practice of using rigorous research to make policy decisions. Often, however, the research these policy decisions are based is not actually rigorous and/or can be misused. Though the practice is increasing in popularity, there is little evidence on whether these evidence-based policies actually lead to the intended outcomes once implemented.
LBJ School Associate Professor and CHASP faculty associate Dr. Paul von Hippel was recently awarded a grant from the William T. Grant Foundation to increase understanding of how research evidence is “acquired, understood, and used” in making policy decisions, specifically to benefit youth.
von Hippel will study if a successful randomized controlled trial (RCT) leads to successful policy. He is focusing his research on Project Challenge, Tennessee’s application of evidence-based policy between 1985 and 1993 to increase student achievement by reducing school class-size.
About Project Challenge
Over the 4 school years between 1985-86 and 1988-89 the Tennessee legislature authorized $12 million for an RCT—known as Project STAR—to estimate the relative effects of small vs. large classes, with and without teachers’ aides, in grades K-3. The STAR results showed that small classes increased reading and math scores by nearly 0.2 standard deviations (while teachers’ aides had no effect). In 1989, the state launched Project Challenge, which reduced K-3 class sizes in 17 of Tennessee’s poorest school districts, primarily in Appalachian east Tennessee, map below. After the evaluation of Project Challenge, the state reduced K-3 classes statewide as part of the 1993 Basic Education Plan.
Although Project STAR is one of the most famous RCTs in policy history, its implication that class size reduction can increase student achievement by nearly 0.2 standard deviations is not widely accepted, in part because later class-size reduction policies in California and Florida caused little or no rise in achievement. We cannot expect a small RCT in a small state to predict the consequences of a major policy change in a large state a decade later. Yet we do this frequently. About the only thing reasonably expected is for an RCT to predict the results of a similar policy implemented in approximately the same time and place. That is why it is important to evaluate the effects of Project Challenge.
About Dr. Paul von Hippel
Paul von Hippel studies educational inequality and the relationship between schooling, health, and obesity. He is an expert on research design and missing data, and a three-time winner of best article awards from the education and methodology sections of the American Sociological Association. Before his academic career, he was a data scientist who developed fraud-detection scores for banks including JP Morgan Chase and the Bank of America. More about Dr. von Hippel.