In a new NBER study, analysts pool estimates from lottery-based studies of the effect of charter school attendance on student outcomes, rescaling as needed so that the estimates of those effects are comparable across studies. They end up with a sample of 113 schools drawn from studies of KIPP and SEED schools, as well as charters in Massachusetts, New York City, Boston, and more.
On average, they find that each year children are enrolled at these schools increases their math scores by .08 standard deviations and their ELA scores by .04 SD on average, yet there's wide variation as expected. They link impact data to school practices, inputs, and characteristics of fallback schools (the non-charter schools that lottery losers attended the following year). They find that schools that have adopted a “no-excuses” model—which typically includes extended instructional time, high expectations, and uniforms—are correlated with large gains in performance. But noting that such schools are also concentrated in urban areas with poor-performing schools, analysts determine that the gains are largely a function of the poor performance of fallback schools. Once they control for the performance of the fallbacks, intensive tutoring is the only no-excuses characteristic that is consistently associated with student improvement. (They also examine and dismiss teacher feedback, data-driven instruction, instructional time, and high expectations.)
Although this latter finding is based on a smaller sample of fifty-seven schools—and is suggestive rather than dispositive, as there’s no random assignment of schools to tutoring—it nonetheless reinforces other studies that demonstrate the impact of tutoring. Keep in mind, however, that many of the qualities that makes no-excuses schools distinctive are hard to measure (like school culture). So we should avoid the temptation to read the study’s message as “Tutoring is all that matters”!
SOURCE: Julia Chabrier, Sarah Cohodes, and Philip Oreopoulos, "What Can We Learn from Charter School Lotteries?," NBER (July 2016).
