Matthew Chingos and Paul Peterson knew there would be attempts to undermine their finding that a New York City voucher program had a positive impact on the college attendance of black students. When the Brookings Institution released the Chingos/Peterson study last month, many news reports unfortunately focused on the fact that only black students seemed to benefit significantly from the small, privately funded program (the voucher was worth just $1,400 annually when it was offered to low-income kids in 1997). Still, while frustrating, the media coverage never cast aspersions on the most significant claims in the study.
While frustrating, the media coverage never cast aspersions on the most significant claims in the study.
That task has now fallen to an academic review from the National Education Policy Center, a group that could never be confused for a friend to school choice. Sara Goldrick-Rab of the University of Wisconsin-Madison makes the preposterous claim that an “unmentioned” measurement error in the dependent variables (college attendance rates) suggests that there really are no statistically significant differences between the voucher’s impact on black students than on other students. Further, Goldrick-Rab argues that Chingos and Peterson fail to account for any negative effects from the voucher program that could explain the overall impact.
At Education Next, Chingos appropriately responds:
- The “unmentioned” measurement error already appears in the standard error for the study. Indeed, Chingos writes, college attendance is imperfect as a measurement because the process used to match students to college-enrollment records is imprecise. But what Chingos doesn’t mention is that it was that process that made the study so enterprising. Because Peterson got Social Security numbers, names, and dates of birth for the voucher students when they first applied, Chingos was able to match that data with the National Student Clearinghouse—which was the best way to track college attendance. And, it should be noted, this study was the first random-assignment experiment of voucher effects on college attendance (the randomized trial is the gold standard of research).
- The claim that the voucher program “failed to increase the college enrollment rates of students from low-income families” is false, writes Chingos, who explains, “The overall impact estimate is not estimated with enough precision to conclude that the voucher intervention had no effect. The overall impact is not statistically significant from zero, but it is also not statistically significant from a negative impact of 3 percentage points or a positive impact of 4 percentage points.”
In the end, Chingos writes, “the one result that can be reached with confidence is that the impact of vouchers for African Americans was positive” (more precisely, Chingos and Peterson found that black students who won the voucher lottery and used the voucher were 24 percent more likely to attend college than students who didn’t win the lottery). Nothing in their report overpromises what a voucher program can offer; in fact, the authors argue that the results cannot be generalized to other settings. “Scaling up voucher programs will change the social composition of private schools,” the report states. “To the extent that student learning is dependent on peer quality the impact reported here could easily change.”
Chingos and Peterson gave us more reason to study how current voucher programs can benefit students who have the greatest need for more educational choices. The Goldrick-Rab review only tries to argue that such an inquiry is a waste of time. That’s a sad addition to the literature.
