A new study from the National Bureau of Economic Research examines how Louisiana’s statewide voucher program affects student achievement. The Pelican State expanded its program statewide in 2012; by 2014, twelve thousand students had applied for more than six thousand slots to attend 126 private schools. Because the program was oversubscribed, the vouchers were randomly assigned such that some kids were offered vouchers and some weren’t. This study focuses on roughly 1,400 grade students in grades 3–8 who applied in fall 2012—the first application cohort after the program expanded.
The primary (and surprising) finding is that attending a voucher-eligible private school reduces voucher students’ test scores in math, ELA, science, and social studies (though ELA is not significantly lower). Math scores go down by 0.4 standard deviation one year after the lottery, and for other subjects, the drop is between one-quarter and one-third of a standard deviation. Voucher use also reduces the probability of being promoted to the next grade and shifts students into lower state performance categories. The outcomes are even bleaker for younger children.
In short, this is all very bad news. But remember that these are first-year outcomes, and first-year evaluations of anything ought to be taken with a grain of salt.
So what explains the negative findings? They stand out in comparison with those of other gold-standard voucher studies, which have identified neutral or positive results—particularly for African American students, who make up the vast majority of recipients in Louisiana. Let’s examine the limited evidence in the study that may provide an explanation.
Some analysts are pointing the finger at “overregulation.” Jason Bedrick of Cato, among others, argues that the best private schools in the state opted out of the program because they didn’t want to comply with its many mandates. Only the most desperate, cash-starved schools participated, which explains the disastrous results. (The private school participation rate in Louisiana—about one-third of schools—is much lower than that seen in most other voucher or tax credit programs.)
Does that explanation hold water? It’s certainly true that Louisiana’s program comes with many requirements for schools. To participate, they have to administer the Louisiana state test to voucher students; receive a rating based on the achievement of their voucher students; and risk being deemed ineligible to enroll new voucher kids if they do poorly for two consecutive years. These accountability provisions are all policies that Fordham has promoted in the past.
But that’s not all. Schools also must have an open enrollment policy, meaning that they cannot apply their usual admissions requirements. Plus, schools must accept the voucher amount as the full tuition payment (which is about $3,300 less than what the students’ “sending” districts spent per pupil). That’s largely a moot point, however, since eligibility is limited to students from families earning below 250 percent of the federal poverty line (on average, applicants’ families earned roughly $15,400).
So it’s not hard to believe that the litany of requirements is keeping many private schools on the sidelines. But is there really evidence that non-participating schools are stronger than the ones that played ball, because all of the participating schools were “desperate” for students and dollars?
In short: no. Results for schools with rapid enrollment declines in previous years were no worse than other schools to which students applied.
Further, there’s no evidence that testing and accountability mandates are keeping many private schools from participating. But an AEI survey in 2015, which focused on Louisiana’s program in particular, found that “administering the state accountability test” was a “major factor” in the no decision for just 34 percent of non-participants. This meshed with a 2013 Fordham study in which our principal investigator, David Stuit, found that 37 percent of non-participants said the requirement to comply with state testing was “extremely or very important” to their decision not to take part.
But it seems just as likely that non-participating schools are concerned about the other regulations at play, and especially the limitation on using their regular admissions standards. In AEI’s study, most of the non-participating principals in Louisiana cited “future regulations that might come with participation” as their biggest hurdle—the fear of the unknown. And in the Fordham study, Stuit found that regulations restricting student admissions and schools’ religious practices are more likely to deter school participation than requirements pertaining to academic standards, testing, and public disclosure of achievement results.
Maybe the testing and accountability requirements are encouraging some private schools in Louisiana to stay away. But isn’t it just as likely that they aren’t thrilled with the idea of accepting extremely poor students (some of whom wouldn’t meet their usual admissions standards) and losing money in the bargain?
All of this to say that, sure, we need to need to be careful about meaningfully safeguarding the autonomy of private schools in voucher programs. But this study left us with a simpler takeaway: Some private schools in Louisiana stink. But if we hadn’t tested those kids, we would never have known that.
SOURCE: Atila Abdulkadiroglu, Parag A. Pathak, Christopher R. Walters, "School Vouchers and Student Achievement: First-Year Evidence From the Louisiana Scholarship Program," NBER (December 2015).
