For five years, the EdChoice Scholarship Program has enabled students to escape low-performing schools (those rated D or F for two out of the last three years) in Ohio for, presumably, greener pastures in private schools. Fourteen-thousand students, the maximum allowed by state law, in low-performing schools are using this publically funded voucher to attend private schools of their choice.
Until recently, performance data on EdChoice students have not been available. But this year, thanks to new requirements in state law, the Ohio Department of Education released data comparing how voucher students perform on state achievement tests with their district counterparts. The Columbus Dispatch featured this newly available data and concluded, “On the whole, Ohio students who used tax-funded vouchers to attend private schools last school year did no better on state tests than public-school students.”
The reporter reached this conclusion by comparing voucher student performance to the performance of students in their home districts. While this comparison is a reasonable starting point to understanding how well voucher students are doing, the comparison is far from fair. It is problematic to compare voucher students to the average scores of the entire home districts (which include schools of various quality levels, most of which are not low-performing enough to qualify for vouchers). Voucher-eligible students come from the worst-performing schools and comparing them to an entire district, including high-performing schools, is misleading. Using this comparison method voucher students will almost certainly always fall short on academic achievement tests simply because they are starting behind their peers.
A better approach, and one that is doable with the limited data available, is to compare EdChoice student performance to the performance of students in voucher-eligible public school buildings. This creates a more apples-to-apples comparison by allowing researchers to compare voucher students to the underperforming schools from which they fled.
The Dispatch analysis showed that public school students in Columbus City Schools outperformed voucher students in seven out of twelve academic tests. However, the opposite is true when one compares Columbus’s voucher students to students in the city’s voucher-eligible schools, as chart 1 illustrates.
Chart 1: Columbus: EdChoice Students vs. Voucher-Eligible Students
Source: Ohio Department of Education
Not only do EdChoice students outperform their peers in voucher-eligible schools in eight of twelve tests, they do so in some cases by a large margin. It is particularly interesting to note that in seventh- and eighth-grade reading, voucher students score more than 20 percentage points higher than their peers.
That being said, a look at EdChoice student performance in Cincinnati paints a completely different picture. In the Queen City, as chart 2 shows, EdChoice students only outperformed voucher eligible schools in four of twelve academic tests.
Chart 2: Cincinnati: EdChoice Students vs. Voucher-Eligible Students
Source: Ohio Department of Education
A few additional findings jump out from this sort of apples-to-apples analysis.
Reading proves to be an area of strength for students using vouchers in these two cities. While this is interesting, perhaps a better question to ask is why voucher students are performing so poorly in math? Is their performance a reflection of their time in the private school or of the teaching they received in the public school from which they fled?
Voucher students had stronger results in the middle grades (6-8) in reading, compared to students in voucher-eligible schools. Voucher students in Columbus and Cincinnati outdid voucher eligible schools by a margin as large as 25.5 percentage points.
Of course even this sort of data and analysis has limitations. For one, students using vouchers might be more motivated and arrive at private schools at a higher level of proficiency than their peers who stayed behind in the public school. This comparison cannot account for that. An ideal comparison would be to look at voucher users and voucher-eligible students on the waitlist, and such data could be made available now that Ohio has topped the voucher cap and has unique student identifiers for all students. Until then, comparing voucher students to voucher-eligible students is the next best thing.
