Last month, my colleagues Mike Petrilli and David Griffith had a conversation with Patrick Wolf, a leading school choice scholar at the University of Arkansas, about the impact of voucher programs on the Education Gadfly Show podcast. Wolf was happy to join us on this fifth episode of our Research Deep Dive series and discuss what he’s learned in his nearly two-decades studying school choice. (Feel free to listen on our website or using your usual podcast platform. And see our discussions in the series on teacher effectiveness, school discipline, school closures, and urban charters.)
School voucher (a.k.a., “scholarship”) programs have offered low-income families access to private schools. Historically, most rigorous studies found positive short- and long-term impacts for participating students—especially for Black and disadvantaged students—and positive spillover effects for students in district schools. However, the literature finds mixed results more recently on achievement in some contexts and for other subgroups. Wolf helps us make sense of this and explains whether the most rigorous and recent scholarship still finds overall positive effects. We hope this summary gives you a good sense of what the data say about school choice.
—
1. Why do we see mixed results on achievement for students participating in voucher programs?
There are two reasons for this: curricular differences and scaling issues. First, traditional public schools have a “homefield” curricular advantage in that they are often required to adopt standards- and test-aligned curricula. Private schools, however, might face misalignment between their own grade-level priorities and what’s covered on annual statewide tests. Similarly, private schools simply aren’t as focused on the test score gains as a metric, but may focus on improving other metrics such as college enrollment, so their students all less prepared for them.
Second, voucher initiatives might not always scale well. Evidence, for example, comes from a 2017 Louisiana study co-authored by Wolf, which found net negative results on achievement. He believes this is because scaling happened too quickly in that state, and that a sudden increase in voucher participation filled many private schools with students whom they may not have been well-equipped to support. Studies find the best results in urban settings, often from Catholic schools that have decades of experience serving immigrant communities and Black and Hispanic students.
Earlier studies: positive results
Jay P. Greene, Paul E. Peterson, and Jiangtao Du, “Effectiveness of School Choice: The Milwaukee Experiment,” Education and Urban Society (January 1999).
William G. Howell et al., “School vouchers and academic performance: results from three randomized field trials,” Journal of Policy Analysis and Management (2002).
Jay P. Greene, “Vouchers in Charlotte,” Education Matters (Fall 2001).
Recent studies: mixed or negative results
Jonathan N. Mills and Patrick J. Wolf, “Vouchers in the Bayou: The Effects of the Louisiana Scholarship Program on Student Achievement After 2 Years,” Educational Evaluation and Policy Analysis (February 2017).
R. Joseph Waddington and Mark Berends, “Impact of the Indiana Choice Scholarship Program: Achievement Effects for Students in Upper Elementary and Middle School,” Journal of Policy Analysis and Management (August 2018).
Ann Webber et al., “Evaluation of the DC opportunity scholarship program: Impacts three years after students applied,” Institute for Education Sciences NCEE 2019-4006 (2019).
2. Do voucher programs offer benefits to participating students beyond test score impacts?
The research finds consistent, positive impacts on the likelihood that high school students will graduate, that they will enroll in college, and recently and most promising, that they will earn a degree. Wolf explained that five out of the six most recent studies on high school graduation found gains ranging from 4–21 percentage points. Seven out of eight found a clear positive effect on college enrollment, with one finding no effect. And four recent studies on college completion found positive effects. The student subgroup that most benefits has been Black students. Here are some of the studies he referenced:
M. Danish Shakeel, Kaitlin Anderson, and Patrick Wolf, “The Participant Effects of Private School Vouchers Across the Globe: A Meta-Analytic and Systematic Review,” EDRE Working Paper No. 2016-07 (May 2016).
Megan Austin, R. Joseph Waddington, and Mark Berends, “Voucher Pathways and Student Achievement in Indiana’s Choice Scholarship Program,” The Russell Sage Foundation Journal of the Social Sciences (March 2019).
Matthew M. Chingos and Paul E. Peterson, “Experimentally Estimated Impacts of School Vouchers on College Enrollment and Degree Attainment,” Journal of Public Economics (February 2015).
3. What explains the positive outcomes on high school graduation?
Is it peer effects? Do private schools have lower graduation standards? Or are the private schools doing something right when it comes to getting kids across the finish line? Wolf believes it’s the lattermost explanation. He talks about a Milwaukee private school vice principal who explained his philosophy: Student success is a three-legged stool that requires the participation of the student, the parent, and the school. But if a student drops out and quits, the vice principal said, he blames the school. Wolf believes that this sense of ownership is part of the secret sauce in affecting student outcomes, including graduation rates, but also better civic participation and lower rates of criminality.
4. What are the competitive effects of voucher programs on traditional public schools?
There is overwhelming evidence that competition helps public schools. Wolf says that “this idea that public schools are a fragile ecosystem, and they can only serve students if they have no competition...that claim has been completely debunked.” He explains that twenty-six out of twenty-eight studies on the effect of choice program launches or expansions show positive effects. The research for competitive effects actually has identified several mechanisms behind the impact. It finds that public schools improve communication with parents and are more likely to fire ineffective staff and try new interventions when faced with the possibility of losing students to private schools.
Anna J. Egalite, Jonathan N. Mills, “Competitive Impacts of Means-Tested Vouchers on Public School Performance: Evidence from Louisiana,” (January 2021).
David N. Figlio, Cassandra M.D. Hart and Krzysztof Karbownik, “Effects of Scaling Up Private School Choice Programs on Public School Students,” NBER Working Paper #26758 (February 2020).
David Figlio and Krzysztof Karbownik, “Evaluation of Ohio’s EdChoice Scholarship Program: Selection, Competition, and Performance Effects,” Thomas B. Fordham Institute (July 2016).
Cecilia E. Rouse, Jane Hannaway, Dan Goldhaber, and David Figlio (2013), “Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure,” American Economic Journal: Economic Policy (May 2013).
5. How challenging is it to conduct studies on vouchers in districts that have a mixed education ecosystem, such as those with high charter school enrollment?
Isolating the effects of one form of choice makes it harder to evaluate effectiveness and puts downward pressure on the effects. In other words, school choice studies may be underestimating the impact of certain kinds of choice because of the abundance of options. However, many scholars have found ways of controlling for this by using distance from public schools or the concentration of private schools in an area in their analyses. Here are two examples of studies using these kinds of variables:
Anna J. Egalite, Ashley Gray, and Trip Stallings, “A Comprehensive Evaluation of the North Carolina Opportunity Scholarship Program,” Reports 1–7, North Carolina State University (2017–2020).
David Figlio and Cassandra M. D. Hart, “Competitive Effects of Means-Tested School Vouchers,” American Economic Journal: Applied Economic (2014).
6. The final question: What is the fairest critique made against the use of vouchers by thoughtful scholars who see the research but remain skeptical?
Giving parents the freedom to choose doesn’t necessarily mean they’ll make the choices that some experts prefer, or the choices that lead to what some may deem to be the most socially desirable outcome. It means policymakers and reformers must accept some loss of control. For instance, families of color may choose urban charter schools where their ethnic or racial group is the majority, leading to segregation. But the question Wolf raises to skeptics of vouchers is: “Do the benefits outweigh the costs?” To which he answers, “Yes, they do.”
