The Education Gadfly Weekly

As Supreme Court justices weigh the constitutionality of Cleveland's voucher program in the next few months, their assessment of the benefits of school choice is apt to influence their decision. If there is evidence that voucher programs offer sound educational opportunities to poor children and prompt the reform of low-performing urban public schools, the "brethren" are more likely to find ways to uphold such programs' constitutionality.

Mindful that the research evidence is likely to carry weight in this important case, many organizations have sought to spin it to suit them. The main strategy of those hostile to vouchers has been to describe the evidence as "mixed" or "inconclusive" at best. Saying that research findings are inconclusive has a reasonable, moderate tone that appeals to journalists who seek balance and tend toward skepticism, even when such balance and skepticism are unwarranted.

In the case of voucher research, those labels are indeed unwarranted, though it's not hard to make it look otherwise. To turn strong and consistently positive results on behalf of vouchers into mixed and inconclusive ones, all that is needed is to apply an unreasonable high standard - perfection - to the voucher research.

One large example of this is "Rhetoric Versus Reality: What We Know and What We Need to Know About Vouchers and Charter Schools," a report issued by the Rand Corporation last month. "For most of the key questions," the Rand authors assert, "direct evaluations of vouchers and charter schools have not yet provided clear answers, and the list of unknowns remains substantially longer than the list of knowns." They do their best to cast doubt on existing studies of the effects of vouchers by using an approach most commonly seen in graduate school, where budding researchers are trained to critique extant scholarship harshly so as to hone their intellects, burnish their methodological sophistication and spot complex questions to explore in their dissertations. In the real world, however, policymakers cannot afford the graduate student luxury of critiquing ad nauseam. Since they must choose among options presented by the real world, based on the information that is available, they require a practical standard that enables them with reasonable confidence to compare various policy alternatives with the status quo. They know that more evidence will continue to roll in, policies will continue to evolve and programs will continue to be refined, replaced and renewed.

When compared with research used to support other education policies, the evidence supporting school choice is remarkably strong. Five random-assignment studies have investigated whether vouchers provide academic benefits to students who use them. They examined the impact of vouchers on children in Charlotte, Dayton, Milwaukee, New York, and Washington, D.C. Random assignment is the gold standard of research designs and program evaluations. Because chance distinguishes who is in otherwise identical treatment and control groups, this approach minimizes the risk that outcomes are caused by unobserved characteristics of students and families - or other events in their communities - rather than by the policy being studied.

All five of these studies showed that students using vouchers experience significant academic benefits. These consistently positive findings have held up under analysis and re-analysis by sundry researchers from Harvard, Princeton, the University of Wisconsin, Georgetown, Mathematica, and the Manhattan Institute. I know of only one other education policy that has been the subject of even one major random assignment study: the class-size reduction experiment in Tennessee. Most education policies are adopted without any experimental evidence, gold standard or less, concerning their effectiveness. (For example, the recently renewed federal Title I program, after 35 years, can muster no such evidence concerning its effectiveness in boosting the achievement of disadvantaged youngsters - its ostensible purpose.)

While the evidence that vouchers benefit youngsters who use them is strong, existing research has important limitations. As the Rand report correctly notes, voucher-induced learning gains are largely restricted to African-American students. Of course, this is partly due to the fact that the vast majority of students in the five voucher experiments conducted so far have been black. To learn more about possible voucher benefits for students from other backgrounds, we would need larger samples of students from these groups. Instead of hailing the finding that vouchers offer significant benefits to African-American students, however, Rand focuses on the "unknowns" for other children. Some people dwell on emptiness in glasses that are even three-quarters full.

As to whether voucher programs can have a positive impact on public schools in the district, the Rand report is even gloomier: "Whether the introduction of vouchers/charters will help or harm the achievement of students who stay in conventional public schools remains for the moment entirely unknown." The authors can say this because they excluded from their conclusion all studies that are not direct examinations of the effects of voucher programs on local public schools. For example, they give no weight to research by Harvard economist Caroline Minter Hoxby that shows that public schools demonstrate higher academic performance when they face additional competition (because they are located in metropolitan areas with more school districts). They ignore my own work with the Education Freedom Index that finds that states with a broader range of choices offered to parents have higher test scores. In short, they spurn all analyses of the current education marketplace that find that greater competition improves public school performance. This is akin to excluding animal studies when trying to determine if a chemical causes cancer in humans - or whether a new treatment might help cancer patients.

The Rand researchers also missed another important Hoxby study that directly examines the effect of new school choice programs on public school performance. She probed charter-school programs in Arizona and Michigan as well as the Milwaukee voucher program and found that the more exposed public schools were to these sources of competition, the greater their academic improvement. This important study, published in the Winter 2001 issue of Education Next, became available on the web in mid-2001, yet Rand's analysts missed it in their "exhaustive review of the existing literature on vouchers and charter schools."

On the collateral question of how school choice affects civic values, they again arrived at a negative conclusion only by excluding significant studies and relevant analyses. In this case, they missed a significant study by Georgetown professor Patrick Wolf and others that found that recipients of a voucher in the D.C. experiment were significantly more likely to be tolerant of the political activities of disliked groups (which is the conventional measure of political tolerance in social science research) than were the public school students who applied for a voucher but were denied one by lottery. Note that these differences were produced after just two years in different schools.

Such gloomy assessments of voucher research are possible despite relatively strong evidence to the contrary because the standards to which critics seek to hold voucher research are different from those they apply to other policies and programs. That's the essence of a "double-standard." Of course there's much still to be learned about the effects of school choice. But unless we try voucher programs on a larger scale and continue them for a longer period, we'll never be able to resolve today's uncertainties or gather more definitive evidence. So long as the existing research is assessed using an overly strict standard, however, policymakers may wrongly be discouraged from trying bigger and bolder programs, scholars may be discouraged from gathering further evidence and the Supreme Court may even be discouraged from allowing today's small programs to continue at all.

Jay P. Greene is Senior Fellow at the Manhattan Institute for Policy Research. The opinions expressed here are his own

"Phonics and testing, we're meant to believe, are an intensive therapy set to turn around laggard schools," writes Stephen Metcalf in The Nation, "But administrators, teachers, parents and children know better." The real story behind President Bush's education plan, says Metcalf, is that "The big players now at the education table, some with a considerable financial stake in the new regime, believe that money is best spent on testing and textbooks, rather than on introducing equity into the system over the long term." Readers may be entertained by the pages he devotes to exposing an alleged plot behind the spread of scientifically based reading instruction, and by the fingers he points (at textbook publisher McGraw-Hill, among others). But they shouldn't be surprised; after all, "to teach phonics you need a textbook," as well as worksheets and tests, all items sold by McGraw Hill, Metcalf points out. "The amount of cross-pollination and mutual admiration between the Administration and that [McGraw-Hill] empire is striking," Metcalf writes, with the McGraw Foundation awarding Bush Education Secretary Rod Paige its highest educator's award and Paige later serving as keynote speaker at a company conference. As for reading, Metcalf is alarmed by the Bush administration's emphasis on phonics and its use of the findings of the National Reading Panel, which he alleges was wrongly presented to the public as an end to the Reading Wars. Not so, accordingly to this screed. Rather, the powerful Washington public relations firm hired by the government to promote the report has close ties to McGraw-Hill's flagship literacy product, Open Court. If you savor leftist paranoia and aren't much concerned about facts, have a peek at "Reading Between the Lines," by Stephen Metcalf, The Nation, January 28, 2002.

Education giving is also taking a hit as philanthropists' bank accounts shrink and some redirect their resources toward fighting terrorism and supporting domestic relief efforts in the aftermath of the September 11 attacks. In a commentary in Education Week, the Fordham Foundation's Checker Finn and Kelly Amis express the hope that recession will prompt education donors to focus on high-yield strategies and high-impact projects with the potential to transform the K-12 system instead of just adding resources to it. Many philanthropists place their bets on strategies that Finn and Amis judge futile in most cases - strategies that assume the existing public education system wants to change but lacks the wherewithal to pull it off. Philanthropic gifts based on such assumptions only work when the circumstances are exactly right, they argue. More promising, in their view, are philanthropic efforts that seek to alter how the education system operates by pressing upon it from outside - and force it to change out of necessity - through reforms based on standards-based accountability or competition. The authors identify two philanthropists who have pursued wise reform strategies: Tom Luce, a Texas attorney who used his own time and money to launch Just for the Kids, a nonprofit organization that tracks schools' academic progress and makes it easy for parents to investigate and compare schools over the internet, and John Walton, the Wal-Mart heir who has helped launch charter schools and supported scholarship programs to help low-income students attend private schools. The authors end by reflecting on the experiences of the Thomas B. Fordham Foundation in Dayton. See "Making More of Less," by Chester E. Finn, Jr. and Kelly Amis, Education Week, January 9, 2002. This commentary is a highly condensed version of a book released by the Foundation late last year. Making It Count: A Guide to High-Impact Education Philanthropy can be downloaded at http://www.edexcellence.net/detail/news.cfm?news_id=39, where you will also find instructions for ordering a free copy.

The faltering U.S. economy has put an end to a decade of budgetary good times for schools, with the recession opening a $40 billion hole in many states' general funds on which schools rely heavily, reports Daniel Wood in The Christian Science Monitor. Among the hardest-hit states are New York, Florida, Illinois, Minnesota, Oregon, and California, where Gov. Gray Davis announced a $850 million cut to public education last week. Belt-tightening moves by districts in these states include laying off teachers, curtailing textbook and other purchases, raising class sizes, postponing repairs, and scaling back teacher training. While the $26.5 billion education reform law signed by President Bush last Tuesday will provide some relief, those funds are largely earmarked for specific purposes like testing and literacy - and implementing the new accountability regime could cost more than Washington will provide. For details see "Recession saps school budgets," by Daniel Wood, The Christian Science Monitor, January 10, 2002.

Data warehousing, data-driven decision making, or business intelligence - whatever its name, it's the latest thing for managing school systems, according to a short article in this Sunday's Education Life supplement to The New York Times. Mimicking businesses, accountability-minded administrators are gathering up the reams of data they collect each year and storing them on centralized computer systems so they can be linked and analyzed for patterns. Such data include test scores, grades, attendance records, and information about teacher certification, student demographics, immigration patterns, medical problems, disciplinary actions, and student transfers, among other things. Once analyzed, administrators hope the data can be used to improve teaching, raise test scores, and deter dropouts (and to justify new programs to the state legislature). Concerns have been raised that such data may be used to take punitive measures against teachers whose students are performing poorly, the author notes, but school systems say that this is not their intent. See "Business Intelligence: Insights from the Data Pile," by Leslie Berger, The New York Times, January 13, 2002.

Beatriz Chu Clewell and Ana Maria Villegas, The Urban Institute December 2001

Many urban and rural school districts today face a serious shortage of teachers, especially in fields such as bilingual education and special education, and in subjects such as mathematics and science. To help bring new teachers into the profession, the Wallace Reader's Digest Fund has invested over $50 million since 1989 in the Pathways to Teaching Careers program, which prepares classroom aides and returning Peace Corp volunteers to become certified teachers. In this report, Urban Institute researchers look at how the Pathways program has done. They find that it turns out teachers who, overall, are more committed, more diverse and more willing to take on challenging assignments than those emerging from traditional preparation programs. Pathways participants are seen by their supervisors, principals, and independent assessors as more effective than the typical beginning teacher in their schools. Over 81 percent of them remained in the field for at least three years after completing the program. And fully 84 percent of Pathways graduates took jobs in high-need school districts. The success of Pathways serves as further proof that drawing teachers from nontraditional pools is an effective way of attracting good people to a field in need of new and diverse talent. To see the full version, go to http://www.urban.org/education/absence-unexcused.html or order a copy by calling 877-847-7377 or emailing [email protected].

Patrick Basham, Fraser Institute 2001

Home schooling, the leading form of education in North America during parts of the 19th century, is making a comeback in the 21st century. Home-schoolers today may make up as much as 3.4 percent of the school-aged population in the U.S. and 1.0 percent in Canada. This report, written by Cato Institute scholar Patrick Basham and published by the Fraser Institute, explores the recent surge of home schooling in the U.S. and Canada. (It should be noted, though, that Basham's estimates for U.S. home schooling are a lot higher than those recently published by the National Center for Education Statistics.) He describes the ideological and pedagogical factors that motivate American and Canadian parents to teach their children at home and he identifies interesting features of home schooling families. (In the U.S., for example, the fastest growing group of home-schoolers is Muslim Americans.) Basham finds that home schooled students - especially in the U.S. - are just as likely to be involved in social activities outside the home and more apt to succeed academically then their public school peers - all for a fraction of the cost of public education. To order a hard copy of this concise report, contact the Fraser Institute at 1770 Burrard Street, 4th Floor, Vancouver, B.C., V6J 3G7; phone 800-665-3558; or download one from http://www.fraserinstitute.ca/publications/pps/51/homeschool.pdf.

Governor's Task Force on Efficiency and Accountability in K-12 Education December 18, 2001

According to this brief report by Arizona Governor Jane Hull's Task Force on Efficiency and Accountability in K-12 Education, Arizona has no time to waste in implementing standards-based accountability. Citing low achievement, the fact that curricula are not aligned to standards in more than half the state's schools, and a system that is "morally and economically unacceptable," the task force sets forth a series of recommendations to bolster accountability and make good use of the additional funds provided by Proposition 301, which authorized a sales tax increase to support K-12 education. Endorsing Arizona LEARNS, the school accountability plan developed by Superintendent of Public Instruction Jaime Molera, the report calls for: 1) aligning all curricula at every grade level to the state's academic standards; 2) implementing the state Department of Education's "Purposeful Accountability System" which includes greater reporting and dissemination of student and school performance data; 3) implementing a statewide program of student achievement awards and teacher pay-for-performance linked to student achievement; 4) turning underperforming schools around; and 5) directing financial resources toward student achievement. The report describes the "next steps" that must be taken - some requiring legislation, some not - to carry out the task force's recommendations. Puzzled readers may well wonder why this is all again under discussion in a state that's already been through the development of sound academic standards, new tests and accountability systems (as well as the country's largest charter-school program). The short answer, as we observe these matters from afar, is that Molera, the Governor's Task Force, and some legislators are backing away from the "high stakes" approach of former superintendent Lisa Graham Keegan and instead embracing strategies (such as those outlined above) favored by Arizona's public school establishment. We are, accordingly, underwhelmed, but if you'd like to see this pious report for yourself you can find a PDF version at http://www.wested.org/online_pubs/arizona.pdf.

Howard Fuller and Kaleem Caire, National Center for Policy Analysis 2001

Co-written by school choice guru Howard Fuller and Kaleem Caire, CEO of the Black Alliance for Educational Options, this book refutes ten of the most widely proffered arguments against school vouchers, promoted especially by the National Education Association and the American Federation of Teachers. These "myths" include accusations that vouchers "cream" the best students, rob public schools of needed funds, deny service to disabled students, and so forth. Fuller and Caire introduce each allegation with direct quotes from choice opponents and then set the record straight. Myth #5, for example, says that "Tax-funded school voucher programs do not improve the academic achievement of voucher students." The authors found variations on this statement made by AFT President Sandra Feldman and others to contradict the conclusions of several independent evaluations of experiments that have found statistically significant benefits for students who used vouchers. (For more information about these studies, see Jay Greene's editorial above.) Copies of the book - a handy and concise weapon in the choice supporter's arsenal - are available from the National Center for Policy Analysis, 12655 North Central Expressway, Suite 720, Dallas, TX 75243; telephone 972-386-6272.