The Education Gadfly Weekly

I am a conflicted man.

Professionally, I lead Ohio’s auditing staff, a team of financial experts whose job it is to verify that tax dollars are being properly spent and to root out any misuse or theft of public money. That includes charter school spending.

Yet personally, I’m a strong proponent of the charter school movement. I believe in the lifetime benefits of school choice and affording all parents the ability to choose the school that will best serve their children.

My friends sometimes question how I can be so tough on charters when I personally support them. The answer, I tell them, is simple: We don’t play favorites. We can’t. We shouldn’t. Doing so would erode the public’s trust in our office, which we must faithfully and ardently protect. To ignore the misdeeds of the few problem charters would stain the great work of many. Turning a blind eye to the problems in a charter school, or any school, would mean that we failed our children, which is never an option.

It’s a conflict that public officials often face when their official duties require them to make decisions running counter to their personal beliefs.

The mission of the auditor of state’s office is to keep governments and schools honest—to weed out the bad so that the good can flourish. The accountability and transparency provided by my office also shines a light on the charter schools that are doing things right and meeting the true purpose of the community school system.

That purpose is educating kids. I believe that we need to evaluate the success of charter schools not with data alone, but also by considering the lives they touch. While I have cited charter schools for mismanagement or outright fraud, I have witnessed many charter successes as well. The successes don’t make easy headlines. But they are significant nonetheless.

I recently became aware of a young woman by the name of Anna Marie Ridenour. When she was ten, she knew that she loved learning but did not enjoy attending her traditional school. It just wasn’t the right fit. She enrolled in a new online school called Ohio Connections Academy.

What many do not understand, she told the Cincinnati Enquirer, is that “students enroll in a virtual school for various reasons. I needed a more challenging curriculum. But some students arrive struggling academically or socially or with medical problems that make attending a traditional school problematic.”

Ridenour thrived at the online school, and today she is a math teacher there. We’ll never know where her prior path would have taken her, but we know that her charter school had a profound effect on her life. Stories like hers are why charter schools must remain viable and accountable.

The underlying premise of choice for students, parents, and society is rooted in the American principle of freedom. Being able to choose the best education for our children creates healthy competition that should elevate quality and lead to higher performance.

While it’s important to draw attention to problems we uncover, I’ve made it our responsibility to highlight the work of extraordinary schools. We award special citations almost weekly to celebrate those who have achieved a standard of excellence.

In keeping with that principle, my office will host a Charter School Summit on August 11–12 to share the best practices of our charter schools. Charters have different issues than their public school counterparts, and this summit will allow school leaders to receive training, share their experiences, and learn how their peers handle unique challenges.

Some days, the work of my office makes my friends happy. Other days, I frustrate or disappoint them.

In this respect, I am a man without a country. While I greatly value the choice and freedom that charter schools afford us, my loyalty is to taxpayers and our citizens, not special interests. Even those I support.

Dave Yost is Ohio’s elected auditor of state and a recipient of the National Alliance for Public Charter Schools’ Charter Schools Champion Award.

This is the first in a series of essays marking the twenty-fifth anniversary of America’s first charter school law. These commentaries are informed and inspired by our forthcoming book (co-authored with Bruno V. Manno), Charter Schools at the Crossroads: Predicaments, Paradoxes, Possibilities, to be published this fall by Harvard Education Press. Read the other essays here, here, here, and here.

Next month marks the twenty-fifth anniversary of the enactment of America’s first charter school law, which Minnesota Governor Arne Carlson signed on June 4, 1991. This statute birthed a sector that has become not just a source of new schools for kids who need them, but also a structural reform of public education’s governance and delivery systems. It’s as close as K–12 schooling has come to what Clayton Christenson calls “disruptive innovation.”

This is worth celebrating—and charter advocates across the country have planned many festivities and events. But as we applaud this movement and the bold Minnesota lawmakers who launched it, let’s also recall what led up to it and, one might say, made it almost inevitable.

The onset of chartering was no lightning bolt. This audacious innovation had multiple ancestors and antecedents. School choice has colonial roots and was supported by early theorists such as John Stuart Mill. But it got a big boost in 1962, when Milton Friedman published Capitalism and Freedom, which described the potential of market forces to strengthen educational quality, efficiency, and productivity. Friedman favored a competitive, private-sector model and did not think that government should deliver education directly. Instead it would furnish needy families with vouchers that could be redeemed for education at any state-approved school. Friedman expected market forces to cause bad schools to improve or close, motivate decent schools to get better, and invite people to open new ones.

Four years after Friedman’s book, the eminent sociologist James Coleman rocked the education world (and more or less contradicted the central premise of LBJ’s year-old Elementary and Secondary Education Act) by showing that there is no reliable relationship between what goes into a school by way of funding, programs and rules and what comes out by way of learning. This forms the backdrop to the past half-century of what we now know as “standards-based reform,” which includes the crucial charter school concept of holding a school accountable for its results (measured, for better and worse, primarily by test scores).

Along came A Nation at Risk, jarring the country with news that its K–12 system wasn’t working nearly well enough. Its release was followed by a clutch of governors willing to try new approaches to producing stronger educational outcomes, including giving far greater freedom to schools that did so. “Governors are ready for some old-fashioned horse-trading,'' declared then-Tennessee Governor Lamar Alexander. ''We'll regulate less if schools and school districts will produce better results.''

At the same time, frustration was building with the efficiency and effectiveness of myriad governmental services as traditionally delivered. We were witnessing a new impulse to “reinvent government” by outsourcing some of its work to others who, working independently, might do it better and perhaps more economically.

In 1974, Ray Budde (a school teacher, principal, and eventual University of Massachusetts faculty member) had published a paper that described a form of chartering. His concept was focused within districts and on existing schools and designed to give teachers a key role in creating new programs and departments within them. Budde’s initial paper got little response, but he stuck with the idea. After A Nation At Risk and myriad other studies and reports called for sweeping K–12 reforms, he tried again with a 1988 treatise called Education by Charter: Restructuring School Districts. This one caught the eye of the late Al Shanker, who cited it in an influential speech at the National Press Club the same year (as well as in a later New York Times column, “A Charter for Change”).

Shanker expanded Budde’s focus, still seeing chartering as a way to foster teacher professionalism by allowing them to start new schools. He sought to create a quasi-marketplace in which “a school system might charter schools distinctly different in their approach to learning. Parents could choose which charter school to send their children to, thus fostering competition.”

These ideas reached Minnesota, where they caught the attention of a group of educators and policy innovators including Joe Nathan, Ted Kolderie, Curtis Johnson, and State Senator Ember Reichgott, a Democrat who would introduce and help pass that state’s pathbreaking charter law. Kolderie and Johnson, under the aegis of the Citizens League, served on a study committee that probed the chartering idea. The League’s December 1988 report, “Chartered Schools = Choices for Educators + Quality for All Students,” became the basis for the bill that Reichgott introduced in January 1989. The report built upon initiatives that had already made Minnesota a pioneer in school choice. One of these was a post-secondary enrollment option permitting eleventh and twelfth graders to take college courses (1985); another was open enrolment, which enabled children to attend any public school of their choice in Minnesota (1987–8).

Reichgott encountered fierce opposition at the outset, led primarily by the two state teacher unions. Her bill twice failed to clear the legislature. The following year, however, she got a boost when the D.C.-based Progressive Policy Institute published “Beyond Choice to New Public Schools: Withdrawing the Exclusive Franchise in Public Education.” Kolderie was its author, and he summarized it this way: “The proposal outlined in this report is designed to introduce the dynamics of choice, competition and innovation into America’s public school system, while at the same time ensuring that new schools serve broad public purposes.”

By 1991, Reichgott had enlisted more legislative allies from both sides of the aisle. She was finally able to pass her plan, enabling GOP governor Arne Carlson to sign the nation’s first charter school law on June 4. Fifteen months later, City Academy opened in St. Paul, and soon after that, California enacted the country’s second charter law.

With so many tributaries, it’s no surprise that the charter stream contains many different life forms. Its origins come from Left and Right, Democrats and Republicans, educators and economists, union leaders and governors, scholars and doers, from long ago and very recently. Just as important, its founders harbored disparate ideas about why it was needed and what it could and ought to do. It’s wrong, therefore, to try to tack a single “origin story” onto the charter phenomenon. In many ways, it was the composite (if not the consummation) of many impulses, the response to many needs, the embodiment of many hopes. Which also makes it messy. More on that to follow.

Editor's note: This post is the second in an ongoing discussion between Fordham's Mike Petrilli and the University of Arkansas's Jay Greene that seeks to answer this question: Are math and reading test results strong enough indicators of school quality that regulators can rely on them to determine which schools should be closed and which should be expanded—even if parental demand is inconsistent with test results? The first entry can be found here.

The prompt for this forum promised that we would explore “areas of agreement and disagreement.” I’m pleased, Jay (and not altogether surprised), to see that we share a lot of common ground. Let me start with that, then save what I see as our major dispute (what we can learn from reading and math scores) for another post.

I’m thrilled that you dismissed the extreme position of some libertarians, who argue that society should never override the choices of parents. You write:

I…do not mean to suggest that policy makers should never close a school or shutter a program in the face of parental demand. I’m just arguing that it should take a lot more than “bad” test scores to do that.

I agree entirely, and on both counts. First let me explain why “we” should, on rare occasions, close a school that some parents prefer. And second, let me discuss what else beyond “bad” test scores we might consider when doing so.

You and others have heard me argue ad nauseam that because education is both a public and a private good, it’s only fair that both parties to the deal have a say in whether that good is, well, good enough. We both abhor a system whereby a district monopoly assigns children to schools and parents must accept whatever is handed to them. But the flip side is that we should also reject chronically low-performing schools—those that don’t prepare their young charges for success academically or otherwise—and deem them undeserving of taxpayer support. They aren’t fulfilling their public duty to help create a well-educated and self-sufficient citizenry, which is what taxpayers are giving them money to do.

Furthermore, there are real financial and political costs to letting bad schools—including schools of choice—fester. We see this in many cities of the industrial Midwest (Detroit, Cleveland, and Dayton come to mind), where too many schools are chasing too few students. Perhaps the marketplace forces of “creative destruction” will eventually take hold and the weakest schools will disappear, allowing the remaining ones sufficient enrollment to ensure their financial sustainability and a higher level of program quality. But that process is taking an awfully long time, particularly when we’re talking about disadvantaged children who have no time to waste. The charter sectors in these cities would be stronger—academically, financially, and politically—if authorizers stepped in to close the worst schools. But some libertarians see that as paternalistic government intrusion. I think they are misguided; I hope that you agree.

Now to your second point, that “it should take a lot more than ‘bad’ test scores” to “close a school or shutter a program in the face of parental demand.” Hear, hear! This is the genius of effective charter school authorizers that look at a school’s big picture as well as its scores. Fordham’s Dayton office strives hard (and with fair success) to be that kind of authorizer. We certainly look at test scores—especially individual student progress over time (a.k.a. “value added”). But we also examine lots of other indicators of school quality, operational efficiency, and financial sustainability. (See our current accountability framework in the appendix here.) And most importantly, we know our schools intimately. We attend their board meetings, conduct site visits frequently, and get to know their teams.

So when we consider the painful step of closing a school (which we’ve had to do a handful of times), we’re hardly just sitting in our offices “looking at spreadsheets of test scores.” The same goes for other leading authorizers nationwide.

Not that it’s easy to identify measures beyond reading and math scores that are valid and reliable indicators of school success. I share your enthusiasm for character education, non-cognitive skills, high school graduation rates, and long-term outcomes such as college completion and labor market earnings. And I’d love to see states maintain regular testing in history, geography, science, and more. Whenever we can use those scores, we absolutely do. But as the early debate around the Every Student Succeeds Act illustrates, measures of character and non-cognitive skills don’t appear ready for prime time, and they may never be appropriate for high-stakes decisions. High school graduation rates, meanwhile, are perhaps the phoniest numbers in education. Long-term outcomes are just that—long-term, making them both difficult to tie to a school (especially an elementary or middle school) and not very helpful for making decisions in the here and now. And there’s no political appetite for more testing; if anything, everyone wants less. (Let me know if you disagree with my analysis.)

So where does that leave us? As far as I can tell, facing a trade-off, which is the normal state of affairs in public policy. We can either use reading and math gains as imperfect indicators of effectiveness while working to build better measures—buttressed by school visits and the like—or we can succumb to “analysis paralysis” and do nothing.

I know which one I prefer. What about you?

A sixth grader in Mountain Brook, Alabama, can be considered one of the luckiest in the country, enrolled in a district where he and his classmates read and do math three grade levels above the average American student. But a child of similar age in Birmingham, just five miles north on Route 280, would be in considerably worse shape; there, kids perform 1.8 grade levels below average. So how could a ten-minute drive transport students to a different educational galaxy? Well, look at some numbers compiled by a team of Stanford researchers: Mountain Brook is 98 percent white, with a median household income of $170,000. Birmingham is 96 percent black, with a median household income of $30,000. Sometimes the figures speak for themselves.

John Bel Edwards, the recently elected Democratic governor of Louisiana, has had an eventful few months. After being inaugurated in January, he’s wrangled with state lawmakers over their leadership selection process and hustled to patch a huge crater in the budget. But his education agenda, largely aimed at curbing the growth of the state’s charter sector and cutting funding for voucher students, has run aground over the last few weeks. After the state’s newly appointed Republican House Speaker stacked the body’s education committee with charter enthusiasts, the governor’s legislation died quickly. A traditional friend of teachers’ unions, Edwards hasn’t been able to move the ball for the groups that worked the hardest to elect him. As an education commentator affiliated with a local business group put it recently, "There is no compromise on school choice. That would never ever have been on the table."

As Fordham’s Victoria Sears wrote last month, the spring testing season has brought with it a plethora of technical mishaps in states across the country: internet connectivity failures in Alaska, mysteriously vanished student responses in Texas, and fouled-up testing platforms in Tennessee. Now we’ve gotten a closer look at specific complaints in the Lone Star State, where the local STAAR tests apparently included a question that had no correct answer. Meanwhile, in the overwhelming majority of states that have adopted next-generation assessments aligned to the Common Core State Standards, kids have been happily taking their tests without a hitch. (Well, they’ve been taking them, anyway.) What problems have arisen, such as a PARCC computer glitch in New Jersey, have mostly been addressed swiftly. And should we be surprised? Not only are the exams pretty darn good—they actually work.

On this week’s podcast, Mike Petrilli and Robert Pondiscio talk Trump, the role of test scores in determining school quality, and the opt-out movement. During the Research Minute, Amber Northern explains how the threat of NCLB sanctions reduced teacher absences.

Amber's Research Minute

Seth Gershenson, "Performance Standards and Employee Effort: Evidence from Teacher Absences," Journal of Policy Analysis and Management (April 2016).

This new research report from Educational Testing Services is a solid contribution to the evidence base—rather than the opinion base—about the so-called “opt-out” movement. Author Randy E. Bennett finds that parents’ refusal to let their children sit for standardized tests is “a complicated, politically charged issue made more so by its social class and racial/ethnic associations. It is also an issue that appears to be as much about test use as about tests themselves.”

Opt-out true believers will likely dismiss out of hand anything coming from a testing outfit. but they ought to take a long look. The report does a good job synthesizing data from both the national and state departments of education, published surveys, and other sources to put between two covers exactly what is known—and can be sensibly divined—about who is opting out and why. “Parents who opt their children out appear to represent a distinct subpopulation,” the report notes. In New York, for example, “opt-outs were more likely to be white and not to have achieved proficiency on the previous year’s state examinations.” Test refusers are also less likely to be poor or to attend a school district serving large numbers of low-income families or ELL students. None of this is entirely surprising (it largely matches the conventional wisdom), but it does offer light to a vitriolic debate. Opt-out has become a charged civil rights issue precisely because of who is not doing it. Testing means data. Data drives demand for school improvement and school choice. Thus far, the main beneficiaries of education reform—to which testing lends credibility and moral authority—are low-SES urban families, historically among the most likely to be outside looking in when it comes to good schools. The stakes are high for high-stakes tests.

Some parts of the country emerged as opt-out hotbeds, while in others kids, trudged off to school with No. 2 pencils freshly sharpened. Why is that? “Some states, like California, did not link teacher evaluation to student test scores.” Others made linkages but “did not make test scores the preponderant evaluation criterion.” Still others merely delayed tying test scores to teachers. “Finally, most states avoided direct confrontation with teacher unions,” notes Bennett. He concludes that “while the majority of the public opposes opt-out, the minority that supports it is sizable, organized, vocal, and politically effective.”

That being the case, Bennett concludes with a discussion of how the assessment community might respond to the existential opt-out threat. I hope I will not be branded a cynic for thinking that his first suggestion—“more active and effective communication targeted at policy makers, state department staff, local educators, parents, students, and the public”—might suffer from an excess of optimism. The charged politics of opt-out seem unlikely to be ameliorated by “foster[ing] greater understanding of the value of high-quality [Bennett’s emphasis] assessment and its appropriate use” anytime soon.

Similarly, it would surely get the attention of opt-out parents to hear test makers concede “that we agree with opt-out advocates…about the limited relevance—and negative instructional effects—of relying too heavily on multiple choice tests.” But the second part of the message might fall less winningly on opt-out ears: “Common Core assessments have made important moves…such that the term standardized test no longer means ‘fill in the bubbles.’” When the report acknowledges that “the use of test scores to evaluate teachers is a highly controversial practice, even within the educational research community,” you know there’s a “but” coming. And here it is: “That said, the need to evaluate educator performance is a legitimate one, and student results might play some role, pending validation evidence.” In the end, “the community needs to build assessments that encourage participation “because parents, teachers, and students see participation, and preparation, as worthwhile learning endeavors,” writes Bennett.

And I am Marie of Romania.

OK, that was cynical. I take it back. While the ETS report is an important contribution to understanding who is opting out and why, and a worthy effort to engage various stakeholders in a conversation about the form and uses of testing, it’s hard to have a nuanced conversation in a nuance-averse environment. Quelling the opt-out impulse almost certainly has less to do with better tests and communications than how we use test results—including, most notably and contentiously, using them for teacher evaluations. Test makers can’t do that alone.

SOURCE: Randy E. Bennett, “Opt Out: An Examination of Issues,” ETS Research Report Series (April 2016).

Previous research has found that oversubscribed urban charter schools produce large academic gains for their students. But are these results related to test score inflation (defined by one assessment expert as increases in scores that do not signal a commensurate increase in proficiency in the domain of interest)? In other words, do these schools merely figure out how to prepare their students to do well on the high-stakes exam, or are they contributing to real learning writ large?

To explore this question, a recent study examines state testing data from 2006 to 2011 at nine Boston middle school charters with lottery-based admissions. By exploiting the random nature of the lottery system, prior studies have found that these schools produce substantial learning gains on the Massachusetts Comprehensive Assessment System (MCAS).

To carry out the analysis, author Sarah Cohodes breaks down the learning gains by the various components of the state assessment—akin to how one might disaggregate overall gains by student subgroup. A math assessment might contain several different testing domains (e.g., geometry versus statistics), with some topics being tested more frequently than others. Cohodes’s hypothesis is as follows: If the gains are attributable to score inflation, we might expect to see stronger results on frequently tested domains relative to obscure ones. Teachers might strategically focus instruction on topics with the highest chance of appearing on the exam, thus inflating scores. This is a possibility; as the author notes, a case study of Boston charters revealed that “teachers use publicly available MCAS items from prior years...and teachers constantly track their students’ progress on content that is tested.”

The study finds that the Boston charter school effect is dispersed evenly across the test items. In math between the sixth and eighth grades, charter students enjoyed a 0.25–0.35 standard deviation gain along all five topics: geometry; measurement; number sense and operations; patterns, algebra, and relations; and data analysis, statistics, and probability. Comparable results emerged across the two topics within the English language arts exam (reading, as well as language and literature). Cohodes also conducts a few variations of the analysis, including one that assigns test items to their relevant academic standard and then determines how frequently the standard appears on exams. She finds that charter students make gains of similar magnitude regardless of how rare or common the tested standard is. Finally, the analysis considers whether math and ELA results are different than those for science, a lower-stakes test (the gains are of comparable size).

The study cannot prove that test preparation is not occurring at all in these charter schools—the teachers could be very effective at preparing students along the entire spectrum of assessed topics. But it does appear that high-performing charters are not inappropriately gaming the test by focusing on a narrow set of frequently tested items at the expense of others. Perhaps this study can defuse some of the finger pointing aimed at high-performing charters and refocus our attention on learning how the finest charter schools “teach to the student.”

SOURCE: Sarah Cohodes, “Teaching to the Student: Charter School Effectiveness in Spite of Perverse Incentives,” Education Finance and Policy (Winter 2016).

Under President Obama’s stewardship, initiatives to expand access to high-quality early childhood programs have sparked heated political debate. Aiming to ground policy makers and education leaders in this conversation, a recent report from the American Enterprise Institute examines the effectiveness of early childhood education by analyzing and summarizing studies of the country’s ten best-known pre-K programs. It finds that high-quality pre-K works for some students, but the research is inconclusive as to whether it’s beneficial for all.

The report starts with an overview of the four most common research methodologies used to evaluate pre-K programs. These include assessing a program’s long-term impact with Randomized Control Trials, i.e., randomly assigning students to either a program (treatment) or non-program (control) group to measure differences in outcomes; comparing results for participating pre-K students against those for children who were eligible for pre-K but did not enroll; comparing results for participating students with a comparison group based on observable characteristics; and comparing outcomes for pre-K participants before and after the program.

One of the most positive takeaways from the research is that low-income children reap short-term and long-term benefits from high-quality pre-K programs. In Boston’s program, for example, students eligible for free or reduced-price lunch showed larger gains in math and cognitive skills when entering kindergarten. Low-income, high-risk children who attended the University of North Carolina-Chapel Hill’s Abecedarian Project were three times more likely to have earned a college degree—and 42 percent more likely to be employed full-time—than those who did not. And researchers focusing on the Perry Preschool Program in Ypsilanti, Michigan, noticed positive impacts on educational attainment and employment for low-income students. At age twenty-seven, male and female graduates were at least 30 percent more likely to receive their high school diploma or GED.

Yet not all the news was good. A few programs did not result in sustained academic benefits for students. For example, three- and four-year-olds enrolled in Head Start performed no better in reading and math than those in the control group by the end of third grade. And Tennessee Voluntary Pre-K students scored slightly lower than control students in achievement measures at the end of second grade.

The authors are hesitant to draw overarching conclusions from these studies because of methodological differences and shortcomings, as well as variations in enrollment size, design, and operation of the programs. Instead, the authors conclude that we ought to strengthen existing research methods before we base policy decisions on studies’ findings—at least when states and districts are considering “pre-K for all.”

When it comes to disadvantaged youth, however, valid research clearly finds that poor and minority boys and girls can benefit immensely from access to high-quality early childhood education. Thus, states and districts should invest—or continue investing—in programs that directly target these kids. In the words of Georgetown Professor William Gormley, pre-K programs can function “like the lead-off hitter in baseball. The role of the lead-off hitter is to get on base and lay a foundation for what everyone else on the team does. That's what a good early childhood education program can do.”

SOURCE: Katherine B. Stevens and Elizabeth English. “Does Pre-K Work? The Research on Ten Early Childhood Programs—And What It Tells Us,” American Enterprise Institute (April 2016).