The Education Gadfly Weekly

“Twenty-five years ago, there was no proof that something else worked. Well, now we know what works. We know that it’s just a lie that disadvantaged kids can’t learn. We know that if you apply the right accountability standards you can get fabulous results. So why would we do something else?” —Jonathan Alter, “Waiting For Superman” (2010).

When I sat in a Manhattan theatre watching Waiting For Superman a decade ago, fresh from the classroom and new to the world of policy and education reform, the haughtiness and arrogance implicit in Alter’s words made me bristle. A decade later, it’s almost poignant. Did we actually believe ten years ago that America’s education problems were a simple matter of getting “accountability standards” right? That “we know what works”? Were we ever that naïve? I saw the movie with a friend who was not in education and found the film persuasive, even moving. My own response, which has since become my mantra, was “Well, it’s more complicated than all that.”

Ed reform circa 2010 was riding a cresting wave, but in retrospect it was the high-water mark. When the decade began, reform was playing offense, energetically aligned against an intransigent blob, armed with moral authority, bracing and seemingly durable bipartisan support, and a strong (if naïve) faith in the capacity of higher standards to yield broad student gains, a muscular testing and accountability regime, and enhanced choice, mostly in the form of charter schools. And all of it supported by a vast army of philanthropists and advocacy groups.

Ten years later, nearly all of this has been reversed or is in retreat. Big reform is dead. As the decade draws to a close, it takes an extraordinary level of self-deception to believe that the roots of education’s persistent and frustrating malaise are merely structural, that there are vast reserves of untapped capacity or competence in K–12 education awaiting marching orders, or that technocrats far removed from the daily work of teaching and learning hold the keys to unlocking it. This was foreseeable and foreseen. “While public policy can make people do things, it cannot make people do those things well,” wrote Rick Hess in National Affairs back in 2014, the year, not incidentally, that every American student was going to be reading and doing math proficiently by federal law. Policymakers do not run schools, Hess concluded, “they merely write laws and regulations telling school districts what principals and teachers ought to do,” and noting with understatement “there is often a vast distance between policy and practice.”

More recently, Stanford University political scientist Terry M. Moe offered an equally dour view of the inevitable failures of structural reform in his new book The Politics of Institutional Reform. Intransigent vested interests, including teachers unions, school boards, superintendents, and administrators, have no incentive to reform and reliably block it at every turn. “And so here we are,” Moe said recently on Russ Roberts’s podcast EconTalk. “It's 2019. How many kids are in charter schools? About 6 percent nationwide. How many kids have vouchers or use tax credits? Less than 1 percent nationwide.” And accountability? It’s “run into a buzzsaw with the unions hating accountability and being threatened by it,” he explained. Meanwhile Republicans are “getting back to their local government roots and getting all teary eyed about how important it is to have state and local governments run everything.” That means returning accountability to state and local governments to die. “So I think here we are, after all this time, and we've made very, very little progress,” he said.

Moe’s sobering analysis understates the intransigence of the problem. Even where barriers to reform have been limited, the gains wrung from the reform playbook have still been modest. “We all know people that say some version of, ‘Well, we know what to do, we just don't have the will to do it,’” said Mike Goldstein, a self-described “humbled technocrat” and founder of Boston’s acclaimed Match Charter schools, in an interview. “But a lot of times our best examples of gains are shown in careful studies to be much smaller than what we had hoped or thought might be possible.”

Echoing Goldstein is City Fund’s Neerav Kingsland, formerly the head of New Schools for New Orleans, who wrote on his blog this week that “doubling or tripling the postsecondary success rate for low-income students is no small feat. This could help change the lives of millions of students. But this is far below the results I hoped for when I got into this work. I imagine many of my peers feel the same. And I am sure families want more for their children.”

Looking back not just over the past decade but to the dawn of the reform era circa 1983 and A Nation at Risk, it is nearly inconceivable that anyone would view where things stand today as a victory—or even acceptable progress for thirty-five years of effort. The logic of Big Reform and its focus on testing, standards, and accountability assumed that schools and teachers know what to do and that technocratic engineering could deliver a sufficient shock to the system to make them do it. It hasn’t worked. And let me preview my 2029 ed reform retrospective: If the past decade has convinced you that the real problem in American education is institutional racism, segregation, and the broad array of social ills that have diverted education reform’s attention from its core business of teaching and learning, and that those things need to be fixed first, gird your loins for another frustrating decade.

Here’s some holiday heresy as the decade draws to a close: One lesson of the past decade may be that if reducing poverty is your public policy goal, education might not be your primary strategy. Your efforts might be better focused incentivizing work, strengthening families, shoring up personal income, or on housing and health care policy. “Education is life changing for scores of young people, and is a critical policy lever. But at scale, we’ve often approached education as the lever, the Holy Grail for poverty alleviation. The gains, even in the best of circumstances, haven't match our ambitions at the outset,” Paymon Rouhanifard, the former superintendent of schools in Camden, New Jersey, said in an interview. “This speaks to the complexity of poverty.”

In the zero-sum game of education policy debate, much of the above will be read as an indictment, even surrender—an argument that the battle is over and that ed reformers should fold their tents and go home. It’s not, for the reason suggested by Rouhanifard. There have been encouraging developments, notably networks of high-achieving urban charter schools serving tens of thousands of low-income black and brown children whose lives would be poorer without them. They are success stories that ed reform should defend (particularly as Democrats have all but abandoned charter schools and the parents who rely on them), but even the brightest stars in the charter school world are not compensation enough for the broader lack of gains. So much reform, so little to show for it.

Education reform’s reach has exceeded its grasp, but that’s not an invitation to sentimentalize a golden pre-reform era that never existed, nor to throw up one’s hands and quit the field of battle. Moreover, there’s a difference between a cynical anti-reform, just-send-more-money argument, and a thoughtful stance that is clear-eyed and humble, that holds on to its aspirations while recognizing its limitations. We don’t have permission either as educators, policymakers, or citizens to write off boxcar numbers of American children. But neither should we indulge ourselves, digging in our heels and continuing to advocate for policies that we know aren’t effective simply because we don’t want to show weakness publicly.

So where does this leave ed reformers other than chastened and humbled? If the decade to come is to be different than the one now drawing to close, it suggests more modest expectations, situating reform’s center of gravity closer to the ground (read: focusing on practice improvements, not mere structural reform) and in enhanced school choice with parental satisfaction at least as important a measure of success as test scores. Elsewhere in this issue, my optimistic colleague Dale Chu argues that the decade now drawing to a close, for all its Sturm und Drang, has been a good one for choice. He looks at the same sobering data Terry Moe cited above and sees one million more students in charter schools than in 2010, and an additional 400,000 participating in private school scholarships. That’s a long way from scale, or a rising tide lifting every child’s boat, but it’s not nothing—particularly for the low-income, black, and brown families who are its primary beneficiaries. I’m inclined to think that along with a focus on improved curriculum and pedagogical practice, defending and growing school choice in all its forms—not just the ones technocrats encourage and favor—is a more modest and appropriate focus of reform energy in the next decade.

And perhaps as good as it gets.

As 2019 comes to an end, so does a significant decade for education reform. For all the Sturm und Drang over the lack of improvement, the education policy landscape is no longer the same. Granted, the first half of the 2010s looked decidedly different than the latter half, but states have largely stuck with higher standards, and the honesty gap in the reporting of student performance continues to close. It’s all easy to lose sight of in light of today’s fraught national politics and intramural food fights, which tend to shed more heat than light on what’s transpired in America’s fifty laboratories of innovation.

Consider one of my favorite labs, situated at the nation’s crossroads: Indiana. I’ve spilled gallons of ink on my former home, so it’s only fitting that my take on the decade is through the lens of the Hoosier State. In 2010, I was an official at the Indiana Department of Education during the state’s halcyon days under the leadership of Governor Mitch Daniels and State Superintendent Tony Bennett. In a short period (i.e., the first third of the decade), we successfully pushed through a dizzying phalanx of education reforms enabled by a unique alignment of political stars. Key among these were improvements to third-grade reading, teacher quality, collective bargaining, school choice, and accountability.

After our efforts were halted by politics, progress on some of these reforms stalled. But there’s one policy area where the results have been an unqualified success: school choice—especially within Indiana’s traditional public schools (i.e., interdistrict transfers). While the state’s private school voucher program—which we helped establish in 2011—garners the bulk of local and national headlines, the majority of Indiana’s students switch schools using a different system: open enrollment. In fact, the number of Hoosier families exercising the power of school choice is far more than what public policy debates in Indiana would suggest. Nearly 60,000 students (see the figure below from my recent paper), or about 6 percent of students enrolled in public schools, “choice” into a traditional public school other than the one assigned to them based on their address—a population larger than total charter school or voucher enrollment in the state.

Figure 1. Indiana school choice transfers, 2006–2019

What this means is that in spite of the turnover at the top (i.e., three governors and three state chiefs in ten years) and ongoing political turmoil, more Indiana parents are taking the education of their children into their own hands and are none the worse for wear. At the beginning of the decade, about 25,000 students statewide were able to exercise some form of choice; as we close out 2019, that number is closer to 135,000 students—a whopping increase of over 400 percent! By any responsible measure, this is a victory for education consumers and for educational freedom. More importantly, school choice in Indiana is permanent now that families have it. You cannot unring that bell.

Nationally, the choice die has been similarly cast. In 2010, less than two million students were enrolled in charter schools nationwide; today that number is over three million. Likewise, there were about one million students participating in private school choice programs (i.e., education savings accounts, school vouchers, tax-credit scholarships, and individual tax credits and deductions) at the decade’s onset; in 2019, that number is closer to 1.4 million. Both sectors have experienced impressive growth in opportunity.

This is not to say that any of us could have correctly guessed today’s circumstances back in 2010. To be honest, my money would have been on charter schools outpacing the field after we lifted the state’s cap on them. I would have also predicted Mitch Daniels sitting in the Oval Office, with Tony taking his talents to the national stage. None of this panned out, but Mitch was still elected president and we still ended up with an education secretary who has no qualms about taking on the establishment heads on. History unfolds in unpredictable ways.

Turning the page on this chapter, more changes are in store. Beginning in 2021, Indiana’s governor will appoint the state schools chief for the very first time. This brings the possibility of alignment between the two offices after years of being at loggerheads. Only time will tell whether the disorienting effects can be put in the rearview mirror, but the expansion of parental options seems destined to continue. While the pace of growth does show some signs of abating for charters and vouchers, no such trend line can be observed for the state’s open enrollment policy. Yes, student performance has plateaued, and in this sense Indiana is no longer a “reform idol.” But when it comes to providing parents with a panoply of schooling options, Hoosiers continue to be worth emulating.

At the beginning of the modern ed-reform movement, getting onto four decades ago, urban Catholic schools were everywhere, serving as vital proof points in the debate about what was possible. While too many traditional public schools serving disadvantaged communities were either unsafe, failed to produce graduates with even basic skills, or both, urban Catholic schools stood apart. They inspired a wave of reform until, perhaps ironically, the reforms they inspired eventually nudged them out of the spotlight—that plus, of course, the country’s endless hand-wringing over private and religious schools.

Today, due to a combination of increased pressure from charters, fiscal woes, demographic shifts, and in many cases, failure to adapt to the brave new world of accountability and choice, more than 1,200 Catholic schools have closed. And that’s just in the past decade. It also needs to be said that news of the strong results most Catholic schools have long achieved with disadvantaged youngsters has been eclipsed by the impressive outcomes of the nation’s highest performing charter schools and networks.

Notably, those charters have achieved their results in part by borrowing some of the easiest-to-replicate practices of urban Catholic schools. Yet we also see mounting evidence that those commendable charter results are not yet translating into the long-term outcomes we all hope for the students we serve. That’s why, as I argue in a recent Manhattan Institute report, it’s time to take another look at Catholic schools and see whether, while copying their most visible practices—the discipline, the uniforms, the structure—ed reformers may have missed other elements of Catholic schooling that are key to long-term success.

Even top-performing charter networks are struggling to translate their strong test-score gains into college persistence and graduation. Perhaps no charter network has faced this reality more directly and openly than the KIPP network, which in 2011 released a “College Completion Report” that found that just “33 percent of students who completed a KIPP middle school ten or more years ago have graduated from a four-year college.” Those results fell far short of the network’s aspirations to help 75 percent of its graduates earn four-year degrees.

In response, KIPP made a number of programmatic shifts, including KIPP Through College and a “character counts” program intended to develop the values and skills that young people need in order to succeed in college and beyond. Yet earlier this year, Mathematica released a new appraisal of KIPP’s college completion results, noting that, despite its heroic efforts and hefty investments, the network is still struggling to achieve its college completion goals. While attending a KIPP charter school did have a positive impact on college enrollment, it did not have a statistically significant impact on college persistence: Just 33 percent of KIPP graduates persisted through the first four semesters of college, compared with 26 percent in the control group.

Reflecting on these and other results, earlier this week longtime charter advocate and City Fund founder Neerav Kingsland took stock of where the charter-centric approach to ed reform stands today. “The results coming out of the best charter organizations, and cities as a whole,” he wrote, “are sobering…we underestimated how stacked the world is against kids growing up in poverty.... We should admit that we failed to live up to our expectations.”

As leading charter advocate take an honest and hard look at their work, many reformers are also taking this moment to explore what else we can glean from Catholic schools, which have demonstrated more durable results. They’re looking beyond what’s easy to replicate and examining fundamental questions about the values and beliefs that may be foundational to the long-term impacts that Catholic schools have had.

That quest must inevitably lead them to the beliefs and values that form the foundation of Catholic education, for Catholic school success is not simply the product of rigorous curriculum, structure, and order. It’s also the result of a school culture animated by the belief that every child is made in the image and likeness of God and focused on drawing out of every pupil their own God-given potential.

Careful observers will also see that Catholic schools’ academic results are not actually the main goal; rather, they’re byproducts of schools that focus on forming young people not just with the skills that will make them financially successful, but also with the habits of virtue and values that will make them choose to do good and contribute to their communities.

After all, the evidence suggests that Catholic schools continue to demonstrate strong long-term outcomes, even as they struggle financially and struggle to compete for student enrollment and for solid state test scores. For example, girls who attended Catholic schools are more likely to avoid early pregnancy and boys are less likely to face incarceration. Research has also shown that Catholic school graduates are more likely to be civically engaged, to vote, to volunteer, and to give to charitable causes than their public-school peers.

Put more simply: When it comes to long-term life outcomes, students from high-performing charter schools seem to underperform relative to their strong test score results, whereas urban Catholic schools tend to overperform relative to their test scores. And at the core of these successes is the way Catholic schools think about their students and why Catholic educators do this work.

The goal of Catholic schools is not to secure top marks in a school rating system but to form students who opt to serve, who choose right over wrong, and who contribute to their communities. The goal of Catholic schools is not discipline and structure merely for the sake of order but for the sake of building the habits of self-management that lead to virtuous lives.

As ed reformers seek paths forward to the life-changing results they want for students, Catholic schools offer the simple but challenging lesson that purpose must be anchored to something larger and more enduring than individual-level achievement. As we work together to drive continued improvement and systemic change, it’s worth looking beyond easy-to-replicate programs and practices to the foundational values and beliefs that help schools build the habits and skills that serve young people long after they graduate.

According to a recent Hechinger Report article, U.S. schools overlook a staggering 3.6 million gifted students, many of whom come from less advantaged backgrounds. By failing to identify these bright youngsters—much less provide them with special services—the nation leaves vast “unrealized potential” on the table. New research on a Boston acceleration program reveals just how important it is for schools to serve these high-achieving students.

Conducted by Columbia University researcher Sarah Cohodes, this study examines the impacts of Boston Public Schools’ Advanced Work Class (AWC). The program offers qualifying students an opportunity to learn in classrooms dedicated to advanced instruction throughout grades four through six. Program eligibility is based on a third grade normed-referenced test, with approximately 10 to 15 percent of Boston students meeting the test-score cut off. Although AWC participants tend to come from more advantaged backgrounds than their peers districtwide, nearly half of them are black or Hispanic, and two in three are eligible for subsidized meals. Most students who accept the AWC offer in fourth grade stay in the program through sixth grade—on average, students participate for 2.7 years.

To estimate the effects of AWC participation, Cohodes uses a “regression discontinuity” statistical method that in general compares students just above and below the eligibility cut-off. The basic idea is that these students are essentially identical, and by almost random draw, they either qualify for the program or not. In terms of outcomes, she examines state exam results, AP test-taking, SAT participation and scores, high school graduation, and college enrollment. Because the longer-run outcomes require a wider time frame, the analysis focuses on cohorts whose students were in third grade between 2001 and 2005.

Interesting findings emerge, especially for students of color. On state tests scores, black and Hispanic participants perform at slightly higher levels than the control group, though their gains are not quite statistically significant. The results for AP test-taking are likewise positive, though again not significant. More impressive results surface when examining SAT participation and high-school graduation. Among black and Hispanic AWC participants, 86 percent take the SAT compared to 74 percent of their peers (though their test scores are just marginally higher). Meanwhile, students of color post an 87 percent on-time graduation rate versus 67 percent for the control group, a large and statistically significant gain. The program’s impact is more muted, however, for Asian and white students, with largely null findings for those groups.

Perhaps the most important finding is that AWC boosts college enrollment among students of color. Of AWC’s black and Hispanic participants, 66 percent enroll in college right after high school, while just 40 percent of the control group do so. The study also finds that AWC leads to attendance at higher-quality colleges (as measured by graduates’ earnings), and the increase in college enrollment is being driven largely by matriculation at four-year universities. Yet akin to the K–12 outcomes, the impacts of AWC on college-going among Asian and white students are insignificant. The advantages in college enrollment for black and Hispanic participants are somewhat predictable, given that they’re more likely to graduate high school on-time. These effects “do not occur in isolation,” Cohodes writes. “Instead they rely on sequential increases in meeting milestones along the way.” 

What mechanisms explain these results? Supplemental analyses explore whether exposure to higher-performing teachers and classmates are behind the gains for black and Hispanics. Her analyses rule out both as the primary drivers of these gains. Overall, the study concludes: “It appears that AWC is the beginning of a chain of events that causes participants to stay on-track for college throughout high school…. These small gains may build upon each other, with AWC providing the crucial ‘foot-in-the-door’ that begins a chain of positively reinforcing events.” Indeed, schools should strive to ensure that high flyers are identified and offered challenging coursework that enable them to “maintain their altitude.” Taking a look at the successful Boston model would be a good first step.

Source: Sarah R. Cohodes, “The Long-Run Impacts of Specialized Programming for High-Achieving Students,” American Economic Journal: Economic Policy (forthcoming). An open-access version is available here.

Online courses have produced mixed results. They can be good tools for motivated students. But many struggling students use online courses to gain course credit without realizing they aren’t preparing them for college. These negative, unintended consequences may be caused by reduced interaction with teachers for enrolled students. With this in mind, Chad Turley and Charles Graham of Brigham Young University published a recent case study in the Journal of Online Learning Research on the impact of student-teacher interaction on student satisfaction, perception of the course, and completion time for online courses.

The study used an unnamed nonprofit in the western United States sponsored by private, denominationally-affiliated universities with 100,000 annual enrollees that offers 550 online courses from the middle school to college level. They chose two secondary-level courses: math and English language arts, which each offered two models of online education. Model One was more self-directed and had a higher enrollment, while Model Two required teachers to contact five to seven students a week with personalized emails and teach some lessons synchronously with live feedback. Furthermore, the nonprofit sent the researchers student rating data from a twelve-question survey administered at the end of the course to gauge participants’ satisfaction. It used a 1–8 scale, with 8 being the most positive, and had three yes or no questions. Turley and Graham also compiled teacher communication data using a log with eight questions that instructors filled out regarding each interaction with participants. It covered the reason for the interaction, time taken to respond to students, who initiated, and the medium used. The total number of students in the courses were 1,025, and 764 completed the end-of-course survey.

Turley and Graham then analyzed the data to find differences in student satisfaction ratings, completion rates, and teacher time investment between course models and subjects.

In both subjects, participants rated Model Two courses as having more timely and meaningful instructor feedback. And in English, students were more likely to be satisfied with their experience and more likely to recommend the course to a friend.

On completion rates and teacher time investment, Model Two students were 5 percent more likely to complete the course in math and 3 percent more likely in English. But they took longer to complete coursework: five more weeks in math; two more in English. Turley and Graham note that this is probably due to the types of assignments.

Teacher communication in both models was almost entirely through personalized emails, with a few video conference calls and none via phone call. For Model Two, teachers averaged 14.9 hours over four months responding to student emails, but Model One teachers averaged only 2.5 hours. And Model Two teacher-initiated time investments were around 99.1 hours. A large majority of those communications went towards contacting inactive students, while 45.5 and 36.3 percent of student-initiated ones were about grading and course content, respectively. The researchers also note that Model Two teachers weren’t paid more for their extra time.

Turley and Graham’s study makes a good case that some students can work independently at a faster pace, but that teachers make a difference for most students, especially in English courses. Moreover, it presents an important caveat for teachers that their time investment in online courses can end up going towards non-essentials and not be sufficiently compensated. Future studies could shed more light on this issue by looking at student motivation and performance, instead of merely completion.

SOURCE: Chad Turley and Charles Graham, “Interaction, Student Satisfaction, and Teacher Time Investment in Online High School Courses,” Journal of Online Learning Research (2019).

On this week’s podcast, Daniel Showalter, associate professor of math at Eastern Mennonite University and author of Why Rural Matters, joins Mike Petrilli and Victoria McDougald to talk about the many challenges facing rural schools in America. On the Research Minute, Amber Northern examines how local labor market downturns affect college-going.

Amber's Research Minute

Andrew Foote and Michel Grosz, “The Effect of Local Labor Market Downturns on Postsecondary Enrollment and Program Choice,” Education Finance and Policy (March 19, 2019).