The Education Gadfly Weekly

In Enrollment and Achievement in Ohio’s Virtual Charter Schools, the Fordham Institute takes a robust, nuanced look at Ohio’s substantial number of e-schools (more commonly known as full-time virtual charter schools). The report paints a troubling picture of these schools’ performance and offers valuable policy recommendations for driving K–12 online learning toward a better future. But readers should note that it also suffers from four significant limitations that are shared by most other studies of virtual charters. This is not the analysts’ fault; it’s intrinsic to the data that are currently available to measure the outcomes of e-school students.

First, the study does not control for course-taking patterns. It usefully reports that students in Ohio’s full-time virtual schools are “more likely to enroll in basic and remedial math courses” than students in brick-and-mortar schools. It would seem plausible—though the authors don’t mention it—that this might have an adverse impact on pupil achievement as gauged by state tests. Determining why so many more students take lower-level math courses in the virtual environment is an important next step to build on this research.

Second, the state tests given in and before 2013—those whose results were used in the Fordham report—exhibit a serious deficiency: They tested what students should know in a given grade. Computer-adaptive tests that assess above- and below-grade-level material more accurately depict what a student does know and can do, but they aren’t used enough (nor do state reporting systems often capture the granular detail they can provide). To grasp the difference, imagine a pupil who enters the fifth grade but has only mastered what the state says a second grader should know in math. His school can choose to personalize his curriculum by focusing primarily on third-grade math competencies, but doing so means that he will likely fail to attain proficiency on the end-of-year fifth-grade assessment.

Alternatively, a school could decide to simply aim for the test by focusing primarily on the fifth-grade knowledge and skills that a student will need to pass it. This could rob him of a solid foundation in mathematics and pose problems in the future, even if it boosts his odds of passing the fifth-grade test.

Online schools are ideally equipped to personalize learning according to each student’s distinct needs. If an eleven-year-old needs to spend time mastering second-grade math, she can. This is much more difficult in traditional brick-and-mortar schools with whole-class teaching models. But if Ohio virtual schools are indeed personalizing students’ learning, the grade-level tests used to measure their growth won’t be accurate. It’s time for a more robust system of assessments that measures individual student growth—whether at, above, or below grade level.

Third, the report does not investigate whether (as seems likely) certain e-schools perform better than others. Ohio Connections Academy, for example, roughly meets or exceeds Ohio’s average statewide performance on 80 percent of the state assessments. (Its big hole is in math, which has caused the school to redo most of its curriculum for that subject). On these blunt assessments, Ohio Connections Academy outperforms some of the largest e-schools in Ohio (by a significant margin in some cases).

This suggests two recommendations to add to those proffered by Fordham. First, authorizers of e-schools must be steadfast in shutting down the poorly performing ones, just as authorizers of brick-and-mortar charter schools must be vigilant in maintaining their quality. Second, the state department of education should study what practices are enabling providers like Ohio Connections Academy to achieve superior results—and offer its findings as guidance to school operators and authorizers.  

Fourth, although controlling for the demographic and prior achievement variables that are measured at the state level does provide us a window into schools’ average quality, it doesn’t tell us why a student enrolled in a full-time virtual school in the first place. It consequently does not allow researchers, including the authors of the Fordham report, to actually compare like situations and results. For example, a significant percentage of students enroll in e-schools because of a medical issue or bullying, and may thus be using their schools to help escape an unworkable situation—with little regard to what that decision means for the future. In such cases, academic considerations are secondary at best. Understanding this dynamic is critical to making valid comparisons and defining student success.

***

Despite these limitations, the Fordham report does some things really well. It controls for demographics and prior achievement (including whether a student repeated a grade) more than any other study of virtual schools to date. By looking only at students enrolled in e-schools for two consecutive testing periods, it potentially undercuts one popular counterargument from those at the full-time virtual schools: that students enrolled for a short time may not do well, whereas those enrolled for multiple years perform much better. Its policy recommendations are solid and recognize some of the nuances in the full-time virtual school landscape. Instead of calling for a ban on such schooling—which would rob myriad students of education options that have been critical to their lives and their academic success—it presents constructive ideas for improving e-schools, and online learning more generally.

As report author June Ahn says, “Though the age of online learning has dawned…there is much room for improvement as far as online schooling goes.” That would seem to apply not only to the full-time virtual schools themselves, but to policy makers, school authorizers, researchers, and psychometricians as well.

Michael B. Horn is a co-founder of the Clayton Christensen Institute.

The surprising best seller Hillbilly Elegy: A Memoir of a Family and Culture in Crisis has become something of a cause célèbre on the grounds that it explains the appeal of Donald Trump to the white underclass (from which author J.D. Vance emerged). Writing in the American Conservative, Rod Dreher aptly notes that the book "does for poor white people what Ta-Nehisi Coates's book did for poor black people: give them voice and presence in the public square."

The book should also be required reading among those of us in education policy. It reminds us of the roles that institutions play (and fail to play) in the lives of our young people, and further suggests that education reform cannot be an exclusively race-based movement if its goal is to arrest generational poverty. Poverty is a "family tradition" among Vance's people, white Americans of Scots-Irish descent who were once "day laborers in the Southern slave economy, sharecroppers after that, coal miners after that, and machinists and millworkers during more recent times."

Vance emerges as something of an emissary to elite America from Fishtown, the fictional composite of lower-class white America that Charles Murray described in his 2012 book Coming Apart. This growing segment of American society is marked not just by economic poverty, but also by social and cultural poverty: the decay of bedrock institutions like marriage and organized religion, as well as the erosion of cohesive social standards like the two-parent family. Still, the more apt comparison might be to Random Family, Adrian Nicole LeBlanc's 2003 book about two young women caught up in a suffocating web of destructive relationships, teen pregnancy, drugs, crime, and general dysfunction in the South Bronx.

If the connective tissue between the urban poor and downwardly mobile working-class whites is lost on pundits and policy makers, the same isn’t true of Vance, who describes being deeply struck by William Julius Wilson's book The Truly Disadvantaged. "I wanted to write him a letter and tell him that he had described my home perfectly," Vance writes. "That it had resonated so personally is odd, however, because he wasn't writing about the hillbilly transplants from Appalachia—he was writing about black people in the inner cities." Ditto Charles Murray's Losing Ground, "another book about black folks that could have been written about hillbillies—which addressed the way our government encouraged social decay through the welfare state," he notes.

Watching an episode of The West Wing on television, Vance is struck that "in an entire discussion about why poor kids struggled in school, the emphasis rested entirely on public institutions. As a teacher at my old high school told me recently, 'They want us to be shepherds to these kids. But no one wants to talk about the fact that so many of them are raised by wolves.'" The characterization is unkind, but Vance is unsparing in his analysis of the people he loves and the culture they have created. It can include "an almost religious faith" in hard work and the American dream; yet he describes his town as one "where 30 percent of the young men work less than twenty hours a week, and not a single person [is] aware of his own laziness."

Vance comes from "a world of truly irrational behavior." His family, friends, and neighbors spend their way into poverty. "And when the dust clears—when bankruptcy hits or a family member bails us out of our stupidity—there's nothing left over. Nothing for the kids' college tuition, no investment to grow our wealth, no rainy-day fund if someone loses her job. We know we shouldn't spend like this. Sometimes we beat ourselves up over it, but we do it anyway," he writes. Domestic life is a chaotic mess of failed relationships, drug abuse, and self-sabotage. "We don't study as children, and we don't make our kids study when we're parents," Vance acknowledges. "Our kids perform poorly in school. We might get angry with them, but we never give them the tools—like peace and quiet at home—to succeed." It is only when Vance enjoys a few years of relative stability—living full-time with his "Mamaw" (grandmother), herself a tough, foul-mouthed, and violent character—that he is able to begin to turn his life around.

One must be richly skilled in cherry picking, or else deeply in denial, to see clear public policy solutions to the problems illumined in Hillbilly Elegy. While Vance may see personal behavior rather than policy as exerting a greater influence on life outcomes, public institutions—the Marine Corps and Ohio State University most particularly—played a prominent role in arresting his otherwise inevitable march down the road to nowhere. If Vance's hillbillies' lives are chaotic, their politics are incoherent. "Mamaw's sentiments occupied wildly different parts of the political spectrum," Vance writes, ranging from radical conservative to European-style social democrat depending on her mood or the moment. "Because of this, I initially assumed that Mamaw was an unreformed simpleton and that as soon as she opened her mouth, I might as well close my ears.” Eventually, he perceives wisdom in his grandmother's contradictions: "I began to see the world as Mamaw did. I was scared, confused, angry, and heartbroken. I'd blame large businesses for closing up shop and moving overseas, and then I'd wonder if I might have done the same thing. I'd curse our government for not helping enough, and then I'd wonder if, in its attempts to help, it actually made the problem worse."

If there is any theme that has emerged from the fractious state of our political and civic lives in 2016, it is not how divided we are, but rather how deeply and stubbornly obtuse we are about one another's lives. There is a tendency among refomers to sentimentalize the lives of the poor, or to infuse poverty with a note of tragic heroism. Vance seems aware of this himself, noting in his preface that his object is not to argue that working-class whites "deserve more sympathy than other folks" but that he hopes readers "will be able to take from it an appreciation of how class and family affect the poor without filtering their views through a racial prism."

My first attempt to read LeBlanc’s Random Family failed. The despair it conveyed was bottomless, and it took over a year before I was able to return to it. A similar grimness at times weighs down Hillbilly Elegy. It is only the foreknowledge of how Vance's story ends, with a slot at Yale Law School and a job at a Silicon Valley investment firm, that allowed me to keep turning the pages. But none of this makes his story less essential. I used to assign Random Family to graduate students who were first-year Teach For America corps members; I still view it as required reading for anyone teaching low-income, inner-city children. For education reformers, I would now bookend that recommendation with Vance’s memoir. Both books force us to confront simpleminded views of the ills we seek to address, and to be humble about over-optimistic schemes to set things right. For education reformers, I do not recommend reading Hillbilly Elegy. I recommend studying it.

Editor’s note: This post was originally published in a slightly different form in U.S. News & World Report.

Don Hirsch has done it again. Never mind that he’s eighty-eight. Why Knowledge Matters: Rescuing Our Children from Failed Educational Theories, his fifth book on education reform—there were at least five earlier ones in his original field of English literature, criticism, and composition—is as clear and trenchant as Cultural Literacy was in 1987. And it is arguably even more needed, as there’s ample evidence that the “knowledge” part of K–12 education has been backsliding even as we’ve seen slight improvement on the skills side.

There’s the curricular narrowing associated with our reading-and-math obsession and the accountability regimes attached thereto. There’s the perverse effect of Google and other technologies leading us to assume that we “can always look it up.” And most perniciously—it is the theme of Why Knowledge Matters—there’s what Hirsch terms “the tyranny of three ideas” that steer educators in the wrong direction.

Here, in short form, are the mistaken ideas:

• Early education should be age-appropriate and seen as part of a “natural development process.” (“Early education” in Hirsch’s world isn’t preschool; it’s kindergarten and the first several grades of school.)
• Early education should be individualized as far as possible.
• The main aim of education is to develop critical thinking and other “general skills.”

Wrong, wrong, and wrong, as Hirsch musters ample evidence from psychology and social science to prove. What kids mostly need in the early grades is a common, knowledge-centric curriculum (they’ll learn more, too, since they’ll find it so much more interesting). It’ll boost their reading scores; prepare them to succeed in middle school, high school, and beyond, where U.S. test scores (and other metrics) crash; and equalize opportunity in American society in ways that no anti-poverty or compensatory education program can possibly do.

This assessment isn’t coming from an impartial observer. I’m on the board of Don’s Core Knowledge Foundation and helped launch the Knowledge Matters Campaign. I also lent him a bit of help with this book as a longtime friend and admirer. Ever since Diane Ravitch and I happened upon his seminal 1983 article in the American Scholar and encouraged him to turn it into the book that became Cultural Literacy four years later, I’ve found his informed ideas about what ails American education persuasive and sound—far more than those of just about any other thinker, in fact.

Readers of this book will also be exposed to a riveting and well-documented object lesson as Hirsch recounts the sorry fate of academic achievement in France after that country abandoned its longstanding national curriculum. Read it and see for yourself.

Yes, you will likely be provoked by Don’s heterodox thinking about many of reform’s most prominent standards-and-accountability initiatives; by his earnest advocacy of the kind of standardized state or national curriculum that many politicians assume to be a “third rail” proposition; and by his insistence that for knowledge truly to be acquired, the curriculum has to be systematic and cumulative rather than a random assemblage of “units.” Though he sees potential in the “commonness” of the Common Core, he argues that true learning gains will only follow if states turn more demanding ELA standards into a mandate for knowledge-rich curricula and if test makers understand that “close reading” of texts could make things worse unless those texts are integrated with such a curriculum. Otherwise, they will lead to even more class time given to counterproductive efforts at skill building rather than inculcating knowledge. Moreover, kids will be even less prepared than they are today to handle the demands of the upper grades and thereafter.

I don’t know whether this will turn out to be Don’s last book. Informed as it is, however, by almost four decades of experience and research on these issues, it’s probably his best. 

On this week’s podcast, Alyssa Schwenk, Brandon Wright, and David Griffith discuss alternative teacher licensing in Utah and opt-out consequences in Florida. During the research minute, Amber Northern examines the lack of college readiness in Baltimore.

Amber's Research Minute

Rachel E. Durham, "Stocks in the Future: An Examination of Participant Outcomes in 2014-15," Baltimore Education Research Consortium (August 2016).

A new Mathematica study examines whether school-level value-added measures adequately capture principals’ effectiveness. Many districts hold them accountable for their schools’ academic performance; this study probes that assumption by asking an important question: Does school-level value added actually reflect the principal’s contribution, or does it mostly reflect other school-level influences (such as neighborhood safety) that are outside the principal’s control?

The authors use longitudinal data from the Pennsylvania Department of Education to study school and principal effectiveness for grades 4–8 from 2007–08 to 2012–13. They include in the data set principals who have been involved in a leadership transition—meaning that, during the analysis period, they started leading a school they had not led before or were replaced by incoming principals. The authors compare departing principals with successors who assumed their positions during 2009–10 to 2012–13. (Alarmingly, 41 percent of schools serving students between the fourth and eighth grades experienced such leadership changes during the study window.) To disentangle the principal’s contribution to growth from the effect of other school-level factors, they sought to isolate the portion of the principal’s impact that is consistent across time and across different samples of students—i.e., the effects on student achievement that principals persistently demonstrate.

Here’s the bottom line: School-level value added is a poor proxy for showing how principals in particular impact student achievement growth over time. (More specifically, no more than 7 percent of any given difference in value-added between two schools reflects persistent differences in the effectiveness of their current principals.) Of course this does not mean that all principals are alike, since they do vary in their value added. In fact, in this sample, the standard deviation of principal effects is at least 80 percent of the size of the standard deviation of teacher effects as estimated by Eric Hanushek in prior studies. What this study is saying is that very little of this substantial variation can be predicted by school-level value added. The likely explanation is that the latter reflects a combination of influences on student achievement outside the principal’s control—and perhaps on other things that principals do not consistently demonstrate from one year to the next.

So, yes, principals matter. The hard part is nailing down exactly how and why.

SOURCE: Hanley Chiang, Stephen Lipscomb, and Brian Gill, "Is School Value Added Indicative of Principal Quality?," Mathematica (June 2016).

In Educational Entrepreneurship Today, edited by Frederick M. Hess and Michael Q. McShane, a gaggle of authors examines how entrepreneurship can fuel the engine of educational innovation. The authors paint a complex portrait of risk, reward, and regulation.

The book defines educational entrepreneurship as “risk-taking behavior intended to boost school productivity or offer new services in a manner that makes a lasting difference for students.” The authors remind us that the very premise of entrepreneurship is novel within education. Typical initiatives in this realm are-risk averse because failure may harm children. Yet recent years have provided plenty of examples of entrepreneurial effort.

One theme throughout the book is that the structure of organizations and initiatives matter, although the authors differ on what structure is best. Some favor small, precisely targeted programs like the Tiny School Project, which focuses on testing educational ideas on a micro level. Others focus on scaling successful initiatives, such as the KIPP charter network’s growth from a single classroom to over two hundred schools across the country.

Entrepreneurial ventures like Teach For America, TNTP, the Broad Residency, and New Leaders for New Schools have both grown and become pipelines for educational talent to undertake yet more initiatives. The book notes a 2011 study showing that 15 percent of entrepreneurial organizations in education are led by TFA alumni.

Politics and policy understandably loom large in these pages. Governmental efforts such as No Child Left Behind, Race to The Top, the Common Core State Standards, and the Every Student Succeeds Act have all triggered the launch of firms and leaders that seek to meet needs in areas such as curriculum, testing, and teacher evaluation.

Money matters as well. Philanthropy and venture capital have given entrepreneurs and schools alternative sources of funding beyond what government supplies. In 2014, for example, educational technology companies raised $1.36 billion—a 212 percent jump from 2009. The authors argue that this influx of capital has fostered innovation and creativity within the education industry.

The authors are optimistic about the role and future of entrepreneurship in education, confident that when it is supplied with talent, investments, and drive, the free market has the capacity to improve American education and create a brighter tomorrow for students.

SOURCE: Frederick M. Hess and Michael Q. McShane, eds., Educational Entrepreneurship Today (Cambridge, MA: Harvard University Education Press, 2016).