The Education Gadfly Weekly

As I’ve embarked on my weeks-long discussion of how to usher in a Golden Age of Educational Practice, I have heard—often on Twitter, sometimes via email—a clear and compelling message:

FOR THE LOVE OF GOD, MIKE, DO NOT TURN THIS CALL FOR EVIDENCE BASED PRACTICES INTO ANOTHER EXCUSE FOR SO-CALLED EXPERTS TO TELL TEACHERS WHAT TO DO, OR TO FOIST YOUR OWN PREFERRED PRACTICES UPON THE NATION’S SCHOOLS. SHOW SOME HUMILITY, MAN.

To which I say: I hear you, my friends, I really do. And I whole-heartedly agree that we need to approach the topic of evidence-based practice with an enormous amount of humility.

That’s largely because of what Dylan Wiliam likes to say: “Everything works somewhere; nothing works everywhere.” He’s right, of course—the contexts of our schools really do vary dramatically, making the use of evidence an inherently complex and fraught challenge. Plus, in a field where implementation is everything, the only way “doing what works” can be effective is with teacher buy-in and engagement. They call it “winning hearts and minds” for a reason; we can’t expect that evidence alone will win the day.

But perhaps the strongest argument for humility here is because the evidence around educational effectiveness is extremely limited. The number of areas where we have strong science to guide classroom practice is tiny. In fact, it may be sui generis: early reading. There we do have a scientific consensus, or close to it, around “what works,” and in my view it’s irresponsible for us not to ensure that educators are well-equipped to do it.

But that’s a rare case. Much more common are parts of the curriculum and the educational experience where we hardly have any scientific evidence at all. Most of teaching and learning in high school, I’d posit, fits into this box. What’s the “best” way to teach U.S. history? Civics? Biology? Welding? If there are rigorous studies on these topics, that’s news to me. Instead, the best we can currently hope for is to get great educators together—in person, or online—to share “best practices” with one another, and figure it out as well as they can. The Advanced Placement program is an exemplar in this regard.

And then there are practices that fit somewhere in the middle—where we don’t have rock solid evidence like we do for early reading but we do have some strong studies giving us hints at where teaching and learning should go. I’d put E.D. Hirsch Jr.’s decades-long insight about reading comprehension here—that the way to help poor kids especially get better at reading is to start teaching them history and geography and science and the arts much earlier, as systematically as possible, so they have the content knowledge and vocabulary to make sense of what they are reading. Likewise the promising work on instructional coaching, or the importance of same-race teachers for kids of color, or the need for all kids to master their math facts. These point us toward some approaches and away from others, but they surely don’t give us reason to make iron-clad mandates on our schools.

As I continue my series, and start to look at “ways to get schools to use evidence-based practices,” I will strive to keep all of this in mind, and I hope you will too. The limits of what we know—and perhaps what we will ever be able to know—should absolutely inhibit those of us outside of schools from trying to replace educators’ professional judgment with our own.

But let’s also not be paralyzed by this. Telling four million teachers what to do is not the answer, but neither is telling them to close their classroom doors and do whatever they think best. There are lots of ways in between to “get better at getting better.” And we need to embrace them all.

Last week, the first randomized control trial study of “restorative justice” in a major urban district, Pittsburgh Public Schools, was published by the RAND Corporation.

The results were curiously mixed. Suspensions went down in elementary but not middle schools. Teachers reported improved school safety, professional environment, and classroom management ability. But students disagreed. They thought their teachers’ classroom management deteriorated, and that students in class were less respectful and supportive of each other; at a lower confidence interval, they reported bullying and more instructional time lost to disruption. And although restorative justice is billed as a way to fight the “school-to-prison pipeline,” it had no impact on student arrests.

The most troubling thing: There were significant and substantial negative effects on math achievement for middle school students, black students, and students in schools that are predominantly black.

What are we to make of these results? For education journalists like U.S. News and World Report’s Lauren Camera, there’s an easy solution: Don’t report the negative findings and write an article titled “Study Contradicts Betsy DeVos’ Reason for Eliminating School Discipline Guidance.”

When asked why she left her readers in the dark regarding the negative effects on black student achievement, Camera said that it “wasn’t intentional,” explaining that “it wasn’t meant to be a deep dive into the study. And we linked to it, so readers who wanted to follow up could.”

Chalkbeat’s Matt Barnum is somewhat unique among education journalists for his practice of reading academic studies in full before writing about them. Barnum commented, “Well, I will say that the researchers didn’t do any favors in framing the results for reporters. The negative test for effect for black kids is buried on like page eighty with no mention (that I saw) until then…. [T]he research itself is excellent; their choice in framing is…notable.”

Unfortunately, the vast majority of the education policy community will not read this study in sufficient depth to share their disappointment in RAND’s unmistakably slanted editorial emphases. They will read of mostly positive results, and of negative results framed by the RAND researchers as likely attributable to bad implementation of good policy.

It is very sad that the so-called “evidence-based policymaking” community has rendered itself immune to the intellectual breakthrough that enabled the scientific revolution: accepting the falsification of a hypothesis. When it comes to studies of ideologically-preferred policies like restorative justice, the logic all too often is: “Heads, I win. Tails, I would have won if it were implemented correctly.”

The mental itch to label negative results as a product of “bad implementation” rather than a failure is not only anti-scientific, it also short circuits thoughtful policy discussion. Everything beyond empirical data is necessarily the realm of theory, and there are many more interesting and intuitive theories for the failures.

For example, it could be true that the positive effects in elementary school and negative effects in middle school could be reconciled by “bad implementation” in middle schools. Or perhaps the explanation could be that six-year-olds are different than thirteen-year-olds, and that restorative justice works in elementary schools but not middle schools.

It could also be true that the negative results for black students (and for all students in predominantly black schools) could be attributable to bad implementation. Or perhaps restorative justice is uniquely bad for black students. After all, discipline reformers contend that the explanation for the academic and disciplinary racial disparity is teachers’ cultural incompetence when it comes to understanding and relating to black students.

Accepting that premise, the negative results for black students should suggest that restorative justice is an even more culturally incompetent approach.

The researchers conduct an admittedly non-causal empirical dance to support the proposition that the disjunction between teacher and student perception of teacher classroom management is due to non-implementing teachers. That could be true. It could also be true that teachers think that it makes them better, even as students see their classroom climate deteriorating.

For my part, I would privilege the perspective of students over teachers.

Restorative justice is frequently presented to teachers as “evidence-based” and on the cutting edge of “social justice” as something that works if they embrace it. Man’s capacity for self-deception cannot be discounted, and if teachers think they’re doing better even as students think things are getting worse, that would be consistent with the policy drama that has played out writ large over the last two years: In the face of increasingly overwhelming negative evidence, social justice education reformers have only grown more vociferous in their insistence that discipline reform works.  

Right now, the tally of studies on the academic effects of discipline reform on school districts are three negative (Pittsburgh, Los Angeles, Philadelphia) and one null/positive (Chicago). In terms of student surveys, my tally has four negative (NYC, Los Angeles, Washoe County, Seattle) and one negative/positive (Chicago). When it comes to local teacher surveys, I’ve seen eleven negative (Oklahoma City, Baton Rouge, Portland, Jackson, Denver, Syracuse, Santa Ana, Hillsborough, Madison, Charleston, Buffalo) and one positive (Pittsburgh).

Fortunately, now that President Trump has rescinded the 2014 school discipline “Dear Colleague” letter, school leaders can make policy based on the interests of their students, not based on fear of losing federal funding.

And perhaps the most salutary effect of the Trump era is the extreme and extremely justified skepticism of policy elites. Despite the fact that this study represents the most rigorous empirical examination of restorative justice, editorial decisions made in the game of telephone from researchers to journalists to advocates to educators all but guarantee that school leaders would be better informed if they never heard of it.

Max Eden is a senior fellow at the Manhattan Institute.

The long-awaited report of the National Commission on Social, Emotional, & Academic Development is now out and will doubtless make some waves within education’s chattering classes and more broadly among practitioners. But will anyone else notice or care?

Let me state up front that—aside from its abominably ungrammatical, if slightly clever, title—it’s a solid, respectable product, the sort of thing one rightly expects from the Aspen Institute, the blue-ribbon panel that produced it, and the eminent foundations that paid for it. It’s worth paying attention to. But I will also admit to a fairly serious case of déjà vu.

For the Commission’s central message is not new. It’s basically about “educating the whole child,” as we’ve been told to do at least since Dewey, since Montessori, since Rousseau, arguably since Aristotle. We’re admonished, once again, not to settle for the Three R’s, not to treat test scores as the only legitimate markers of school success, not to succumb to the cramped view that schools’ only job is to develop one’s cognitive faculties. So much more is needed…

Yes, it’s needed, and there’s no harm in being reminded of that once again. As Tim Shriver and Rick Hess write, the report serves as an admonition that we ought not be forced into “false choices in education. Schools should not have to choose between chemistry and character; between trigonometry and teamwork.”

But as they also note, it’s not new: “Since the dawn of the republic, teachers and schools have been tasked with teaching content and modeling character.” They need to do both. They need to walk and chew gum. But the sad truth is that our schools haven’t been doing a very good job of either.

The country may also be commission-weary. There have been so many highfalutin reports on education over the decades—and while they often make a few waves in the near term, few have made much tangible difference over the long run. Their membership invariably consists of ten to twenty exquisitely-respectable-yet-wondrously-diverse representatives of the demographic that in England is called “the great and the good.” They invariably yield an elegant, nicely written, and well laid out report. They invariably start with the bad things that are happening to a country that isn’t doing what they think it should—and go on to the many good things that will occur if their recommendations are followed. And then come the recommendations, plenty (but not too many) of them, aimed at various constituencies, even tagged with which constituents should shoulder responsibility for which urgent actions. There’s ample moralizing. And well-paid PR types will have been enlisted to ensure that the whole thing makes as big a splash as possible, complete with myriad spin-off articles, commentaries, and short-form publications and accompanied—this is newer but comes with the era—by fantastic websites and all manner of carefully curated social media.

Jaded? OK, I admit to it, too. But I may not be alone.

Now to some pluses. The Commission does a nice job of explaining the intersection between SEL and cognitive learning and how the former is to some extent a precondition for the latter. The research they cite both in this report and previously is substantial and convincing. Unfortunately, cognitive learning isn’t necessarily a precondition for SEL—it’s not reciprocal in that way—and there are troubling signs that not everyone touting SEL today is as conscientious as the commission about the need for schools to produce academic learning. Lose that balance and SEL risks being treated as a substitute for academics and falling into the trap of touchy-feeliness, which would both get it marginalized (akin to yesterday’s “self-esteem” movement) and also weaken the country’s slog toward higher achievement. Simply put: SEL, properly conceived, can foster achievement; ill-conceived, it can do harm. 

The commission does well—in “takes a village” style—in explaining that, while attention to SEL ought to be paid within classrooms, teachers and schools can’t do this alone. It’s also a community responsibility and apt to be engendered at least as much by extracurricular and out-of-school activities, agencies, and influences as by teachers.

Also to the commission’s credit are several well-crafted supplements to the main report. The one laying out “a research agenda for the next generation” is clear on the need for much better metrics than we have today by which to gauge progress on the SEL front. (The main report, unfortunately, seems relatively content with “school climate surveys,” though these are both subjective and susceptible to manipulation, the more so if used—as in some states’ ESSA plans—for accountability purposes.)

I do wish the commission had hit harder on the character, civics, and self-discipline fronts. To win wide and durable support for SEL, especially bipartisan support, it’s important to underscore the ways in which SEL, properly conceived, contributes to character formation, the preparation of responsible citizens, and reduced need for external discipline, law enforcement, and incarceration. That, plus a firm link to academics, will help secure SEL as something many non-educators can get behind. Absent these, it’s vulnerable to the charge that it’s really just about helping kids feel good about themselves.

Education Week opened the year with a second annual special issue titled “10 Big Ideas” with, wrote editor Elizabeth Rich, “the potential to define—or redefine—education in the year ahead.” Each includes a staff-written essay accompanied by a commentary penned by an outside researcher, practitioner, or advocate.

Some of the “Big Ideas” are fairly predictable. Colleges will keep striving to diversity their enrollments and to devise new ways of gauging applicants’ readiness. Annual testing will remain contentious. Students will continue to be frustrated by the seeming irrelevance of their classroom work to the “real world” outside. Effective school desegregation—and the narrowing of achievement gaps—remains a tangled web. Bilingual education continues to expand, boosted by the newish “Seal of Biliteracy,” but controversy continues around it and the proliferation of native languages spoken by kids in U.S. schools makes it next to impossible to universalize. And inevitably, in the #MeToo era, schools are being urged to “teach consent as a life skill.”

Three of the other topics caught my eye for different reasons.

Associate editor Christina Samuels pondered whether the absurdly overdue reauthorization of the Individuals with Disabilities Education Act (IDEA) might lead to some rethinking of special ed, but she didn’t quite get the ball over the plate I hoped she was aiming for. Instead of going to fundamentals—why not individualize everybody’s education, how to rein in crazy costs, what to do about all the litigation, and how about all the kids who might not need special ed in the first place if they had been properly taught to read?—she bounced the ball, focusing instead on the shortage of special-education teachers and the uneven access of parents to the “rights” the present law confers on them. Usefully, she did urge better tallying of total special-ed costs.

Policy maven Andrew Ujifusa wrote thoughtfully about signs of mounting teacher mistrust of the major institutions in their work-lives: districts, the U.S. Department of Education, even their own unions and schools. Instead, he suggests, many are turning to “communities created and curated for and by teachers,” sometimes face-to-face, often via social media. He didn’t cite specifics, but curricular sites like sharemylesson.com are proliferating, as are grass-roots professional-development-and-networking venues such as ResearchED and APCentral. “As younger teachers exert increasing influence over their profession,” he wrote, “this sort of activism and professional development might solidify and become unremarkable.” It causes one to wonder whether declining trust in traditional institutions might, in this case, give rise to something better!

At least as provocative is the discussion of “innovation” appearing on several pages. EdWeek’s Research Center surveyed educators to get their views on “the introduction and/or creation of new ideas or methods,” and it yielded some interesting bits. While a majority of educators expect innovation to be a priority in their schools this year, that feeling is a lot stronger among district administrators than among principals and teachers—and administrators are the main source of pressure to innovate. Considering that “one new thing after another” was flagged in Ujifusa’s piece as a source of teachers’ mistrust of those above them in the hierarchy, it’s worth reflecting on the possibility that constant pushing to innovate can backfire.

Innovation fatigue may also cause schools to lose focus on fundamentals that need to be built and sustained. Staff writer Benjamin Herold’s thoughtful essay, dealing mainly with tech-based innovation, ends on a cautionary note: the need to balance and integrate “innovation and maintenance.” And the accompanying mini-essay by Lee Vinsel and Andrew Russell hits it even harder: “You wouldn’t be able to focus on the in-flight entertainment on an airplane if there was a good chance the plane will fall apart around you.” Exactly so.

Closing this issue of EdWeek, I also noted some of the hot issues that didn’t make it in. Nothing on any form of school choice (save for a bit tucked under desegregation). Nothing on teacher quality. Career and technical education. America’s flat-lining high schools. The shaky state of personalized learning. The uncertainties of ESSA. It’s easy to go on. Perhaps nobody had any “big ideas” for dealing with them. Or maybe ten just isn’t enough, considering all that we really ought to be thinking about.

One of the longest running debates about early childhood education is how much emphasis teachers should place on academic content. Thanks to changing perceptions, the standards-based reform movement, and accountability policies that have changed early grade instruction, kindergarten classrooms are increasingly focused on academic content and skill development.

These changes have garnered mixed reactions. Those in favor of the increased academic focus cite studies showing that exposure to advanced content is associated with higher student achievement. Opponents, meanwhile, have raised questions about whether kindergartners are developmentally ready for academics, and whether focusing on more advanced skills reduces play opportunities and leads to poorer social-emotional (SE) development.

To address these concerns, a new study examines the relationship between advanced content in kindergarten and children’s academic achievement and social-emotional outcomes. The study’s authors used the Early Childhood Longitudinal Study of Kindergartners in 2010 (ECLS), a nationally representative study of kindergarteners enrolled during the 2010–11 school year. ECLS included approximately 18,200 children from nearly 1,000 schools, but the authors used a specific sample of 11,600 public school kindergarteners and their 2,690 teachers. ECLS collected information during the fall and spring of the academic year about children’s academic achievement and SE skills through surveys and interviews of parents and teachers, as well as math and ELA assessments. The authors control for class size, teacher characteristics, childcare situation, and a host of demographic and family background variables.

The authors defined advanced content as academic skills that were taught in a higher grade than kindergarten. Using the ECLS dataset, they identified which skills qualified as advanced based on the percentage of teachers who indicated that they were taught in a higher grade. They found four such skills in ELA and ten in math.

On average, teachers spent close to nine days per month on advanced ELA content and slightly more than six days per month on advanced math content. Yet the authors found no negative association between advanced content and SE skills. In fact, greater exposure to advanced math was related to some improved SE outcomes (though exposure to advanced content in ELA was unrelated to SE skills). And as expected, they found that more advanced content was correlated with higher test scores in the respective subjects.

It’s unclear why exposure to advanced math was positively associated with SE outcomes and exposure to advanced ELA content was not, but the authors offer a few theories. First, it’s possible that advanced math enhances executive functioning in a way that advanced ELA does not. Second, there may have been variations in what was considered challenging material in each subject. Moreover, these findings are in line with other research that didn’t find negative associations between challenging academic content and healthy SE development.

The authors do note several limitations of their study. First and most importantly, this study is correlational. There is the possibility of self-selection bias, and the authors can’t definitively say whether children’s gains were due to exposure to advanced content or exposure to highly effective teachers; teachers who are more likely to teach advanced content than their peers, for example, may also produce better achievement and SE gains.

With that said, the authors are “cautiously optimistic that advanced academic content can be taught without compromising students’ social-emotional skills.” The research is a promising sign that they’re right.

SOURCE: Vi-Nhuan Le, Diana Schaack, Kristen Neishi, Marc W. Hernandez, Rolf Blank, “Advanced Content Coverage at Kindergarten: Are There Trade-Offs Between Academic Achievement and Social-Emotional Skills?” American Educational Research Journal (January 2019).

On this week’s podcast, Lindsey Burke, a director at the Heritage Foundation’s Center for Education Policy, joins Mike Petrilli and David Griffith to explain why school choice is poised to have a good 2019, despite its many headwinds. On the Research Minute, Amber Northern examines the relationship between automatic-college-admission policies and the college preferences of low- and high-income students.

Amber’s Research Minute

Kalena E. Cortes and Jane Arnold Lincove, “Match or Mismatch? Automatic Admissions and College Preferences of Low- and High-Income Students,” Educational Evaluation and Policy Analysis (December 2018).