Last week, the first randomized control trial study of “restorative justice” in a major urban district, Pittsburgh Public Schools, was published by the RAND Corporation.
The results were curiously mixed. Suspensions went down in elementary but not middle schools. Teachers reported improved school safety, professional environment, and classroom management ability. But students disagreed. They thought their teachers’ classroom management deteriorated, and that students in class were less respectful and supportive of each other; at a lower confidence interval, they reported bullying and more instructional time lost to disruption. And although restorative justice is billed as a way to fight the “school-to-prison pipeline,” it had no impact on student arrests.
The most troubling thing: There were significant and substantial negative effects on math achievement for middle school students, black students, and students in schools that are predominantly black.
What are we to make of these results? For education journalists like U.S. News and World Report’s Lauren Camera, there’s an easy solution: Don’t report the negative findings and write an article titled “Study Contradicts Betsy DeVos’ Reason for Eliminating School Discipline Guidance.”
When asked why she left her readers in the dark regarding the negative effects on black student achievement, Camera said that it “wasn’t intentional,” explaining that “it wasn’t meant to be a deep dive into the study. And we linked to it, so readers who wanted to follow up could.”
Chalkbeat’s Matt Barnum is somewhat unique among education journalists for his practice of reading academic studies in full before writing about them. Barnum commented, “Well, I will say that the researchers didn’t do any favors in framing the results for reporters. The negative test for effect for black kids is buried on like page eighty with no mention (that I saw) until then…. [T]he research itself is excellent; their choice in framing is…notable.”
Unfortunately, the vast majority of the education policy community will not read this study in sufficient depth to share their disappointment in RAND’s unmistakably slanted editorial emphases. They will read of mostly positive results, and of negative results framed by the RAND researchers as likely attributable to bad implementation of good policy.
It is very sad that the so-called “evidence-based policymaking” community has rendered itself immune to the intellectual breakthrough that enabled the scientific revolution: accepting the falsification of a hypothesis. When it comes to studies of ideologically-preferred policies like restorative justice, the logic all too often is: “Heads, I win. Tails, I would have won if it were implemented correctly.”
The mental itch to label negative results as a product of “bad implementation” rather than a failure is not only anti-scientific, it also short circuits thoughtful policy discussion. Everything beyond empirical data is necessarily the realm of theory, and there are many more interesting and intuitive theories for the failures.
For example, it could be true that the positive effects in elementary school and negative effects in middle school could be reconciled by “bad implementation” in middle schools. Or perhaps the explanation could be that six-year-olds are different than thirteen-year-olds, and that restorative justice works in elementary schools but not middle schools.
It could also be true that the negative results for black students (and for all students in predominantly black schools) could be attributable to bad implementation. Or perhaps restorative justice is uniquely bad for black students. After all, discipline reformers contend that the explanation for the academic and disciplinary racial disparity is teachers’ cultural incompetence when it comes to understanding and relating to black students.
Accepting that premise, the negative results for black students should suggest that restorative justice is an even more culturally incompetent approach.
The researchers conduct an admittedly non-causal empirical dance to support the proposition that the disjunction between teacher and student perception of teacher classroom management is due to non-implementing teachers. That could be true. It could also be true that teachers think that it makes them better, even as students see their classroom climate deteriorating.
For my part, I would privilege the perspective of students over teachers.
Restorative justice is frequently presented to teachers as “evidence-based” and on the cutting edge of “social justice” as something that works if they embrace it. Man’s capacity for self-deception cannot be discounted, and if teachers think they’re doing better even as students think things are getting worse, that would be consistent with the policy drama that has played out writ large over the last two years: In the face of increasingly overwhelming negative evidence, social justice education reformers have only grown more vociferous in their insistence that discipline reform works.
Right now, the tally of studies on the academic effects of discipline reform on school districts are three negative (Pittsburgh, Los Angeles, Philadelphia) and one null/positive (Chicago). In terms of student surveys, my tally has four negative (NYC, Los Angeles, Washoe County, Seattle) and one negative/positive (Chicago). When it comes to local teacher surveys, I’ve seen eleven negative (Oklahoma City, Baton Rouge, Portland, Jackson, Denver, Syracuse, Santa Ana, Hillsborough, Madison, Charleston, Buffalo) and one positive (Pittsburgh).
Fortunately, now that President Trump has rescinded the 2014 school discipline “Dear Colleague” letter, school leaders can make policy based on the interests of their students, not based on fear of losing federal funding.
And perhaps the most salutary effect of the Trump era is the extreme and extremely justified skepticism of policy elites. Despite the fact that this study represents the most rigorous empirical examination of restorative justice, editorial decisions made in the game of telephone from researchers to journalists to advocates to educators all but guarantee that school leaders would be better informed if they never heard of it.
Max Eden is a senior fellow at the Manhattan Institute.
