Monday was insane, with everyone and his grandmother (and her pet dog) attempting to make insightful, quotable comments on the avalanche of new data from the Nation’s Report Card. Some of it was indeed insightful, but much was simply self-promoting, as were many attempts to position oneself in advance as an expert to be taken seriously.
(This cake was taken by eight Michigan State professors who jointly announced on Friday that they’d be “available to comment” and previewed what they might say.)
Speaking as a fully-credentialed, deeply-respected, and widely-published world authority on NAEP myself, of course I have comments of my own on what the data show. But it’s more important to flag some big-picture items.
First, it’s probably unwise for state and local results to come out two weeks before a contentious election. Pre-pandemic, the Congressionally-mandated two-year cycle for fourth and eighth grade reading and math results hit in odd-numbered years. But because NCES felt it could not safely and reliably administer the NAEP tests in spring 2021, they delayed a year, pushing the assessment itself into spring 2022 and assessment results to October. While the three-year gap turned out to be well-timed for purposes of before-and-after Covid calculations, landing in even-numbered years has (obviously) led to a frantic effort to associate a state’s or city’s NAEP results with the policies and actions (or inactions) of incumbent governors and mayors.
On this cycle, the next big NAEP release containing state and local achievement data and trend lines will fall just before the 2024 election, when eleven states will have gubernatorial races, not to mention nationwide results that are bound to be argued over as evidence of what the Biden administration and/or Congress did or didn’t do.
We tend to forget how little direct influence governors and mayors—and presidents and members of Congress—have on what schools do and what kids learn (or not), although it’s not crazy to point to the influence they and their subordinates had in many places on school closures during Covid. Generally speaking, however, the country will not benefit from having its NAEP results—which are seriously meant to be objective, non-partisan, and apolitical—associated in any way with who is running for what. Or, for that matter, with who might be impeached for what. The National Assessment Governing Board should seek Congressional approval for another three-year cycle that puts NAEP back into an odd-year pattern.
Second, please notice the NAEP dog that didn’t bark on Monday: the twelfth grade. That’s because when lawmakers mandated NAEP to deliver state-level results in reading and math, they specified only grades four and eight. The fact is that (mandated or not) there is no subject for which NAEP delivers end-of-high-school data at the state (or local) level.
This is nuts. The end of high school is when state leaders should most want to know how their K–12 students are performing in core subjects. Maybe not reading, but surely math, science, history, and civics! That’s when kids are turning into adults, heading for college, emerging into the world of work. What do they know? What can they do? Who’s not making it? Must we forever settle for SAT and ACT scores, even as those tests fade and mindful that they were never intended for anything except gauging college readiness, often in a self-selected population?
Where’s the clamor for the Nation’s Report Card to tell us how our kids are doing at the conclusion of their compulsory education?
Third, can we please remind ourselves that NAEP results cannot explain why trends go up, down, or sideways? Summative assessments, whether taken by everyone or (in NAEP’s case) taken by a carefully selected sample of everyone, don’t deal with causation. At most, they can be correlated with other data and developments in the world around the kids who sat for them. We can reasonably infer that a key reason for the widespread declines that were reported on Monday is that so many schools were closed for so long because of the pandemic, yet NAEP data themselves cannot prove that that’s the cause. (It could be famine, hurricanes, divorces, curricular changes, absenteeism, measles.) To claim that NAEP results show the effects of such-and-such—e.g., teachers union leaders pressing to keep schools closed—is a form of “misNAEPery.” What we can safely say—and to me it’s pretty damning—is that union leaders wanted schools closed, school did close, and scores fell, all during the same period.
What’s more, because NAEP samples cohorts of students and yields no results for individuals, it doesn’t show whether Johnny, Michael, or Mary learned more or less between test administrations. It only shows that Missouri’s fourth graders, say, in 2022 were less proficient in reading and math than were that state’s fourth graders in 2019. For purposes of tracking actual gains and growth or stasis or losses in real kids, state assessments are more valuable because they yield year-to-year data on individual test-takers (and individual schools).
So where do we go from here? Everyone says we need to bestir ourselves to help kids “catch up” or risk a “lost generation.” Jeb Bush outlined a promising battle plan in the Wall Street Journal on Monday.
But are those who run our schools and school systems up to the challenge? Tuesday’s Washington Post featured two morning-after-NAEP revelations that point in opposite directions.
The “metro” section abounds with vows and plans by state and local leaders in D.C., Maryland, and Virginia to make “heavy investments to improve students’ performance.” Big numbers are tossed around for tutoring, summer programs, and such, with promises to deploy (in Maryland state superintendent Mohammed Choudhury’s words) solutions that “must be innovative, collaborative, and bold.”
Back on page one, however, the Post’s national team reveals that “despite having access to the dollars, school systems throughout the country reported spending less than 15 percent of the federal funding...the most recent installment of Elementary and Secondary School Emergency Relief, during the 2021–22 school year.” Half of the districts with students who were furthest behind spent less than 5 percent of available federal recovery dollars.
Put that in your pipe and try to avoid an explosion. Scores plummeted even as tens of billions meant to pay for recovery went unspent.
Achievement is challenging, and spending alone doesn’t change outcomes. What wasn’t happening in many places over the past two years was intensive tutoring, compulsory summer school, Saturday catch-up sessions, extending days, talking kids into repeating grades, and most of the rest of the Jeb Bush plan. Doing such things means changing an entrenched, calcified system that almost never does things that are “innovative, collaborative, and bold.”
Talking about change is easy. Getting the K–12 dinosaur to do anything different may be impossible.
