Trouble continues at the National Assessment Governing Board (NAGB), the policy body for the National Assessment of Educational Progress (NAEP). Last week, I wrote about the draft “gag order” that board chair (and former Mississippi governor and RNC chair) Haley Barbour had issued to members, which would radically cramp their ability to communicate with each other and with outside advisors, critics, friends, and random folks. Subsequently, the board’s executive committee and staff have reportedly set about to find a less burdensome way of allowing members to engage in reasonable communications without running afoul of government-wide open-meeting requirements. We’ll hope that goes well.
Meanwhile, however, a legitimately controversial new framework for future reading assessments is back on the board’s agenda at its quarterly meeting this week, and it’s not looking good, either for comity within the board or for the future of the country’s foremost assessment of students’ reading prowess.
NAEP’s reading assessments in recent years rely on a 2009 framework that replaced one in use since 1992, but results have been able to be reported on an unbroken trend line dating all the way to 1992 (save for a “dotted line” prior to 1999 when testing accommodations began for children with disabilities). It’s a very important continuous gauge and one that shouldn’t lightly be replaced.
Assessment frameworks do get revised, updated, and sometimes replaced from time to time, however, and in late 2017, NAGB’s Assessment Development Committee (ADC) began a review of the reading framework, including a quest for expert input, research, and advice. A year later, the board awarded a contract to WestEd to “conduct an update of the NAEP Mathematics and Reading Assessment Frameworks,” as well as specifications for the tests themselves. That led to the assembling by 2019 of an elaborate set of consultants, committees, and staff: a thirty-two-member “visioning panel,” a seventeen-person “development panel,” an eight-member “technical advisory committee,” and ten WestEd employees.
They were ambitious, to put it mildly, and some of their ambitions were destined for controversy. The “visioning” group, in particular, wanted future NAEP reading assessments to incorporate reading in various media, not just words on paper; to emphasize the “deep reading” and “disciplinary reading” that are stressed in some new state academic standards (and the Common Core) and a variety of new reading tests used elsewhere; to recognize that “texts inevitably are cultural and political in nature”; to take account of differences among test-takers, not only in cultural and racial contexts, but also in levels of motivation and interest; and to compensate for differences in students’ background knowledge and vocabulary.
The goal was to develop a new reading framework that the board would adopt in time for the 2025 assessment cycle, which has since shifted to 2026 due to an overall Covid-induced delay in much of the NAEP calendar. But when a draft was put out for public comment in mid-2020, it soon became clear that reaching agreement would not be easy.
Thousands of comments flooded in. Many welcomed the developers’ very different “vision” of reading and its assessment. But there was also pushback on a host of issues, including two that proved especially volatile. The drafters had set out to reframe reading and its assessment in “socio-cultural” terms, seemingly in opposition to the “cognitive” terms that have long undergirded the assessment process—and not just in reading. And in the name of “equity,” the drafters proposed a variety of changes in test design, administration, scoring, and analysis, as well as provision of various “scaffolds” for test-takers, such as supplying background knowledge to help them understand the content or meaning of terms and texts that they encounter in the course of the assessment. The lead advisors on this new framework (notably David Pearson and Gina Cervetti) have done their utmost throughout to minimize or compensate for the role of background knowledge in NAEP’s assessment of students’ reading comprehension, even though—as thoroughly documented by the likes of cognitive psychologist Dan Willingham, E.D. Hirsch, and many others—such knowledge is omnipresent (or not) and fundamental, which becomes immediately obvious when one exits the classroom and enters the real world of reading. They and a host of other framework advisors and staff members have also striven throughout the process to view reading through a “socio-cultural” lens rather than as a universal skill.
This led observers such as Johns Hopkins professor (and former New York State education commissioner) David Steiner and Emory English professor Mark Bauerlein to warn that making such changes in NAEP’s approach to reading would actually mask crucial information that educators need to help children read better. “Rather than allowing poor performance to serve as a signal that large knowledge gaps should be fixed through better education,” they wrote in City Journal, “we will simply lower the impact of background knowledge on the reported test performance.... If NAEP follows this route,” they concluded, “its assessment will no longer be a reading test that we can trust to demonstrate where students need more help—and where teachers should focus their efforts.”
Much debate followed within the board, following which changes were made in this very lengthy and dense document. The current version, which goes before NAGB this week (though final decisions have been put off until August), runs more than 150 pages. The ADC and its advisors have indeed made a number of revisions, but most appear to be cosmetic and rhetorical, intended to mollify critics more than to alter the fundamental changes that the new framework would make in reading assessment.
The impact of those changes goes far beyond NAEP, of course, as the framers of this framework understand and presumably intend, for a recasting of NAEP’s view of reading proficiency and its way of testing that proficiency will gradually percolate through the entirety of K–12 education in America, seeping into state academic standards and assessments, into textbooks, teacher preparation, and much more.
A number of NAGB members expressed serious reservations about the ADC’s recasting of reading, at least as troubled by how it is justified and presented than in the actual changes it will produce in the test materials that kids will face, although it’s not yet clear whether those changes will result in NAEP having to start a new trendline when it reports future reading scores. The heat generated by these intra-board disagreements is a problem in its own right, as one of NAGB’s foremost assets over the past several decades has been its capacity to reach consensus on important matters. Will this important twenty-six-member governing body henceforth come to resemble the factionalized, politicized, schismatic (and gridlocked) ways of so many other key policy bodies, most visibly the U.S. Congress? Yet nothing is more fundamental to what NAEP does and what NAGB presides over than assessing the most basic of all academic skills, the first of the three R’s.
Seeking to find a way out of this grave situation, Board member Russ Whitehurst, himself the founding director of the Institute of Education Sciences and a formidable authority on education research, volunteered to edit the framework document. His goal was to shorten it, make it accessible to ordinary readers and purge it of the faux research, controversial rationales, and inflammatory rhetoric that had made it so contentious.
The ADC chair threw a fit, insisting that Whitehurst shouldn’t do this, but Chairman Barbour encouraged him to proceed. And about ten days ago, Whitehurst submitted his slimmed down, cooled down, and simplified revision, noting as he did that his edits “do not require any changes in the plans for the 2026 reading assessment.” It was, in my view, a huge improvement that might actually lead to something the entire board—and, more importantly, an on-edge, overheated country—might comfortably live with.
But what followed was pretty awful. On Friday evening, through the dubious magic of Zoom, I watched ADC members (all of them, as far as I could tell) savage the Whitehurst redraft—and Whitehurst himself—for having dared to mess with their handiwork. Worse, they may have been egged on by a NAGB staff member who, when circulating the Whitehurst edits to board members, accompanied them with her own lengthy memorandum purporting to show how all of his edits did violence to the committee’s own redraft. This was the most glaring example of staff manipulation of members of a governing body that I’ve seen in a long while, seeming to exploit the fact that the ADC is itself comprised almost entirely of well-meaning teachers and other building-level practitioners and laymen, rather than experts on the sometimes-obscure issues and research that’s involved.
The session resembled a Maoist “denunciation” of someone for not wholeheartedly supporting the cultural revolution. Fortunately, the chosen victim, Russ Whitehurst, had wisely opted not to be insulted and reviled in person. In that sense, it was actually more like a modern-day faculty meeting or editorial board meeting where an absent individual gets “canceled” for deviating from the progressive party line.
Whitehurst was excoriated up one side and down the other for his effrontery, for being out of step with feedback the committee had received from “the field,” but above all for changing the framing and rationale for the new framework itself.
Before the ADC members piled on, several members of the technical panel that had advised them on the new framework rushed through a confusing presentation of an ambiguously-worked survey of state testing directors. The techies sought to present those survey results as evidence that states try to exclude “knowledge” from their reading tests. That’s not what I was seeing as the slides flashed past on the screen, nor did anyone pause on the sticky question of whether vocabulary itself is a proxy for knowledge that state tests do not in fact exclude.
The “get Russ” attack that followed was led by committee member Christine Cunningham, whose day job is Professor of Practice in Education and Engineering at Penn State and who is, to my knowledge, the only academic on the ADC. She is formidably articulate and was loaded for bear, but her own training is in biology and science education, not cognitive science or reading. Almost all the evidence she cited regarding education research came from a single source.
Dr. Cunningham swiftly took the (virtual) floor and was prepared with a lengthy bill of particulars as to what she found wrong both with Whitehurst’s rewrite and with his motivation in doing it at all. She periodically interrupted herself to ask fellow committee members to raise their hands to signal agreement with her on various points. Another observer analogized this to a series of “loyalty oaths” among committee members. And yes, the other members did pile on, all decent, well-intentioned people, but not visibly conversant themselves with the research base for reading comprehension.
Students’ background knowledge was one big issue. Dr. Cunningham clearly belongs to the faction that thinks it should have no place in the assessment of reading comprehension. The other big issue was whether to ground the new reading framework in a “cognitive” understanding of reading or a “socio-cultural” view. Dr. Cunningham insisted on the latter and declared (citing her sourcebook) that the “latest” research requires it.
It was, frankly, a horror show—but nobody on the NAGB staff said a word, not even to remind committee members that Chairman Barbour had invited the rewrite. Unless cooler heads somehow prevail, we can expect further fireworks and denunciations at this week’s meeting of the full board. Which leaves me wondering whether there’s any way to unscramble these eggs at this point—and what this portends both for the future of reading assessment and for the future of the National Assessment Governing Board.
