The prolonged fracas within and far beyond the National Assessment Governing Board (NAGB) concerning a new “framework” for NAEP’s future assessment of reading has been ominous on several fronts—as I haven’t hesitated to say. But the past few weeks have markedly improved the situation to the point that a reasonable and viable consensus is coming into view.
Trouble began two years ago when the Board named a cast-of-thousands “visioning panel” that came up with a plan for a very different sort of reading assessment, one that emphasized race, deemphasized knowledge, risked concealing bona fide reading-comprehension problems, and threatened to disrupt NAEP’s multi-decade “trendline” of reading achievement.
Most of its members were remote from classroom realities and real-world reading practicalities, and many had drunk deeply at the fount of current fads and dubious research. Some had thrown themselves into contemporary cultural conflicts. That group’s wholesale redesign of NAEP’s approach to reading was destined for controversy.
Hands were wrung, blogs were written, testimonies were submitted, feathers flew, the governing board itself seemed on the verge of coming unglued. So revisions were made—sorely inadequate revisions, unfortunately, from the standpoint of key board members and a number of outside critics (myself included). Once hints began to be dropped that some states (mostly of a vermillion hue) might refuse to participate in assessments erected upon such a framework, NAGB chair Haley Barbour (himself the former governor of one such state) took corrective action. He formed a working party of board members drawn from across several NAGB committees (and factions) but who craved consensus on reading. Ably led by vice-chair Alice Peisch, they worked hard and fast and with awesome celerity came up with yet another revision, this one dubbed the “Chair’s Draft.”
This was put out for public comment late on June 25, an “informational webinar” was held on July 1, and any written comments were requested by July 9 (i.e., tomorrow!). Barbour’s intent is for the Board to vote approval in early August and—if approved—for NCES then to commence the very heavy lift that will enable the new framework to drive NAEP reading assessments from 2026 onward.
The revision is a tour-de-force that the Board should adopt without changing a word. Sure, there are a few words I might change if I were king, but building a livable consensus around the assessment of reading is far more important than trying to please everyone, which is never possible anyway.
The Chair’s Draft is shorter, pruned of much jargon, and much more reader/user friendly. It’s also markedly less ideological—and unlikely to disrupt the vital NAEP reading trendline. Its drafters deserve huge applause.
Yet mice are nibbling. Some, for example, including some on the original “visioning panel” that caused so much grief, want the final Framework to revert to using “unfamiliar” rather than “obscure” to describe words and terms encountered in reading passages that justify the use of “informational UDEs.” This sounds ridiculously wonky but in the long run it matters quite a lot.
UDE is shorthand for “universal design elements,” originally developed to help children with disabilities access and participate as fully as possible in assessments and other school activities. In the case of NAEP, they’re no longer limited to test-takers with disabilities. They come in several kinds, including “informational.” Within the reading assessment, an “informational UDE” might be available to test-takers in the form of a definition or “pop-up” or link, even a short video, intended to explain or define a word or term that the child may not understand. An example in the draft framework involves a passage from a story set in Turkey, where UDEs supply English definitions (translations, really) of two Turkish words.
Those words (“hodja” and “akche”) are indeed “obscure” for almost every U.S. reader in grades four, eight, and twelve who is apt to find him or herself taking part in the NAEP reading assessment. Because it’s hard to comprehend well what’s going on in that passage without the explanation, and since comprehension is the point, supplying a UDE in this case seems fully justified.
But were a new reading framework to open the door to providing such help for any word, term or expression that’s simply “unfamiliar” to readers, much mischief could commence. That’s because “unfamiliar” begins a slippery slope toward the profligate use of “informational” UDEs in any/every situation where some readers may not yet possess certain vocabulary or knowledge even though proficient readers do—and all should. And that leads on to other “informational” situations that bring us swiftly to “background knowledge,” not just vocabulary. Which would hurl NAEP back into a swirling controversy about the role of background knowledge in reading comprehension—a controversy that NAGB members in the webinar last week made clear they did not want to take a position on and did not want to affect the NAEP reading assessment.
It’s one thing to supply help when (as in the Turkish terms in the draft framework) a word is completely foreign. A somewhat similar situation arises with rarely-used—and genuinely “obscure”—English words, such as those that kids get asked to spell during the national spelling bee. (From the 2019 Bee: “The words spelled by this year’s co-champions in the final round include ‘auslaut,’ ‘bougainvillea,’ and ‘pendeloque.’”)
But words that are simply “unfamiliar” to students taking a NAEP assessment will include many that they should know. As soon as NAEP starts providing “informational” assistance with such words, it’s on its way to masking children’s actual reading comprehension. For instance, an eighth grade NAEP reading passage in 2013 included the word “epitome.” Should that have an informational UDE to define it? What if a passage were to mention the Caspian Sea, say, or Indonesia, or cantaloupes? It’s no secret that the original framework “visioners” wanted to suppress background knowledge and, when they couldn’t do that, wanted to supply it to test takers. But the cognitive scientists and reading experts who know the most in this realm view that as a hopelessly misguided quest.
They know, too, that the road to “unfamiliar” never ends. The current NAGB and NCES teams might not travel far down it, but what about their successors? Journey a bit further and NAEP risks artificially inflating students’ scores by assisting them to parse words and terms that they should already know. That’s because they’re words and terms that, when encountered in the “real world” of reading, nobody will come rushing to supply the missing information. Sure, motivated readers with sufficient bandwidth could consult the internet, but it’s impractical to stop and look up every word more challenging than “see Spot run.” Which means that if NAEP papers over things that kids should know but don’t, it will cease to be a trustworthy gauge of their true reading comprehension, at which point it loses value both for policymakers and for educators.
So may the Board in its wisdom please refrain from shifting back from “obscure” to “unfamiliar” or anything in that vein. (If for some reason they need to make a change, how about “esoteric”?)
That aside, Barbour and Peisch and their colleagues deserve kudos and thanks. And may consensus prevail on this as it ultimately has done on just about every major issue facing the Governing Board for three-plus decades. Absent such consensus, NAEP’s credibility (and value to the country) is much diminished. And in no realm would that be worse than in the core skill of reading.