It’s now official. Unless Congress pushes back on Secretary DeVos’s request—which seems unlikely—the National Assessment of Educational Progress (NAEP) tests of fourth and eighth grade reading and math that were scheduled for 2021 will be deferred at least a year. That means this vital source of data on student achievement won’t tell us anything between spring ‘19 and autumn ‘22, when we can reasonably expect the next results to appear. This means, in effect, that NAEP is AWOL for the pandemic and its school shutdowns, turning a blind eye to the learning losses that they’re causing.
This also means that state testing in spring 2021 is now more important than ever—and that President Biden’s education secretary should resist all demands for another round of waivers from ESSA’s requirement that all students in grades three through eight should be assessed annually in reading and math. Absent such data, we’ll see both a collapse, perhaps permanently, of results-based school accountability and—more immediately—an appalling dearth of information about who is and isn’t learning what during these challenging times for K–12 education. (Test scores like those released this week from NWEA are welcome, but as that organization itself acknowledges, they aren’t nationally representative, especially given how many students—most of them Black, Brown, and poor—are missing from the data set.)
Bear in mind that states need not request waivers, even when they’re available. They can—and should—continue with their assessment plans, even if the incoming administration flinches when its pals in the teacher unions demand another testing hiatus. Nor do states need to pause their non-ESSA assessments—end-of-course exams and such—even when Uncle Sam wimps out. Bear in mind, too, that state tests are much less complicated to administer than NAEP during times when not all schools are open and not all students are physically present in the schools that are.
NAEP is sample-based, and the accuracy of its ESSA data hinges on whether its tests can be administered to a representative sample of students in every state, as well as the country as a whole. Even for non-ESSA data—i.e., when no state-level results are to be provided—NAEP’s contractors (WESTAT, ETS, etc.) still need a proper sample of students nationwide in the tested grades. That means properly sampling pupils from every demographic group, as well as rural and urban, public and (usually) private, district and charter, etc. This grows difficult, if not impossible, when some schools have kids sitting in them while others don’t, when this pattern varies from state to state and sometimes week to week, and when only some students are physically present in the schools that are open that week. Lacking a valid sample, NAEP data are useless. And while NAEP tests are now taken online, it seems that nobody has yet figured out how to administer it to students who aren’t physically in school on the dates when it’s given. For all these reasons, much as I regret the NCES-NAGB decision to defer NAEP ‘21, I can’t really dispute it.
State assessments, by contrast, don’t depend on sampling. They’re given to every student who is present, and they yield results for those students, results that are reported to parents for their very own daughters and sons. These data are then aggregated at various levels—classroom, school, district, and state—and are sliced and diced in various ways. It’s true that aggregations may be misleading when some kids are tested and others aren’t. But a great deal can be learned about what has and hasn’t been learned about those who are tested. And in many places, the school and district data will be reasonably accurate, at least for some grade levels, and can be compared with 2019 data to generate some of the “growth” data that will prove essential for identifying which schools and districts did particularly well in addressing the Covid-19 challenge and which face the heaviest remediation burdens.
State education leaders should also consider the possibility of slipping their 2021 assessments from spring to fall, by which time we hope that just about every school and student will be back in place. If swiftly scored, that could yield welcome information about learning levels and losses for teachers commencing the 2021–22 school year, and would also provide baseline data against which spring 2022 test results can be compared and achievement growth calculated. For these reasons, if the U.S. Department of Education opts to offer ESSA waivers for 2021, which I hope it won’t, one version of such waivers might require autumn testing in those states that punt in the spring.
Returning to NAEP, let me flag some other portentous and provocative points that Secretary DeVos made in her letter to Congress seeking a delay in the 2021 assessments. In that document, she also outlined and asked for a major overhaul of NAEP operations, stating that the current arrangements are too cumbersome and costly to sustain.
Consider the implications of this excerpt:
It is time for Congress to rethink the entire NAEP program and improve its efficiency, efficacy, and budget. Since 2002, after the No Child Left Behind Act mandated state-level NAEP tests, the program has become increasingly complicated, difficult to administer, and costly. In FY 2002, NAEP’s budget was $42 million, and $23 million was spent on the mandated fourth and eighth grade tests. By FY 2019, NAEP’s budget was $151 million (a more than 250% increase), and the budget for mandated tests was $88 million (a nearly 300% increase). On a per student tested basis, the cost of the core NAEP test has skyrocketed from $67 to $293. Despite the growing costs and complexity, the country still benefits from reliable, periodic national measures of academic progress, but perhaps not at the current pace.
It seems clear that the costs of conducting the core NAEP tests every two years far outweigh the benefits of marginally enhancing well-established achievement trend lines. This is why our FY 2021 budget request proposed exploring the option of administering reading and math assessments on a staggered, four-year schedule. Reading could be paired with writing in a given “literacy” testing year. Two years later, math and science tests could be administered in a “STEM” testing year. This cycle would repeat, providing thematically focused results every two years and covering all subjects in four years.
This would save as much as $20 million per administration—money that could be redeployed for the needed modernization of NAEP or used to expand other important tests, including civics. Other changes that I encourage Congress to explore in concert with IES, NAGB, and the assessment community include moving test administration to the cloud—to eliminate the logistical complexity and cost of sending thousands of test administrators to thousands of schools—and restructuring NAEP assessments on a foundation of technology-enabled adaptive testing, where students’ right and wrong answers drive which questions the student receives, producing more accurate results and, in particular, generating far more accurate information about the lowest performing students who are the primary focus of federal education programs and policies.
Much to chew on here, at least for NAEP mavens such as myself. Moving to a four-year cycle for ESSA testing makes sense to me, at least during non-pandemic times, as does retrieving some of a tight budget for other subjects besides reading and math (and, I would add, some budgetary slack with which to supply regular state-level data for twelfth graders). Moving to a cloud-based adaptive testing system may also make sense but risks forcing a fresh start for every trendline in what’s called “main NAEP.” It may also threaten NAEP’s long-term trend, which now reaches back almost half a century.
NAEP was last authorized in 2002, and it’s surely time for Congress to take a fresh look. But a lot of babies are now splashing around in the NAEP bathwater, and great care must be taken before any of them are discarded—or left soapy and unrinsed.
