The science of reading is thankfully supplanting dubious methods of teaching young children to read, as state policymakers across the country are increasingly expanding, refreshing, and/or adopting new early reading legislation to prioritize what works. This recent study, conducted by Michigan State University researchers John Westall and Amy Cummings, asks whether such legislation impacts short-term reading and math achievement, as well as economic and racial test-score gaps. They also investigate whether potential impacts depend (or not) on particular policies or groups of policies to move the needle.
They use three data sets, the first of which is a state-level roster of sixteen early literacy policies, such as requiring materials that reflect the science of reading, training teachers on them, and informing parents when their children are falling behind. They use SEDA data to measure academic outcomes across states, supplemented with national NAEP data to determine whether outcomes differ by low-stakes (NAEP) or high-stakes assessments (state tests). An event study design leverages, across states and over time, the differences in both the staggered adoption of early literacy policies and what they comprise. Analysts compare states’ average test scores several years before and after passage of any early literacy policy, and of the complete suite of sixteen policies, to a comparison group of states that do not (or did not yet) have these early literacy policies in place. They also examine third grade reading retention policies in particular since prior research has shown them to be particularly efficacious for younger students.
Before turning to the findings, a couple caveats merit mention. Since some states have amended their early reading policies piecemeal over time, Westall and Cummings chose the earliest adoption date and interpret their results as “intent-to-treat” effects (so states may or may not have particular policies in place), which is not as precise as it ideally would be. The categorization of states is also somewhat fuzzy. Treated states must have adopted at least one policy before the last year of test score data in the analysis (2018 for SEDA and 2019 for NAEP), so that leaves out at least five states that implemented policies in 2021, while including two in which it’s unclear whether they had policies at all (because the source document was missing evidence).
The key finding is that adopting any early literacy policy improves elementary students’ reading achievement on high-stakes assessments, but those effects fade out by middle school. Adopting a comprehensive suite of policies shows bigger and more sustained increases in reading scores (ranging from 0.03 to 0.1 standard deviations) that persist for several years, especially when third-grade retention requirements are included. The analysts find what they initially thought were spillover effects into math, but ultimately suggest that those gains were more attributable to Race to the Top interventions, which were occurring simultaneously in most states during the study period. There were no significant increases in low-stakes reading scores, except in states with comprehensive policies. They also find some suggestive evidence that early literacy policies reduce socioeconomic and racial gaps on high-stakes tests, but these effects are small.
On the one hand, it is encouraging to see any influence of state policymaking on student outcomes given the multitude of intervening variables from the statehouse to the classroom. The fact that the study picks up some positive (albeit fleeting) signals means that districts (and schools and teachers) are implementing the policies to some degree. On the other hand, we need more information relative to which methods of implementation are most effective to carry out particular policies—and lessons learned from states that have experienced favorable outcomes. These are the keys to long-term success and replication of early reading legislation.
SOURCE: John Westall and Amy Cummings, “The Effects of Early Literacy Policies on Student Achievement,” Education Policy Innovation Collaborative (June 2023).
