A new study examines the effects of No Child Left Behind on children’s socioemotional outcomes. Prior studies have found that consequential accountability systems like NCLB have yielded positive gains in achievement; others have shown that they narrow the curriculum by focusing on tested subjects. But very few have looked at the potential impact of the legislation on socioemotional or “non-cognitive” outcomes.
The authors use student-reported survey data from the Early Childhood Longitudinal Survey-Kindergarten Cohort of 1998–99 (also called ECLS-K), which is nationally representative. Data used for the study were collected in the spring of students’ third and fifth grade years—the same time of year that students typically take standardized tests. Note that NCLB legislation was signed in January 2002 in the middle of the third-grade year for the sample. During spring of that year, students took tests that established baseline scores for judging school performance in subsequent years. Schools could fail to make adequate yearly progress (AYP) in 2002–03 (students’ fourth grade year); thus, by the time that they were surveyed during the fifth grade year in spring 2004, NCLB consequences were widely in effect. Authors use a “differences in differences” strategy where they compare states that already had consequential accountability plans in place before NCLB took root to states that did not, borrowing from earlier studies by Tom Dee, et al. and Eric Hanushek, et al. It assumes that schools in states without prior consequential accountability systems experienced a much greater increase in test pressure after NCLB took place (aka the treatment states); states with prior consequential accountability systems were the comparison group. The assumption, as the authors explain, is that “changes in the comparison states serve as a counterfactual for the changes that we would have observed in treatment states in the absence of NCLB since these changes should be caused by non-NCLB factors such as other school reforms or shifts in the economy.”
They find that NCLB did not have a consistent effect on the ten socioemotional outcomes they examined, including things like academic anxiety, perceived competence in math and reading, and perceived interest in school subjects. Though they found both modest increases in academic anxiety and interest in math, they couldn’t rule out that these results may have occurred by chance, mostly because they ran numerous significance tests for each of the ten outcomes. Exploratory subgroup analyses also suggest that high-stakes testing produces an increase in academic anxiety for those in the top half of the income distribution (.07 SD) and that students in the bottom half of the distribution experience increases in their competence and interest in math (both about .09 SD). But again, given the drawbacks of the study’s design, we can’t put much trust in these findings. These drawbacks include relying solely on self-reported measures from elementary school students, rather than psychological tests or classroom observations; looking only at the early years of the effects of the policy; and missing standardized test data in both the treatment and comparison groups for several states during the study’s timeframe (i.e., when states were in the midst of the assessment ramp up).
Studying the impact of any intervention on non-cognitive outcomes is not easy, in part because we are still defining and attempting to measure such outcomes. Many smart people are laboring to carry this work forward. For now, this study’s authors warn that “states may not need to spend as much time trying to minimize potential negative consequences on these socioemotional outcomes…” Given where we are now, I’d say that’s a pretty safe bet.
SOURCE: Camille R. Whitney and Christopher A. Candelaria, “The Effects of No Child Left Behind on Children’s Socioemotional Outcomes,” AERA Open (July–September 2017).
