Thomas Dee and Brian Jacob
NBER Working Paper
November 2009
Researchers have produced a slew of studies in the last several years that purport to address No Child Left Behind’s impact (or lack thereof) on student achievement. This latest from well-regarded economists Thomas Dee and Brian Jacob uses state-level (and low-stakes) NAEP data from 1992-93 to 2006-07 to trace the influence of the landmark legislation. They analyze changes in NAEP scores for two groups: states that already had school-accountability policies in place prior to the year when NCLB was fully implemented (the control group) and those that did not (the treatment group). The idea is that NCLB would represent less of a “treatment” in states that already had NCLB-like accountability policies in place; hence, they can serve as foils to states that lacked such practices prior to NCLB’s full implementation in 2002-03. They conclude that NCLB produced statistically significant increases in the average math performance of 4th graders, and in the highest and lowest achievement percentiles. They also witnessed improvements in 8th grade math, especially among low-achievers. But there was no evidence of a similar effect on reading achievement in either grade. Since more than half the states had introduced school accountability policies within the four years prior to NCLB’s implementation, they also conducted a separate analysis that omitted these “late adopters” (since newly-birthed state policies and NCLB policies could easily intermingle and understate the latter’s effect). As expected, the gains attributed to NCLB rose. Of course the integrity of the study rests on whether the pre- and post-NCLB groups make a valid comparison. Dee and Jacob use a prior framework (developed by Eric Hanushek and Margaret Raymond) to identify states that had NCLB-esque systems pre-NCLB. But they also added some states and re-coded a handful of others based on new information and data that they assembled. Though they appear to approach the coding of states carefully, there’s an element of subjectivity to virtually any coding effort, which means the impact of NCLB on achievement might be more or less robust than Dee and Jacob contend. You can purchase their study for a small fee here.
