A new working paper from the Stanford Graduate School of Education uses roughly 300 million state math and English language arts test scores from 2009–15 for students in third through eighth grade in over 11,000 school districts across the country to take a really-big-picture look at patterns of academic achievement. The analysis allows users to compare the growth rates across U.S. school districts, a view of educational quality that is rarely seen at a national level. The findings—broken down over time, by geography, and into various subgroups—should be of interest to all education stakeholders.
The data come from NAEP and state assessments via the National Center for Education Statistics and exclude only the smallest districts for whom data on test scores and/or socioeconomic status (SES) are not available due to small sample sizes. Data on students in bricks-and-mortar charter schools are also included, rolled into the data of the district in which each school is located. Data on students in online charter schools, which enroll without regard to district boundaries, is excluded. The author of the study estimates that the data account for almost 99 percent of all public school students.
The best news comes from the temporal analysis: How a child scores at the end of eighth grade has more to do with the academic growth rate he attained between third and eighth grade than where he may have started in third grade. Third grade test scores—the starting point for each cohort—showed gaps that correlated strongly with the socioeconomic status of the community in which schools were located, indicating that education outcomes between pre-K and third grade are heavily influenced by resources available to students—in school, in families, or both. Lead researcher and author Sean F. Reardon then looked at where those same students finished up at the end of eighth grade, trying to pinpoint the contribution made by a child’s educational experiences between grades three and eight. Reardon’s findings indicate that average third grade scores do not predict the rate at which average scores change between third and eighth grade. Starting out ahead was no guarantee of superior growth; students who started out behind could catch up and exceed the growth rate of their peers. What’s more, students in low-SES districts showed high growth rates somewhat more often than did students in high-SES districts.
All of this points to the fact that living in poverty does not axiomatically guarantee poor academic outcomes for students, even if it’s commonly associated with starting from behind. However, Reardon’s geographic and subgroup analyses bear out that poverty often does equate to poor academic outcomes. Taking a wide-angle look at the areas where third grade test scores start out high and are followed by high growth shows a concentration in areas one might expect: suburbs and exurbs in wealthy areas of the northeast and California, for example. But low scores followed by low growth patterns are more common in the Deep South and rural areas in the West. There are also many districts across the country where high scores are followed by average or below average growth. Importantly, tracking the districts where third grade test scores start out low but are followed by high growth picks out some areas of Tennessee and the city of Chicago, among others. (Check out this nifty tool from the New York Times based on this analysis, with which users can compare the growth rates of various districts across the nation.)
It cannot be overstated that this is a very-high-level analysis. A school district is too large an entity with which to make meaningful determinations of school-level quality, charter schools are lumped in with districts, and Reardon creates a single score aggregating both math and English language arts test scores as a means of comparison. But these design facets should not take away from the overall message: Quality education matters at all ages. High quality early education can give young children a firm foundation for learning, but that foundation can be eroded by lower quality education going forward. Similarly, low quality early education or socioeconomic challenges can result in a poor start, but steady doses of high quality education over time can help those students make up ground.
To get a clearer view of school quality, users need to dig into finer-grained data to uncover those places—individual schools or classrooms—that are providing the most beneficial education. In short, good schools matter. Those providing both a firm foundation and strong continuous growth must be identified and their practices replicated. But figuring out which districts are having the greatest success lifting achievement and where they are located is a good start.
One final note: Continuous growth in higher grade levels appears to be Reardon’s holy grail, but his analysis offers no indication of how many of the students’ test scores approach proficiency at any point in time. Growth matters, without question, but it must have a proper end goal—what a student must know and be able to do at any given end point. Growth by itself is not a worthy enough benchmark.
SOURCE: Sean F. Reardon, “Educational opportunity in early and middle childhood: Variation by place and age,” Stanford Graduate School of Education working paper (December, 2017).
