Natasha Ushomirsky and Daria Hall
The Education Trust
February 2010
This report asks an important question: What happens to low-performing schools over time? Specifically, how many schools stay low-performing, how many make significant gains, and how many fall somewhere in between? Analysts for The Education Trust examine five years of grade 3-8 reading and math data from Indiana and Maryland. (Only the reading findings are discussed in this report.) They conclude that from year one to five, a whopping 64 percent of Maryland’s low-performing (defined as in the bottom quartile) elementary and middle schools improved faster than three-fourths of the schools in the state. Indiana’s low-performing schools weren’t so successful; during a similar five-year period, over a third of them stagnated, over a third ranked as high-improving, and nearly a quarter fell somewhere in between. But should we now conclude that Maryland has mastered some magical turnaround strategy? No, because the way the EdTrust analysts define and measure “improvement” is problematic on three grounds. First, they failed to account for regression to the mean--the statistical phenomenon whereby scores on the extreme end of a distribution will naturally move towards the average over time. So surprise! Low-performing schools averaged more growth. Second, they couldn’t look at individual student progress over time (they had only school-level results) so they looked at the percentage of students reaching the “proficient” level instead. But because Maryland’s proficiency cut scores are so low, many of its initially “high-performing” schools couldn’t “improve” because they were already near 100 percent of their students at or above “proficient.” Third, the standard by which schools were deemed to be “high-improving” was modest at best. They only had to have made gains that were higher than the majority of schools in the state (roughly five points per year), not move from the bottom quartile to the top or anything else dramatic. Bottom line: These schools haven’t turned around, or even closed the gap with other schools in the state; they’ve merely made a positive step in the right direction. Surely no bad thing to do, but let’s not get carried away. You can read the report here.
