History and research make clear that, often, the most disruptive interventions in low-performing schools are those most successful in improving student outcomes. But organizational inertia and active resistance from teachers and administrators can serve as impediments to success if they are not addressed. A recent study from NBER—conducted by Eric Hanushek, Steve Rivkin, and others—examines one such groundbreaking effort in the Dallas Independent School District.
The Accelerating Campus Excellence program (ACE) was launched in Texas’s second-largest school district in 2016 by then-superintendent Mike Miles (now the state-appointed supe in Houston). Its primary purpose was to offer salary stipends to effective teachers and principals in exchange for their teaching in or leading the lowest-performing schools. It was a natural outgrowth of a rigorous new teacher (and principal) evaluation system implemented over the previous two years. That system used multiple measures of effectiveness, including growth and achievement, classroom observation data, student surveys, and more, with the intention of supplanting the traditional pay schedule with one based more on performance than seniority. ACE included other components like afterschool time for kids, but the salary stipends were the centerpiece, and they were Texas-big. Principal pay could increase by roughly $13,000 per year, by $11,500 for assistant principals, by $6,000 for instructional coaches, and by $6,000 to $10,000 for teachers depending on whether they were at the highest level of effectiveness or the next rung under it.
To get a pay bump, teachers had to be ranked in one of those top two categories and be accepted to teach in one of the lowest-performing schools in the districts, hereinafter denoted as “ACE schools,” and that wasn’t an easy task. Existing teachers at ACE schools had to undergo a rigorous screening process to keep their jobs. Ultimately, less than 20 percent of them were retained in those buildings. The rest (including principals) were replaced from the pool of highest performers. Before ACE, the vast majority of district teachers were in the bottom three rungs of effectiveness; after, most were in the top two categories. For teachers accepted to teach in ACE schools, 40 percent received a pay increase of $10,000 per year, 28 percent received $8,000, and 32 percent received $6,000.
The researchers had access to math and reading test scores, demographic data, teacher evaluation data, and salary and stipend amounts. They focused on elementary schools, constructing a panel that linked students, teachers, and schools from 2011–12 to 2018–19. ACE started in 2016, rolling out in four elementary schools, then in an additional five in 2018—so analysts track ACE Cohort 1 and ACE Cohort 2 in their study for up to three years of impact. They use a difference-in-differences research design where similar, low-performing Dallas schools without ACE intervention serve as the control group. Key to the study methodology was that schools in both groups followed a similar pre-treatment trend in performance.
In a win for full-on disruption (and for kids!), ACE schools showed an immediate and very large increase in achievement upon program implementation. Math scores for both cohorts exceeded 0.4 standard deviations, and 0.3 SDs for reading, which brought average achievement close to the district average. Meanwhile, performance barely budged in either control group in the same period. For students with two or three years of treatment, ACE led to larger increases in achievement following matriculation to sixth grade for both cohorts. Students with only one year of ACE schooling ended up with substantially higher fifth grade scores but not subsequent sixth grade scores. In other words, the more time in the treatment schools, the better.
The program turned out to be so successful at boosting student achievement that three of the four initial ACE schools no longer qualified for it by 2019. Stipends were thus eliminated (along with afterschool and other programmatic components), resulting in over 40 percent of high-performing teachers leaving their ACE schools. Those who remained were reassigned as leaders outside the classroom, responsible for providing professional development.
Subsequently, the former ACE schools saw a sharp decline in achievement—so substantial that it reversed much of the prior benefit. Test scores fell significantly in the wake of effective teachers’ exits and reassignments. During the same period, control-group schools had less turnover overall and did not fall victim to the selective turnover of high performers like the ACE schools did.
In the end, the roller coaster of programmatic changes meant that student performance also experienced a dizzying loop of rises and falls. By removing chronically-low-performing teachers and principals, bringing in proven high-performing teachers and principals, and providing sizeable salary increases that rewarded a job well done, ACE demonstrated how pay for performance done right can benefit students. But its demise also revealed ingrained reversion to the status quo in our schools, the quick erosion of success when effective programs and incentives evaporate, and worst of all, students stuck on the carousel of reform when adults fail to prepare for the accomplishment they’d hoped for.
SOURCE: Andrew Morgan et al., “Attracting and retaining highly effective educators in hard-to-staff schools,” NBER Working Paper (March 2023).
