Spurred in large part by an infusion of over $4 billion in federal Race to the Top funds, beginning in 2009, nearly all states and the District of Columbia implemented major reforms to their teacher evaluation systems. In contrast to previous evaluation regimens that resulted in 99 percent of educators receiving “satisfactory” ratings, the new evaluation systems aimed to better differentiate performance amongst educators.
While well-intentioned, these new reforms were unsurprisingly met with much resistance from many educators and teachers unions, and negatively affected teachers’ perceptions of job security, autonomy, and satisfaction. The controversial policies often required teachers to be regularly evaluated based on multiple measures, including students’ academic growth, and in some states, the ratings that followed were used to make personnel decisions (such as dismissal or tenure, or providing bonuses to teachers rated as “highly effective”). But did this era of “new and improved” teacher evaluation reforms actually strengthen teacher quality, and most importantly, did it increase student achievement and longer-term outcomes, such as high school graduation and college enrollment?
To find out, a team of researchers recently linked data on the timing and nature of states’ teacher evaluation reforms to district-level student achievement data based on standardized math and English language arts (ELA) test scores from 2009 to 2018, plus data from the American Community Survey about high school graduation and college enrollment rates.
Their nationally-representative study sheds much-needed light on the impact of a decade of teacher evaluation reforms.
Disappointingly, after controlling for student demographics and district characteristics, as well as pre-reform differences in student achievement and attainment, the analysts found that “on average, state teacher evaluation reforms had no discernable effect on student achievement in math or ELA.” They also found “no evidence that teacher evaluation reforms impacted high school graduation or college enrollment rates.”
To understand whether major differences between state teacher evaluation policies might impact student outcomes, the researchers also constructed an index of ten common evaluation reform components, such as requiring observations to be part of teacher evaluations and using teacher evaluation ratings to grant tenure, pay “highly effective” teachers bonuses, or fire “ineffective” teachers. They found that for math, the effect of teacher evaluation reforms did not vary by program design. For ELA, they found “some evidence that states that adopted the fewest evaluation components experienced a slight decline in student achievement post-reform,” but those that adopted the most common teacher evaluation components still had little effect on student achievement. And finally, they tested whether reform impacts varied by student body characteristics, and found scant evidence that teacher evaluation reforms impacted achievement or educational attainment for any subgroup.
While at first blush this study’s findings are resoundingly disappointing, it nonetheless offers important insights into the overall impact of these reforms, not just in terms of student achievement at a national scale, but also into how these reforms impacted students’ longer-term outcomes such as obtaining a high-school diploma or enrolling in college.
Note, though, that this analysis was national in scope. As the authors acknowledge, some prior studies of individual district’s teacher evaluation reforms found more positive impacts on student achievement, such as research on the District of Columbia Public Schools’ “DC IMPACT” system, as well as on reforms in Cincinnati and Chicago. As Kate Walsh, President of the National Council on Teacher Quality, recently stressed, “We know it’s possible for teacher evaluation [reform], when well-implemented, to achieve great outcomes,” adding that “We know it’s theoretically possible, and we know it’s practically possible.” While these cases are not generalizable to the nation as a whole, they can offer important insights into teacher evaluation policy and implementation done right—such as the importance of designing high-quality, multiple measure evaluations, and linking high-stakes accountability and teacher salaries to performance ratings, as well as reforming evaluation policies at the district, and not state, level.
Of course, implementation is also key. While state policies may have required districts to incorporate new, more rigorous components into their teacher evaluations, “the vast majority of states provided school districts with some degree of autonomy in designing and implementing teacher evaluation, either by allowing local discretion within a state-designed system or permitting districts to develop their systems given a set of guidelines.” This study does not delve into how state policies were actually implemented on the ground in schools and districts, with what fidelity, or for how long.
States also reformed their teacher evaluation policies over a wide period of time, ranging from 2009 to 2017, so some may not have had much time for their policies to make an impact during the window of time studied. And some of the states that overhauled their policies to include student growth measures in their teacher evaluations later reversed course during this exact same window, again potentially affecting student outcomes.
Given that teachers remain the most critical school-based factor affecting student achievement, strengthening the teaching profession is essential, and figuring out how to evaluate teachers well and fairly is a key piece of the puzzle so leaders can reward and retain high-performers and better support struggling teachers. This work is all the more necessary amid the ongoing Covid-19 pandemic, with so much ground to make up in terms of both teacher morale and student learning.
SOURCE: Joshua Bleiberg, Eric Brunner, Erica Harbatkin, Matthew A. Kraft, Matthew Springer, “The Effect of Teacher Evaluation on Achievement and Attainment: Evidence from Statewide Reforms,” Annenberg Institute at Brown University (December 2021).
