Gallons of ink, some on this blog, have been spilled about what Ohio should do about academically troubled school districts. Put forward in the Senate Education Committee, the latest proposal would overhaul the current turnaround model by replacing academic distress commissions (ADC) enacted via House Bill 70 in summer 2015 with a softer, more patient form of state intervention.
ADC critics repeatedly argue that they “don’t work,” cause disruption and demoralization, and have generally made bad situations worse. Unfortunately, no evaluation of the academic impact of ADC policy exists, so these claims often go unchallenged. Yet with the release of the 2018–19 school report cards, we can begin to see high-level trends that might indicate whether the most strident criticisms are merited, or whether we actually see signs of increased student achievement.
Youngstown
Intervention in Youngstown predates the HB 70 revisions to ADC law, as the district first came under the thumb of the state in January 2010. Indeed, the district’s lack of progress under Ohio’s old (and meek) turnaround framework prompted legislators to adopt a more muscular approach. In June 2016, Krish Mohip took the helm as the district’s first CEO, a leadership position called for in HB 70. Though likely a transition year, 2016–17 is thus a logical baseline to begin tracking performance under Youngstown’s ADC.
The table below displays data compiled over the past five years on two key measures of performance on state exams: 1) value added, a measure of student growth over time that doesn’t strongly correlate with demographics, and 2) performance index, a composite measure of student proficiency across grades and subjects.
Table 1 shows that academic performance was bleak in Youngstown in the year prior to the ADC, and progress since then has been slow. The districtwide value-added rating hasn’t budged, nor do we observe higher percentages of students attending “A” rated schools on this measure. Kudos to Youngstown Early College High School, the only school to accomplish this feat since 2014–15—four years running. Overall, Youngstown students still struggle mightily on state exams, and too few attend the high-performing schools needed to change these trajectories. That being said, the results registered after ADC implementation are not appreciably worse than in the year immediately prior.
Table 1: Key improvement indicators for Youngstown School District
* These are unofficial ratings based on one-year value-added index scores available via the “District AMO Calculation” tables. (ODE began incorporating one-year value-added scores into the Gap Closing component in FY 2018.) Ohio’s official value-added ratings, used in the District Value-Added Rating and the Schools Rated “A” rows, are based on index scores averaged over three years. While multi-year scores improve the stability of ratings from year to year, single-year scores can provide useful information in turnaround situations. The statewide performance index score was 81.6 in 2015–16 and 84.7 in 2018–19. †Ohio used different state assessments in 2014–15 than in the more recent years; the statewide average PI score that year was 84.3.
Lorain
Akin to Youngstown, Lorain was also under state oversight for low academic performance prior to the 2015 reforms. After the new ADC law passed, David Hardy was hired as the district CEO in July 2017. His first year, 2017–18, was likely a transition year, but we can now track the performance of Lorain for two years after ADC implementation.
As might be expected in a district under state oversight, Table 2 shows that Lorain posted poor results before ADC intervention. In the two years immediately prior, the district received an “F” value-added rating and no students attended an “A” rated school according to this measure. Lorain’s overall value-added rating has remained depressed in the two most recent years, though improvements are visible when focusing on the growth data from only 2018–19. In both core subject areas, the district’s single-year value-added index scores were equivalent to “C” ratings. Moreover, one Lorain school, Steven Dohanos Elementary School, posted an “A” value-added rating last year, the only school to accomplish that feat during this period. The district’s performance index scores have also risen over the past two years, yet another promising sign of improvement. Last, though not displayed in Table 2, the district overall rating changed from an “F” to “D” in 2018–19.
Table 2: Key improvement indicators for Lorain School District
* For more on these data points, see the notes under Table 1.
East Cleveland
East Cleveland is the third district currently overseen by an ADC. Its commission was established halfway through the 2018–19 school year, with CEO Henry Pettiegrew hired in February 2019. Thus, we cannot yet track performance under an ADC model.
Districts on “ADC watch”
ADCs not only seek to turn around chronically-low-performing districts but they may also spur improvements among other struggling districts by the prospect of intervention. Under the HB 70 framework, districts earning three consecutive overall “F” ratings fall into state receivership via ADC. Last year, thirteen districts were on “ADC watch” by merit of a failing rating. Of these districts, just one, Dayton, had received two such ratings.[1]
Anecdotal evidence indicates that at-risk districts took measures to avoid intervention by working to improve their ratings. The results shown in Table 3 suggest that the threat of intervention may have led to higher student achievement. In all but one of these districts, performance index gains exceeded the statewide average, which includes the vast majority of districts not in jeopardy of intervention. Ashtabula and Mansfield, for instance, registered strong improvements—+4.3 and +2.4 points respectively—as did big-city districts such as Columbus (+1.7) and Dayton (+1.6). While these results aren’t conclusive, they at least hint at the possibility of the positive effects of accountability pressures.
Table 3: Districts under “ADC watch” in 2017–18
***
Because they call for bold changes in district governance and leadership, ADCs have left many unhappy. Appeasing dissatisfied interest groups might be politically expedient, but legislators should also consider whether it’s fair to students to pull the rug out from beneath them before promising reforms can take hold. The early results from Youngstown and Lorain show that ADCs aren’t the academic catastrophe that the harshest critics claim. Nor are they an overnight miracle-cure. In the end, we must remember that improving performance in long-dysfunctional districts takes patience and resolve, qualities that seem to be lacking in the debate around ADCs.
Update (10/9/19): The unofficial value-added ratings based on one-year math and ELA data from the Gap Closing component and reported in the tables above reflect just grades 4-8. Since the publication of this post, ODE has released one-year scores based on all grades-subjects with value-added data (including also high-school EOCs and science). Had data from all state exams been used to calculate one-year value-added scores, both Lorain and Youngstown would have received an equivalent of an "F."
- Aaron Churchill
[1] Cleveland school district also received an overall “F” rating in 2017–18 but was not under threat of an ADC, as it already has an alternative governance structure and is undertaking an improvement plan.
