Under federal and state law, Ohio policy makers are responsible for gauging and reporting on the performance of its 3,000 public schools and 600 districts. To do this, Ohio has a report card system that assigns A-F grades based on a variety of performance indicators. While Ohio does not currently roll up these disparate component grades into a final “summative” rating, in 2017-18, the Buckeye State will join thirty-nine other states that do just that.
Why summative grades? They are intended to accomplish a number of purposes, including improving the transparency of complicated rating systems, helping families decide where to send their child to school, and guiding local decision making on which schools need the most help and which deserve recognition. With the importance placed upon these overall ratings, it is critical to examine the grading formula that Ohio policy makers will use to calculate schools’ final letter grades—specifically the weights assigned to each element of the school report card.
Current weights
Ohio law requires the State Board of Education to create the summative school rating formula within two key parameters: 1) it must include all six main components of the state report card; and 2) it must equally weight the Progress and Achievement dimensions. Set forth in administrative code, the state board has established weights for each component as displayed in the table below. When a component is absent for a school—e.g., K-3 Literacy wouldn’t apply to a high school—the existing weights are adjusted in a way that maintains proportional relationships between the components.
Table 1: Ohio’s overall school-rating weights
Are these weights fair?
The central problem with Ohio’s current system of weights is that five out of six components are highly correlated with student demographics or prior achievement. This creates an uneven accountability system that strongly favors high-income schools while disadvantaging low-income ones. In effect, Ohio’s state report card punishes schools simply for serving needy children while giving accolades to those with less disadvantaged pupils.
Using Ohio’s component A-F grades from 2015-16, the tables below demonstrate the extent of the problem.
Table 2: Distribution of A-F school ratings on Ohio’s report card components, 2015-16
Consider the following observations:
- Over 90 percent of high-poverty schools received Ds or Fs on four report-card components: Achievement (94%), Gap Closing (99%), Prepared for Success (95%), and K-3 Literacy (92%). The comparable D or F grades for low-poverty schools are much lower, ranging from 21 percent on Prepared for Success to 73 percent on Gap Closing. These patterns are predictable, given the achievement gaps between low-income students and their peers on state and college admissions exams.
- Graduation rates for low-income students have historically lagged those posted by high-income students. This means that the Graduation Rate component also correlates with demographics: 68 percent of high-poverty schools received a D or F, while just 2 percent of low-poverty schools were assigned such grades.
Meanwhile, the Progress component—value added or student growth—is the one measure in which high-poverty schools can and do perform well. Consider Table 3, which shows that 32 percent of Ohio’s high-poverty schools received an A or B, while 48 percent received a D or F—a less skewed distribution of grades relative to the other components. Ohio’s growth measure is designed to be more poverty-neutral, leading to the different distribution of grades.
Table 3: Distribution of A-F grades on Ohio’s Progress component, 2015-16
Taken together, the five demographically correlated components will constitute 80 percent of a school district’s grade and anywhere from 70 to 80 percent of a school’s grade depending upon the grades served. Meanwhile, just 20 to 30 percent will be based on Progress—or student growth—a measure independent of demographics or prior achievement.[1]
As a result, the weighting system developed by the State Board will assign almost all high-poverty schools a D or F grade. This is not only inaccurate—there are excellent high-poverty schools—but could also have unintended consequences such as: a) misleading school-shopping families about the quality of their options; b) subjecting high-performing, high-poverty schools to intrusive interventions; c) failing to recognize or reward high-performers; d) discouraging educators (and students) in high-poverty schools who start to believe that the grading system—no matter what—results in them being deemed failing.
A weighting proposal
Ohio policy makers should reweight the overall school-grading formula in the way that places more emphasis on growth. Table 4 displays my proposal for reweighting the various components.[2]
Table 4: Proposed summative grading formula
This proposal would make three key changes in Ohio’s weighting system. It would:
- Increase the weight on Progress, the component that houses Ohio’s student growth measures. Several other states already place disproportionate weight on their growth measures, and according to surveys of Ohio parents, growth is viewed as a very useful gauge of school quality.
- Reduce the weight on Gap Closing, Graduation (in district accountability), and K-3 Literacy. These measures that have challenges, notwithstanding their correlation with demographics. For example, the problems with graduation rates include improper assignment of blame or credit for graduation/non-graduation in certain cases and they are open to “gaming” through shoddy credit recovery programs. K-3 Literacy has challenges, too. For instance, it relies on district-selected assessments and the rigor of such exams could vary, giving some districts a slight advantage if they administer a less demanding test.
- Differentiate more clearly the grading formula for K-8 and high schools. Growth should account for somewhat less weight in high school accountability, primarily because students are drawing close to taking their next step in life. Hence, indicators of readiness for post-high school success become increasingly critical.
* * *
There is no one scientifically correct way to determine school grading weights. It will ultimately come down to judgments on issues of what we prioritize and value, how technically sound an indicator is (all measures have their challenges including, yes, value added), how we think certain measures will affect behavior, and how we think about fairness to schools and to students. Yet we must acknowledge that not all report card measures are of equal or similar importance—which appears to be one of the assumptions behind Ohio’s current weighting approach. Buckeye policy makers should revisit these summative weights and work toward prioritizing student growth in the accountability system.
[1] A K-8 school would not have high-school specific components (Graduation Rate and Prepared for Success) and the weights would be as follows: 72.5% on Achievement, Gap Closing, and K-3 Literacy combined and 27.5% on Progress. A grade 9-12 high school would have 77% combined weight on Achievement, Gap Closing, Prepared for Success, and Graduation Rate and a 23% weight on Progress. See pg. 3 of this ODE document for the various permutations in the absence of a component(s).
[2] To implement changes such as this, Ohio legislators would need to repeal the statutory clause mandating equal weight on Achievement and Progress [ORC 3302.03(C)(3)(f)]. Then, the State Board of Education would need to adopt new rules for the new weighting system.
