Is Your School "Celebration Eligible"? - Rating the School Ratings

Since the beginning of the No Child Left Behind era, most schools in all 50 states have been given an evaluation of student performance and an overall rating. While crafting a thoughtful and nuanced accountability system is a frequent topic of discussion on The Gadfly (and is really what matters most), here I simply want to discuss the label that sums up a school's overall evaluation. Some might say it's wrong on principle to label schools. Others worry (and sometimes justifiably so) that a nuanced view of schools get lost when we attempt to boil it all down to a single school rating. Moreover, some may see these labels as nothing but a value judgment about "good" schools and "bad" schools when it's clear that parents value many different things about a school. From academics and facilities to safety and course offerings, even the "best" school might not be best for all kids. 

However, we can make objective judgments in some areas.  In addition, use of these labels is not only widely supported, it's also ingrained in federal policy through both NCLB and waivers. So the question is: If we're going to put schools into categories, what should those categories be called?  A few ideas to consider:

1.       States should avoid too few or too many categories - One of the major gripes about No Child Left Behind at the time of passage was that it treated similar schools very differently. The law initially set a bar for schools to clear called Adequate Yearly Progress (or AYP). Under this old system, School A, which barely missed the target, would be subject to sanctions, but School B, which barely met the target, would not. The difference in learning between the two schools might be minimal, but the consequences might not be. For this reason, it is important to have at least three categories, with the middle category serving as a kind of buffer. There should also be a purpose to each category, even if it's just to separate the struggling schools that need a watchful eye from those that need a significant intervention immediately. By the same token, too many categories can be cumbersome and confusing.

2.       Categories need to describe a measure of quality (or lack thereof) - Making labels transparent is tougher than it might seem. Many of us might remember the Blue Birds and Red Birds reading groups in elementary schools, where one group was full of strong readers and the other with struggling readers so that the teacher could differentiate instruction. Students eventually figured out which was which, but the labels were, for obvious reasons, unclear by design. When it comes to school ratings, however, we want the public and parents to know how schools are performing. Some states (like Michigan, with its strange color scheme) have unfortunately attached near-meaningless labels that make the overall picture less clear.

3.       Rating categories need to be tiered (and point to which way is up) - Categories should make sense as a part of the whole. It's not enough to simply choose some nice-sounding descriptors. Each one needs to make sense as part of the larger group and, ideally, lend clues as to what the other categories might be. The categories also need to make clear which way is up. A 1-5 rating scale might seem sensible, but there is a fatal flaw: Is a 1 best or is a 5? Indeed, a "1" is best in Maryland and Massachusetts but worst in Oregon. A somewhat similar (though less serious) problem occurs in using a star-rating system that might have a maximum of four stars or five.

4.       Avoid grade inflation - Some states have been tempted in the past to have labels that are essentially: Pretty Great, Great, and Very Great. If everyone is "great," then there is less likely to be media and public pressure to improve (which may be the point of such happy-sounding labels). Regardless of the rating system, states could always put the vast majority of the schools in the best category, no matter what it's called. That's another form of grade inflation that must be avoided. But at the very least, the labels assigned should not give the misleading impression that everyone is doing just fine (or terribly).

To rate each state, I'm using the above criteria to come up with my own rating system for state rating systems. States should have the right number of categories, and they should be generally understood to measure quality. The best rating systems have only one order that makes sense, are not overly exuberant, and allow users to infer what the other labels might be even if they only know one of them. The same will be true here. As a bonus, since this is an education-related rating system, I want something with a deep history in schools. To meet all of those stringent criteria, I've decided to go with "A" "B" "C" "D" and "F." Governor Jeb Bush figured this out in the 1990s and, more recently, has helped other states adopt this system as part of his Foundation for Excellence in Education's "Reform Agenda." So far, an impressive 15 states have gone this route, with hopefully more to follow.

Here's how every state's rating system (as evaluated by ECS and some additional research) stacks up:

A - There's really only one rating system that fits here.  Alabama, Arizona, Florida, Indiana, Louisiana, Maine, Mississippi, New Mexico, North Carolina, Ohio, Oklahoma, South Carolina, Utah, Virginia, and West Virginia (A, B, C, D, F)

B - Systems that almost make the cut but don’t completely meet the criteria laid out above.

• Alaska, Idaho, and Nevada use a star-rating system (*****, ****, ***, **, *)
• Georgia (index up to 100)

C - Somewhat coherent word jumbles and more. Many states in the next few categories use "Focus" and "Priority" labels, which often have specific meanings as part of their waiver from provisions of the federal No Child Left Behind law.

• Kansas (Exemplary, Exceeds Standard, Meets Standard, Approaches Standard, Academic Warning)
• Kentucky (Distinguished, Proficient, Needs Improvement)
• Missouri (Accredited with Distinction, Accredited, Provisionally Accredited, Unaccredited)
• Nebraska (Exceeds Standards, Meets Standards, Below Standards)
• New York (Reward School, Good Standing, Local Assistance Plan, Focus, Priority)
• Rhode Island (Commended, Leading, Typical, Warning, Focus, Priority)
• Washington (Exemplary, Very Good, Good, Fair, Struggling)
• Wisconsin (Significantly Exceeds Expectations, Exceeds Expectations, Meets Expectations, Meets Few Expectations, Fails to Meet Expectations)[i]
• Wyoming (Exceeding Expectations, Meeting Expectations, Partially Meeting Expectations, Not Meeting Expectations)

D - Incoherent word jumbles and more.

• Arkansas (Exemplary, Achieving, Needs Improvement, Needs Improvement Focus, Needs Improvement Priority)
• Colorado (Performance Plan, Improvement Plan, Priority Improvement Plan, Turnaround Plan)
• Connecticut (Excelling, Progressing, Transition, Review, Focus, Turnaround)
• District of Columbia (Reward, Rising, Developing, Focus, Priority)
• Hawaii (Recognition, Continuous Improvement, Focus, Priority, Superintendent Zone)
• Maryland, Massachusetts, and Oregon (Level/Strand 1, 2, 3, 4, 5)[ii]
• Michigan (Green, Lime, Yellow, Orange, Red) - Lime: worst rating category runner-up.
• Minnesota (Reward, Celebration Eligible, Continuous Improvement, Focus, Priority) - "Celebration Eligible" wins the award for worst rating category!
• Pennsylvania (Reward: High Achievement, Reward: High Progress, Focus, Priority)
• South Dakota (Exemplary, Status, Progressing, Priority, Focus) - "Status” as a status?
• Tennessee (Reward, Priority, Focus)[iii]

F - The remaining states have binary systems in which one group of schools are considered great and the other group terrible. It's possible many of these states will change (or are already working to change) their systems in order to win a waiver from provisions of NCLB, but for now their systems are not particularly helpful.

• California, Delaware, Illinois, Iowa, Montana, New Hampshire, New Jersey, North Dakota, Texas, and Vermont[iv]