Important education insights sometimes arise from developments in other fields.
This happened to me twice in recent weeks. Both episodes bear on results-based accountability, how it works, what can go awry-and what's wrong with the usual substitutes.
First, a new study of hospital accreditation looked into whether it makes any difference for the quality of patient care. Note that 95% of U.S. hospital beds are in health care institutions accredited by the Joint Commission on Hospital Accreditation of Healthcare Organizations. Researchers at the University of Michigan School of Public Health looked to see whether such accreditation is a good predictor of the safety and quality of health care, according to a January 14 Wall Street Journal article. (The study itself-a technically sophisticated bit of data analysis-appears in the winter 2001 issue of the journal Quality Management in Health Care.) After studying 700 hospitals, they found "that even hospitals with higher-than-average rates of deaths and complications receive favorable scores" from the Joint Commission. No doubt that's related to the fact that the "commission almost exclusively relies on surveying a hospital's structure and processes to determine whether to accredit a hospital, and doesn't give any weight to performance measures such as the number of deaths or unexpected complications or the ability to adapt to the latest treatments." In other words, the accreditors look at inputs, programs and activities, not results.
Predictably, the Joint Commission fought back, insisting-watch the nuanced words-"that accreditation assures patients that a hospital 'complies with a set of standards identified by health-care professionals as important things that lead to safety and quality and care.'"
Sound familiar? "Standards identified by professionals as important things that lead to" the desired results? Trust us experts. We oughtn't be judged by what actually happens to patients. Judge us and our institutions by whether they do the things that we professionals believe contribute to the desired results.
This calls to mind the debates about whether teacher-training programs must be accredited by NCATE (which some states require), whether public school teachers themselves need to be licensed (which every state requires), and whether teachers certified by the National Board for Professional Teaching Standards should get extra pay and status on grounds that they're more effective (which more states are offering). In all these cases, the essential question is whether an inputs-and-process heavy evaluation system is a reliable way to assure satisfactory results.
When critics charge that such hoops, hurdles, input controls and professional judgments do not correlate with actual institutional effectiveness or student achievement, they get told that education should be more like medicine or it will never be a true profession. How many times have I heard that "You wouldn't want your brain operated on by an unlicensed doctor; why let your child be taught by a non-certified teacher or one who went through an unaccredited preparation program?"
Well, now we know that accreditation doesn't amount to much in hospitals, either. And it isn't just a few isolated scholars saying this. Three years ago, the Inspector General of the federal Department of Health and Human Services also faulted the Joint Commission's accreditation system for "major deficiencies" and said its reviews are "unlikely to detect substandard patterns of care or individual practitioners with questionable skills."
Second insight: as the Enron swamp deepens, we see that neither the company, nor its accountants at Arthur Anderson, nor sundry government oversight bodies did an acceptable job of protecting the public interest (which includes the interests of company shareholders and employees). But what's the remedy? And how does it differ from our approach to failing schools? I addressed this topic in the 12/13/01 Gadfly but it's worth another comment. In business, Americans take for granted that government has little direct ownership and management role and that private firms are the main actors. They're free to run themselves as they think best but not to keep secrets about how they're doing. Our economy depends not just on private ownership but also on a well-informed marketplace. Firms that are publicly traded must disclose all manner of financial and other information. Outside auditors are meant to keep the company treasurer honest and the stockholders informed. The accounting profession sets guidelines for how things are reported. And the government monitors and, when necessary, polices all this.
That system usually works fine. The economy thrives. Capitalism flourishes. And nobody proposes that the government should run the companies. In the Enron case, however, it broke down. People lied. What was disclosed wasn't the whole story. The accountants were, evidently, complicit in the misbehavior. And government oversight bodies sat on their hands.
Recriminations are now flying, Congressional hearings are underway, politics is heating up and from all sides we hear a clamor for reform of the accounting profession and its standards. Indeed, that clamor shows how important accurate information is to our economy and what a challenge it is to design and sustain (and modernize) systems that ensure that the public has (and can trust) the information it needs. It's impossible to bar all chicanery but we can do our best to create systems and rules that lead to the best data possible and to develop checks and balances that make it difficult for anyone to conceal the truth from the marketplace for long. Nobody doubts that government has a proper role in closing loopholes in reporting requirements and accounting practices that allow companies to mislead the public. (If Enron's questionable bookkeeping methods had been prohibited, we would have known far earlier about the company's problems and a lot of misery might have been averted.) But nobody, to my knowledge, has suggested that government should itself run-or replace-the companies themselves.
In K-12 education, on the other hand, we don't assume independent operation of schools in a marketplace supplied by government-mandated information. Rather, we assume direct government operation of the schools themselves by state and local "systems." We settle for bureaucracy instead of public information. That's one reason we seldom have good data about what's going on in individual schools or what results they're producing. Despite its flaws, American capitalism provides investors, customers and outside watchdogs with tons more detailed information about companies than parents, teachers or students have about their schools. In education, these data have traditionally been held close. Instead, we've been told that government operation of the schools will look after the public interest. We don't need to know much about how they're doing because supposedly we can trust the government to run them properly.
Then a few exceptions are made, such as charter schools, home schools, private schools, and outsourced public schools run by private firms.
Those exceptions don't always work well, to be sure. I've been to my share of bad charter schools and the evidence is mixed on outsourcing. In education, though, people tend to assume that, if one of the exceptions isn't working, the remedy is to put government back in direct control of the situation, or at least to make more government rules apply to it. That's because education has no effective market to solve the problem. If it did, we'd demand full information-and expect government to help us get that information-then let the market work its will. An Enron-like school would lose customers, file for educational bankruptcy and probably shut down.
Perhaps the greatest virtue of the new federal No Child Left Behind law is its earnest efforts to force better information from the education system via school report cards, "adequate yearly progress" reports at the building level, the disaggregation of test results, data on how many of a school's teachers are "highly qualified" (and how many are teaching out-of-field, etc.) and other mechanisms. Taken as a whole, it makes a valiant start at getting education consumers (and practitioners) the kinds of data that investors and customers have long expected from the companies they deal with. In that way, No Child Left Behind begins to redefine government's role in education, not just as bureaucratic overseer (though that's still there, too) but also as a marketplace facilitator and honest broker of vital data.
What's still missing in education, of course, is a true marketplace. That's where the federal legislation wimps out. So we continue with an awkward hybrid, a basically bureaucratic accountability system onto which is heaped the kinds of consumer information that would be needed by a market-style accountability system. One hopes we'll get that sorted out in the years to come.
"Hospital's Accreditation Status Proves a Poor Predictor of Care, Study Says," by Barbara Martinez, The Wall Street Journal, January 14, 2002. (available only to subscribers)
