The College Board, the Educational Testing Service, Pearson Educational Measurement, and the rest of them should be ashamed of-and held accountable for-the recent spate of screw-ups in SAT scoring, as well as the less-visible but recurrent delays and glitches in state test scoring and reporting. (One wonders how much of this has never been reported.) They are big, rich organizations that are as culpable for flaws in their products as Guidant is when its pacemakers kill people, or Ford was when its Pintos were catching fire. These education behemoths can afford to solve these problems, and they should be shown no mercy until they do
Testing will never be error-free, but, as with any other high-stakes process that we depend on-for the cleanliness of our food, the efficacy of our drugs, the safety of our airplanes, the purity of our water, the fire-resistance of our toddlers' pajamas, the veracity of corporate stock offerings, the integrity of bank accounts-it's got to be mighty damn close to perfect; close enough that we can trust it. And if government is going to deploy tests as part of its public policies, which is happening all over the land, the more so thanks to NCLB, government has a responsibility to ensure that they work as intended and are vulnerable to no more than the rarest of glitches.
There are umpteen ways of doing this, and they all require quality control measures that inevitably cost money. Conceivably, the testing industry can police itself, particularly if government requires properly "policed" tests before it accepts their results. Self-policing is always preferable to the heavy hand of government bureaucracy, but it doesn't always work. And when it doesn't, government must take a more direct role, creating the testing equivalent of an FDA or SEC. That's costly and intrusive, yes, but if Washington can send meat inspectors into virtually every slaughterhouse in the land, it can send test inspectors into state and local accountability systems and college entrance-test groups. (Tom Toch's recent report has some good ideas on this front worth considering.)
"Experts" have a role to play here, too. They spend gobs of time and energy on esoteric testing issues such as reliability, alignment, comparability, and validity. They sometimes attend to test security (how to minimize cheating and fraud). But in my experience, they pay scant heed to whether the scores are accurate. They should. Test scores need auditing-and generally accepted rules by which to audit them-just as much as corporate books and school expenditures.
Of course, the testing companies should already be doing this, checking and double-checking, submitting to independent reviews and audits, spot-checking individual score sheets, running them through twice (and through different machines), and insisting that human beings eyeball them for defects and peculiarities, just as egg inspectors and underwear-wrappers do.
I'm sure some of that already goes on. But obviously, there isn't enough. Testing firms are out to make a buck, too. Yes, even those that fly the "non-profit" flag need to accumulate surpluses to finance R & D and expansion, and to make up for deficits in other parts of their operations. They are all out to hold down costs, too, not least because the competitive bidding process that underlies much of their work rewards those who charge the least. (That's something else state and federal officials need to attend to.) In the world of college entrance testing, charging too much means some kids may not even be able to apply-or cannot afford the second and third rounds of re-testing that might boost their prospects of getting into Brown or CalTech.
Lack of competition in the testing industry makes all this worse. College applicants have at most two tests (and test suppliers) to choose between. States developing NCLB tests or high-school exit exams can turn to only a few firms. Near-monopoly, combined with penurious customers, combined with little transparency and less oversight, is a wicked blend. Drink enough of it, and sooner or later you're bound to trip and fall.
It doesn't help that the testing firms won't own up to their shortcomings. They always insist that they're on top of the problem-if they even acknowledge one-but then the problem recurs. And they make absurd excuses. In this latest round, the cake was taken by the corporate spokesman who said rainy weather had caused test papers to swell, thus throwing off the test-scanning machines and leading to lower scores. (Does dry weather cause scores to rise?)
This problem needs solving, and fast. Unsolved, it jeopardizes the entire regimen that we know as standards-based reform and results-based accountability. You can already see the threat. The anti-testing crowd is having a field day, saying, "I told you so. This is what happens when you put too much faith in tests, you wreck kids' lives for spurious reasons, you can't even trust the blasted things to be scored correctly-even if they did measure anything important, which they don't."
Yes, I paraphrase. But you get the point. The testing baby could be discarded along with the soiled bathwater that a few careless firms have allowed to accumulate-then excused away or belittled. I can hear the politicians scribbling bills, amendments, and stump speeches. Shame on them, too.
Shame on the greedy, smug, secretive firms. Shame on their customers. Shame on the experts. And shame on the government.
