The miserable failure of most states to implement the requirements of the 1994 federal education amendments in timely fashion had already cast a veil of doubt over the prospects for No Child Left Behind: the stark fact that states don't necessarily make the changes that Washington expects of them-and then get away with it.
But what happens when states do comply with the formal requirements of federal legislation, yet do so in such a way that they defeat its main purpose?
No Child Left Behind allows for that possibility, via one of its central "federalism" features: the expectation that every state will set its own standards and (with Washington's approval) select and score and report the results of its own tests. The only uniform requirement is that every state has the same twelve years to get all its pupils up to the level that it designates as "proficient."
Many have noted that this arrangement could easily encourage low standards, i.e. that a state with lower standards has a better chance of making "adequate yearly progress" toward, and getting everybody up to, those standards within the prescribed dozen years than a state with loftier standards. It's obvious: if you set the bar lower, more people will successfully jump over it without having to struggle very hard.
But what, exactly, does "low standards" mean and by what mechanisms could a state, intentionally or inadvertently, end up with them? Who would know that this was occurring? How could one tell?
It turns out that education bars can be set low via more mechanisms than you might suppose-and that it's not always easy to know when this is happening. I can think of four ways that state standards might end up being low in practice, yet only one of these will be readily apparent to outsiders unless the state discloses complete information about its tests, what's on them, what the actual test questions are, how they're scored and how a student's or school's performance is calculated. (And, of course, unless someone closely analyzes the information that the state makes available.) Note that three of these four paths to low standards involve tests as well as academic standards, and be aware that tests (and their scoring) are usually far more elusive than the standards themselves in the eyes of outside analysts and whistle blowers. Many states simply don't release them, or release only portions of them, or release the tests but not the scoring information.
Here are the four ways:
First and most obvious, the published academic standards are themselves undemanding. Their content is easy. They expect little by way of skills and knowledge from students at various grade levels. Third grade math standards, for example, are limited to addition and subtraction, not multiplication and division. When multiplication is reached, only one or two digit numbers must be multiplied. Fractions are never complex, numbers never irrational. On the English standards, sentences are never compound, adjectives and adverbs never need to be distinguished, reading passages are simple, words are short. Precisely because this version of low standards is so apparent to anyone who looks-the standards, typically, are posted on the Internet-and because various organizations (e.g. the American Federation of Teachers, from time to time the Thomas B. Fordham Foundation) know how to find and appraise them, states may be too shrewd to follow this path.
Second, though the standards cover the full spectrum of skills and knowledge that well-educated students should possess, the tests focus on the easy end of that spectrum. In other words, they're poorly aligned with the standards, and the mis-match leads to an over-concentration on the more basic skills and rudimentary knowledge. Thus, for example, the posted math standards for 8th grade include algebra yet most of the test questions deal with arithmetic. This means one could do well on the test without knowing algebra. The English standards may expect students to explain the plot of a work of fiction, but the reading passages on the test are pegged at the level of Harry Potter rather than Lord of the Flies. Nor would anyone know this unless they could inspect the actual test items-all of them, or at least a representative sampling, not just a handful that the state selects for release.
Third, the test is nicely aligned with the standards and the questions thus "cover" the right skills and content, but the individual test items are easy to answer correctly. If they're multiple-choice questions, the "distractors" are ridiculous, making even a weak student more likely to select the correct response. If they're "open response" items, the "scoring rubric" is simplistic, such that a student can receive full credit for a superficial answer. (For example, he explains that the Constitution establishes our framework for government but says nothing about its key principles and provisions.) If a state (or its testing contractor) used this ploy, it would be hard for outsiders to detect unless they had access both to the test questions (and possible answers) AND the scoring rubrics.
Fourth, the state's standards are solid, the tests are properly aligned with them and the questions are suitably challenging-but the "cut score" is set low. A student might, for example, have to answer only half the questions correctly to be said to have "passed" the test-and those could be the easier half of the questions dealing with the easier half of the standards. (A variant: the "proficiency" score for individuals is appropriate, but only a smallish fraction of the kids in a school-or demographic subgroup-needs to attain that score for the larger unit to be deemed to meet the standards.) This gambit would only be visible if one had access to information about passing scores and how they're calculated for individuals, schools and groups, and that information would be truly helpful only if one also could inspect the test items and scoring guides (and could "map" them onto the state's academic standards).
How to guard against such eventualities leading to a cheapening of standards and evasion of the point of NCLB? One might trust the state and its testing contractors to do the right thing, although in a high-stakes era, when political reputations (and future contracts) hinge on these results, that would seem risky. One could replace state-determined standards and tests with a single national set-but in the present political context that seems entirely unthinkable and many believe it's undesirable. One can wait for NAEP, in its new role as external auditor of state academic performance, to reveal discrepancies and then poke around for explanations in individual states-but that'll take quite a while, and NAEP has its own vulnerabilities, which may be compounded by the pending NAEP-OERI reauthorization. Or one can rely on sunlight and external scrutiny of state standards and tests, but that is realistic only if the state accountability system is highly transparent, if tests are fully released (which is costly), if scoring guides and rubrics are in the public domain, and if outside groups know how to monitor all these moving parts and make sense of how they fit together.
Not a reassuring picture, eh?
Please suggest additional solutions-and help us alert Gadfly readers to other ways of manipulating the system, frustrating the honorable goal of leaving no child behind and, thereby, continuing to leave millions of young Americans at risk. We will consider for publication in this space any cogent and well-written submissions.
