No Child Left Behind made many promises. One of the most important of them being a pledge to Mr. and Mrs. Smith that they would get an annual snapshot of how their little Susie is doing in school.
But is NCLB-spawned information coming to Susie's parents and teachers truly reliable and trustworthy? This fourth-grader lives in suburban Detroit, and her parents get word that she has passed Michigan's state test. She's "proficient" in reading and math. Mr. and Mrs. Smith naturally take this as good news; their daughter must be "on grade level" and on track to succeed in later grades of school, maybe even go to college.
Would that it were so. Unfortunately, there's a lot that Mr. and Mrs. Smith don't know. They don't know that Michigan set its "proficiency cut score"--the score a student must attain in order to pass the test--among the lowest in the land. So Susie may be "proficient" in math in the eyes of Michigan educationists, but she still could have scored worse than almost all of the other fourth-graders in the country.
Susie's parents and teachers also don't know that Michigan has set the bar particularly low for younger students, such that Susie is likely to fail the state test by the time she gets to sixth grade--and certainly when she reaches eighth grade--even if she makes regular progress every year. And they also don't know that "proficiency" on Michigan's state tests has little meaning outside the Wolverine State's borders; if Susie lived in California or Massachusetts or South Carolina, for example, she would have missed the "proficiency" cut-off by a mile.
Mr. and Mrs. Smith have been told that Susie is "proficient." What they don't know is that "proficient," defined in this way, means little. Thus, our new study, The Proficiency Illusion, asks whether states' NCLB "cut scores" on their tests are high, low, or in between. Whether they've been rising or falling (i.e., whether it's been getting harder or easier to pass the state test). And whether they're internally consistent as between, say, reading and math, or fourth and eighth grade.
To examine states' cut scores carefully, you need a yardstick external to the state itself, something solid and reliable that state-specific results and trends can be compared with. So we turned to the Northwest Evaluation Association, which has a long-lived, rock-steady scale and a "Measures of Academic Progress" assessment used for diagnostic and accountability purposes by schools and school systems in many states. Not all states, to be sure, but it turns out that in a majority of them (26, to be precise), enough kids participate in both MAP and the state assessment to allow for useful comparisons to be made and analyses performed.
The findings of this inquiry are sobering, indeed alarming. We see that "proficiency" varies wildly from state to state, with "passing scores" ranging from the seventh (MAP) percentile to the 75th. We show that, over the past few years, twice as many states have seen their tests become easier in at least two grades. (Though we note, with some relief, that most state tests have maintained their level of difficulty--such as it is--over this period.) And we learn that only a handful of states peg proficiency expectations consistently across the grades, with the vast majority setting thousands of little Susies up to fail by middle school by aiming low in elementary school.
America is awash in achievement "data," yet the truth about our educational performance is far from transparent and trustworthy. It may be smoke and mirrors. What to conclude?
First, Congress erred big-time when NCLB assigned each state to set its own standards and devise and score its own tests; no matter what one thinks of America's history of state/local primacy in k-12 education, this study underscores the folly of a big modern nation, worried about its global competitiveness, nodding with approval as Colorado sets its eighth-grade reading passing level at the seventh percentile on the NWEA scale while South Carolina sets its at the 71st percentile. A youngster moving from middle school in Boulder to high school in Charleston would be grievously unprepared for what lies ahead. So would a child moving from third grade in Detroit to fourth grade in Albuquerque.
Second, many states are internally inconsistent, with more demanding expectations in math than in reading and with higher bars in seventh and eighth grade than in third and fourth (though occasionally it goes the other way), differences that are far greater than could be explained by conscious curricular decisions and children's levels of intellectual development. This means that millions of parents are being told that their eight- and nine-year-olds are doing fine in relation to state standards, only to discover later that (assuming normal academic progress) they are nowhere near being prepared to succeed at the end of middle school. It means that too little is being expected of millions of younger kids and/or that states may erroneously think their middle schools are underperforming. And it means that Americans may wrongly conclude that their children are doing better in reading than in math--when in fact less is expected in the former subject.
What to do? It's crazy not to have some form of national standards for educational achievement--stable, reliable, cumulative, and comparable. (This week's persuasive New York Times op-ed by Fordham trustee Diane Ravitch sketches what this might look like.)
Yet even if national standards are not in the cards in the near term, state standards need an immediate and dramatic overhaul. In our view, the place to start isn't third grade; it's the end of high school, with the expectation that students be college and work-ready when they graduate. Then everything else should be "backward mapped" so that standards in the various grades proceed cumulatively from kindergarten to graduation and it becomes possible to know at every stage along the way whether a child is or is not "on course" to meet the twelfth-grade exit expectations.
And as for NCLB, it's time for Congress to back itself out of the "100 percent proficient by 2014" provision--a mandate that is clearly dampening state expectations. Consider what an official from the Maryland Department of Education told the Washington Post in response to our new study: "We think our cut scores are reasonable for what people are being asked to do by 2014, especially given that it's for all subgroups--students who don't speak English or students with special needs." Well.
The country is ready, we submit, to begin thinking afresh about standards-based reform in general and NCLB in particular. For this enterprise not to collapse, we need standards and tests that are suitably demanding as well as stable, cumulative (all the way through high school), trustworthy, and comparable. American K-12 education is a long way from that point today.
