Almost three years ago, Fordham and the Northwest Evaluation Association published a landmark study, The Proficiency Illusion, which found that state ?proficiency cut scores? varied tremendously, not just from state to state but also within states. Cut scores for elementary school kids were lower than for middle school kids; cut scores for math were higher than for reading; cut scores tended to drift downward over time.
As in most states, there was an achievement bubble in New York State. Now there's been a correction.
This created an incredibly misleading portrait of achievement. In the report's Foreword, Checker Finn and I wrote:
What does this mean for educational policy and practice? What does it mean for standards-based reform in general and NCLB in particular? It means big trouble?and those who care about strengthening U.S. k-12 education should be furious. There's all this testing?too much, surely?yet the testing enterprise is unbelievably slipshod. It's not just that results vary, but that they vary almost randomly, erratically, from place to place and grade to grade and year to year in ways that have little or nothing to do with true differences in pupil achievement. America is awash in achievement ?data,? yet the truth about our educational performance is far from transparent and trustworthy. It may be smoke and mirrors. Gains (and slippages) may be illusory. Comparisons may be misleading. Apparent problems may be nonexistent or, at least, misstated. The testing infrastructure on which so many school reform efforts rest, and in which so much confidence has been vested, is unreliable?at best.
The study didn't examine New York, but now we know that the ?proficiency illusion? infected the Empire State, too. What's notable is that Commissioner David Steiner is willing to admit the problem and take action to fix it. According to the Times, he told his board last week: ?The word ?proficient' should tell you something, and right now that is not the case on our state tests.?
So he boosted the state's cut scores in order to ensure that a student who is deemed ?proficient? is actually on track to pass the Regents' exams in high school?and succeed in college and career. Though common sense, this move is hugely courageous, as it immediately illustrated just how far many kids and schools have to go to reach that real-world standard.
Sure enough, if you raise the bar, lots more kids will fail to meet it. So demonstrated the results released yesterday. Again, according to the Times:
This year, 61 percent of state students were deemed passing, or at grade level, in math, compared with 86 percent last year. Students also performed dismally on the English tests, with 53 percent passing, down from 77 percent.
As I told the Times, these revelations complicate the Bloomberg Administration message. That's not Mayor Bloomberg's or Chancellor Joel Klein's fault, of course. They don't control the state test, and they deserve no blame for it becoming such an unreliable indicator of achievement. Still, the sky-high proficiency rates that they've crowed about in recent years have turned out to be, well, illusory.
But that doesn't mean that Big Apple schools aren't making progress. Gotham Schools quoted Merryl Tisch, the chair of the state Board of Regents, thusly: ?If you haven't noticed that the city school system is improving, then you're walking around with blinders.? True. And the city makes a good case that, regardless of where the proficiency bar is set, its kids are making strong progress. Its average ?scale scores? in math and reading have risen significantly over the past five years, both on average and for individual ethnic groups, plus the latest results from the National Assessment of Educational Progress show significant gains from 2007 to 2009 in fourth-grade reading.
As in most states, there was an achievement bubble in New York. Now there's been a correction, and we can start fresh from a more honest and accurate position. This is good news, and another illustration of how badly the country needs standards and assessments it can believe in.
-Mike Petrilli
