This week's firestorm over the performance of charter schools can be traced to mischief by the charter-hating American Federation of Teachers and a (generally very able) New York Times reporter's susceptibility to being drawn into its web.
For months, it appears, AFT analysts have been beavering away at their own analysis of new data from the National Assessment of Educational Progress (NAEP) regarding the scores of 4th and 8th graders attending a sample of charter schools in 2003. (The 8th grade sample, for the most part, proved too small to draw conclusions.)
I had played a tiny role (with Education Leaders Council chief Lisa Keegan) in persuading the feds that charter schools deserve to be noticed by NAEP, much as private schools have long been. If they are a legitimate, durable form of schooling - after all, there are now 3000+ of them enrolling around 700,000 kids - it's important for the "Nation's Report Card" to monitor their performance.
So the schools participating in NAEP in 2003 were selected in such a way that a representative sample of charter schools with 4th grades was included. (These schools are located in six states.) The resulting reading and math scores have been sitting for nine months on the website of the National Center for Education Statistics, waiting to be mined, massaged and analyzed by anyone with the requisite prowess and motivation. Meanwhile, NCES analysts have been (slowly) working on their own report.
The "charter movement" (myself included) ought to be chagrined that the AFT did this first, because that meant the maiden NAEP-based report on charter school performance would come chock-full of union spin and political agenda.
Pause, for a moment, on the politics of this. The main thrust of the AFT press release accompanying its report is that the federal government has "repeatedly delayed" the release of the charter-school results. This despite the manifest fact that the data have been accessible to the entire world for most of a year! (Else the AFT couldn't have analyzed them.) This being Washington, the smirky, ugly implication is some sort of "cover up" of bad news by the Bush administration, which the AFT would like to depict as a major booster of this hated education reform. The Times piled on a day later with an editorial making the point explicit.
One might reasonably fault NCES for sloth in this analysis, as in others. (A recent and very interesting report on kindergarten participation was four years in production.) But the cover-up hint is ridiculous. The data have been in plain view. Besides, while the White House and Bush-Cheney team have said little about charter schools for months, guess who is urging the country to create many more of them? Yes, indeedy, the Democratic Party and Senator John Kerry, the AFT's candidate, a man who in 1998 urged that every public school become a charter school! (Today, the Senator's website says: "Enacting public school choice programs and expanding the number of charter schools in the country are important ways to provide students and families with the ability to choose the schools that best meets their needs.")
The AFT shrewdly offered Times reporter Diane Jean Schemo an exclusive, and she set to work. That included asking my opinion of some charts drawn from the report. (She didn't disclose their provenance; in retrospect I surely should have asked.) I said a lot of things but, as journalists customarily do, Ms. Schemo picked the quotes - about "dismaying" charter scores and the need for "tough love" from charter authorizers - that buttressed her story line.
That line - the headline story in Tuesday's Times - was mostly wrong, however. The lead said the NAEP-based comparison "shows charter school students often doing worse than comparable students in regular public schools."
I can scarcely count the ways that is wrong, or partly wrong, or at least misleading. But let me note five points that any fair-minded reader needs to consider.
First, on key comparisons, especially by students' race, there is no statistically significant difference between the performance of kids in charter schools and traditional public schools. This is especially salient considering how heavily charter schools are patronized by black and Hispanic families. Their kids aren't doing worse in charter schools. (Unfortunately, they're also not doing better. But they may be more content with the new schools on other grounds, such as safety, size, etc.)
Second, one-time "snapshot" data of a single cohort of kids, which is all that NAEP can supply the first time around, tell you nothing about the academic achievement of children before they entered their charter schools - and just about everyone knows that a big fraction of the youngsters enrolling in charters were already behind the education eight-ball as a result of dismal performance in previous schools. Parents whose kids are thriving in traditional public schools are not apt to move them. Those transferring their daughters and sons into charters are often desperate. (NB: As Paul Peterson and colleagues pointed out in yesterday's Wall Street Journal, if the AFT is going to ignore these "selection effects" when examining charter schools, it should also do so when looking at private schools - and admit that their superior NAEP results say something about their superior performance!)
Third, when judging a school, one ought not settle for absolute test scores alone. (This is a shortcoming of NCLB, too, at least as currently interpreted by the Education Department.) What one most wants to know about a school is how rapidly its pupils are making progress from wherever they started, i.e. how much academic value is the school itself adding. You can't determine that from NAEP data. But other studies, such as those of Tom Loveless at Brookings, indicate that charter pupils are making greater gains than their age-mates in traditional public schools, even if their absolute test scores remain too low. (I'm reliably informed that new analyses from Florida and California show the same pattern for those states' charters.)
Fourth, charter schools are astoundingly varied. We've known for ages that hanging a "charter" sign over the door doesn't assure a good school, or predict a bad school, nor can one readily generalize about them. In fact, the variability among charter schools surpasses that of regular public and private schools. That's one reason they're hard to study - because having a "charter" may be less important than the school's core mission, which might be dropout recovery, or the arts, or bilingualism, or giving new options to disabled children. Some of the best schools I've ever been in are charter schools, some of which are blowing the lid off test scores in such vexed communities as Boston, New York and Chicago. And some of the worst - and flakiest - schools I've ever been in are charter schools. Yet people are choosing them.
Fifth. Or they're not choosing them. Unlike traditional public schools, the charter movement buries its dead. (At least it does when authorizers are conscientious.) Perhaps the cheapest shot in the Times article is the suggestion that the closing of some 80 charter schools (the Center for Education Reform says the correct number is more like 300) represents some sort of institutional failure or accountability malfunction. Nothing could be further from the truth. The disappearance of unsuccessful charter schools is one of the great strengths of the whole concept. Would that it happened more often!
Bottom line: Beware teacher unions bearing gifts, especially in an election year. And don't assume that the news the Times deems fit to print is always accurate or complete.
"Nation's charter schools lagging behind, U.S. test scores reveal," by Diana Jean Schemo, New York Times, August 17, 2004 (registration required)
"Bad news on the charter front," New York Times, August 18, 2004 (registration required)
"Classes of last resort," by Floyd Flake, New York Times, August 19, 2004 (registration required)
"Charter school performance study stirs debate," by Dennis Kelly and Liz Szabo, USA Today, August 18, 2004
"Dog eats AFT's homework," by William Howell, Paul Peterson, and Martin West, Wall Street Journal, August 18, 2004 (subscription required)
"Times Crusade I - Anti-NCLB," by Mickey Kaus, Kausfiles on Slate.com, August 18, 2004 (scroll down)
"Charter schools produce strong student achievement," Center for Education Reform press release, August 17, 2004
"Paige issues statement regarding New York Times article on charter schools," U.S. Department of Education press release, August 17, 2004
"Charter Schools: Achievement, Accountability, and the Role of NCLB," Brookings Institution, 2003
"First-Ever NAEP Charter School Results Repeatedly Delayed," American Federation of Teachers press release, August 17, 2004
"Charter School Achievement on the 2003 National Assessment of Educational Progress," F. Howard Nelson, Bella Rosenberg, Nancy Van Meter, American Federation of Teachers, August 17, 2004
Education Department Spokesperson Issues Statement Regarding New York Times' Articles on Charter Schools, August 18, 2004
