The Education Gadfly Weekly

Naomi Chudowsky and Victor Chudowsky
Center on Education Policy
July 2009

Move over, Mt. Everest! The Center on Education Policy has found no proof that the so-called "plateau effect" in testing is widespread. Originally used to describe Florida's 1977-1997 test score trajectory, the term "plateau effect" is now commonly used in policy discussions when increases in test scores appear to taper off after larger initial gains. CEP set out to expand on that lone Sunshine State study; they found 16 states with 55 test score proficiency "trend lines" between 1999 and 2008 that met their criteria: The test had to remain substantially unchanged; it had to be given for 6 to 10 years starting in 1999 or later; and the state had to have kept its cut scores unchanged over that period as well. Only 15 of the 55 trajectories (or 28 percent) showed conclusive signs of a plateau for the percentage of students who scored "proficient," while 21 showed steady increases and 19 zig-zagged. The most striking conclusion--appropriately labeled an "informed conjecture"--is that NCLB may have had a lot to do with gains in 20 of the 55 trend lines between 2003 and 2004--the year when NCLB really got rolling. It's important to remember that this report looks at trends for the percent of students reaching "proficient" every year, not average scores for all pupils. So it's still possible that students are, on average, hitting a plateau on the test but there are still more kids getting over the proficiency bar every year. Confused? Dig into it yourself here.

Executive Office of the President, Council of Economic Advisers
July 2009

Foreign capital. Trade deficits. High quality education. Inventory investment. As a question in a game of "which one of these is not like the other," that one's a cinch. Yet this report from the President's economic council finds education and economics to be deeply entwined. It examines how, after the worst recession since the Great Depression, America can and should prepare its workforce for the future. In particular, jobs will increasingly require critical thinking abilities, instead of specific technical skills. But the path to gaining these skills does not lie solely in the direction of what we now know as "21st Century" skill-based standards and their ilk, explains the report. Rather, high quality primary-secondary education must focus on "strong basic skills," "quality instruction," "high standards," "rigorous assessments," and "strong accountability," too. The report also finds that the education industry is "expected to contribute most substantially to job growth," as students look to become educators and administrators; we, too, found that science and math majors in our home state of Ohio were also interested in the education field. Though this report devotes the majority of its pages to post-high school training, the bottom line is that our economic crisis should be addressed from the bottom up--and the bottom starts young. It's an important reminder from a new administration that strong standards, teacher quality, and accountability reform are not only beneficial to the individual student, but to the nation as a whole. Read the report here.

Government Accountability Office
June 2009

Never known for sugar-coating, the Government Accountability Office (GAO) made no exception in this review of education in Washington, D.C. This report was commissioned by Congress, which wanted to review the progress made since passage of the 2007 Public Education Reform Amendment Act. That act gave Mayor Adrian Fenty control over schools and heralded the arrival of his hard-hitting schools chancellor, Michelle Rhee. The GAO review used both quantitative measures (e.g., test scores) and qualitative data (e.g., lots of interviews). Analysts also looked at four other cities with mayoral control (Boston, Chicago, Cleveland, and New York City) for comparison purposes. The authors acknowledge that D.C.'s battle is steeply uphill and its myriad problems deeply ingrained. Yet they also conclude that the steps taken by the mayor and his chancellor have had mixed results. By launching so many different initiatives so quickly, Rhee may have overwhelmed the schools. And some of her reforms (especially regarding the financing and staffing of schools) have worked out to the detriment of at least a few schools. The good news is that DCPS has recognized these shortcomings and is moving to fix them. Of course it still has a long way still to go. One assumes that DCPS leaders will read it carefully. You can read it here.

President Obama's attention to high school dropout rates has brought an already-contentious issue to the national scene. The U.S. can hardly be expected to compete in a global economy with so many of its young people failing to make it to and through their senior years, or so the argument goes. "[Our] high school dropout rate has tripled since the 1970s," Obama told the Hispanic Chamber of Commerce on March 10, 2009. But had it really? Where had Obama gotten his numbers? Immediately, members of the education community disputed his figures. How we calculate the number of students dropping out and graduating is a key element to the graduation-rate debate, yet one little understood. The fact is that the exact same body of students can have vastly different completion rates, depending on how you calculate them. And No Child Left Behind's consequences for schools, districts, and states that fall short on this front make comprehending these rates that much more pressing.

That's why Fordham asked Christine O. Wolfe, herself a former federal official and Hill staffer who worked assiduously on this issue, to explain these complex formulae and the cogitation surrounding them in a new primer, titled "The Great Graduation-Rate Debate." In these twenty pages, she lays out the most commonly used rates, how they are calculated, where they get their data, and what kinds of assumptions they make. Then she discusses why some rates have prevailed over others, which are more trustworthy, and where the discussion will likely go from here.

Wolfe points out the many variables at play here: Should we count only students who graduate in four years? How do we calculate the number of first-time ninth graders? What about students with nonstandard diplomas or GEDs? How do we tally dropouts and transfer students? Answering these questions raises still more. For example, if we emphasize four-year graduations, are we encouraging "seat-time" over actual learning? And when we talk about ninth-grade enrollment, are we distinguishing between first-time ninth graders and those who are repeating this first year of high school?

These issues aren't just academic. As with many aspects of K-12 education, NCLB raised the stakes on graduation rates. Under its provisions, for the first time, states, districts, and schools must report their graduation rates, disaggregated by subgroup. The thought was that a bit of sunshine and shame should encourage states to shape up--and prevent high schools from pushing out low-achievers as a way to boost their test scores. But spotlighting graduation rates meant that even more parties grew unhappy with how they are calculated. So in 2005, states came together under the aegis of the National Governors Association and agreed to what is now called the "NGA Compact Rate." The NGA recently reported that all states are on track to report their graduation rates via this metric by 2011.

As the Bush Administration neared its close, Secretary of Education Margaret Spellings added a new twist by deciding to reinvigorate NCLB's graduation-rate regulations. Pursuant to her changes, states are required to adopt a common rate for all students, to use the same calculation formula within a few years, and to embed graduation rates--disaggregated by all of NCLB's subgroups--into measures of "Adequate Yearly Progress." In other words, to make AYP, high schools would now have to both hit test score targets for all of their subgroups, and hit graduation rate targets for all of these groups too.

These complications leave us with many remaining questions, centered mostly on how the Obama Administration and Secretary Arne Duncan will deal with graduation rates, especially as these pertain to accountability and NCLB. We've had a few hints--including Duncan's decision to support Spellings's 2008 regs--but big questions remain: Will the Department enforce the new regulations and how far will it go to intervene in states that fall short on making sure more students graduate?

As they work through these decisions, they should pay attention to some key issues. First, it is not clear that graduation rates reflect school effectiveness. Is it fair to penalize high schools when their feeder schools send them sorely underprepared students?  And by tying graduation rates to AYP metrics, aren't we encouraging states, districts, and schools to lower their diploma standards in order to get more students across the finish line? The problem of perverse incentives has plagued other aspects of NCLB, and we'd be wise to be wary of spreading it to yet another important metric--and turning a diploma into a slip of paper that means even less tomorrow than it does today.

It appears increasingly likely that President Obama and Secretary Duncan are at risk of doing to charter schooling, merit pay, and school "turnarounds" what the Bush administration did to educational accountability. That's not meant as a compliment.

The Bush team took the sensible and broadly-supported notion of holding schools accountable for their returns and then pursued a vision that is so prescriptive, so overwrought, and so divorced from a coherent rendering of what the feds can actually do that they managed to largely unravel a solid bipartisan commitment in support of the underlying idea.  As a result, most of the country wants to see NCLB overhauled or dumped outright.

What's easy to forget, of course, is that NCLB was once enormously popular. In Bush's first couple of years, it was touted as a triumph of bipartisanship and a signal accomplishment.  Similarly, Messrs. Obama and Duncan are today basking in laurels for their "Race to the Top" efforts. (This four billion dollar program, part of the stimulus package, will reward reform-minded states with big incentive grants.) But what did Bush in was not his support for accountability, which continues to enjoy broad support in principle, but his effort to force onto states a particular vision of accountability that paid too little heed to organizational dynamics, or the predictable perils of implementation.

The Bush Administration learned the hard way that, while Uncle Sam can coerce states and school districts to do things they don't want to do, he can't make them do those things well. Every state now has standards and tests in reading and math in grades three through eight, and a definition of "Adequate Yearly Progress" that meets the minimal requirements of federal law. Every state also has a system of free tutoring for poor students in schools that don't make the grade. But most of those standards and tests are set at laughably low levels, the definitions of "AYP" are riddled with holes and twisted by game-playing, and the free tutoring isn't reaching anywhere near a majority of its intended beneficiaries. States followed the letter, but not the spirit, of the law.

Some Bush officials and their Washington allies reacted by bemoaning the intransigence of states and districts. They surely had a point.  But these "frustrations" are the price of democracy, federalism, and living in a nation of laws--those same American legacies that school reformers laud when they don't find them inconvenient. Moreover, turnarounds or charter schooling will not work as intended if pursued only in response to pressure coming from Washington. This is a battle for hearts and minds, not a war of brute force.

Which brings us back to the Race to the Top. The Obama Administration has a big carrot to offer the states. It could have said to them, "Show us your best ideas for raising standards, improving teacher quality, and turning around low-performing schools, and we'll fund the most compelling and thoughtful ones." Instead, it said, "Here are our best ideas for reforming schools; the more you agree to implement these, the better your chances at getting the dough."

Particularly worrisome is President Obama's claim that the Race to the Top criteria are "evidence-based."  Measures like alternative licensure, charter schooling, and efforts to promote aggressive school restructuring are terrific ideas (and we think the administration deserves kudos for pushing them), but even we would demur from claiming that they are "evidence-based" in any meaningful sense. And the evidence on school turnarounds barely exists as yet, though research from other sectors ought to give the Secretary of Education some pause as he discusses the thousands of schools he yearns (with ample justification) to turn around. This kind of overreach and over-claiming is dishearteningly similar to the Bush-Spellings penchant to overhype NCLB's ability to boost test scores or to transform troubled schools. Over time, when the results fail to match the hype, the credibility of the reforms (and reformers) suffers and frustrated voters and policymakers find themselves inclined to toss the baby with the bathwater.

Here's what's apt to happen: States will check as many boxes as they can, make many promises they can't live up to, get the money, spend the money, and go through the motions of reforming. In other words, it will be déjà vu all over again. If these measures are implemented ham-handedly or with insufficient care, as seems likely in many places, the consequences can be severe. First, good ideas will be executed poorly, undermining support and engendering skepticism. Second, such an approach will fuel backlash. One need only recall the past decade's experience with NCLB or Reading First to know how this story goes.

There's another way, but it takes patience and perseverance--and probably longer than four or even eight years. That's to nurture the development of reform-minded political leaders and educators at the state and local levels, and to foster the efforts of entrepreneurs who are solving problems related to teacher quality, assessment, and charter schooling. This entails getting reformers elected to school boards and legislatures; appointed as advisors to governors; and elected as governors and state school chiefs, too. It requires pushing for the flexibility that will enable dynamic providers to emerge and grow.

Smart federal policy can provide "air cover" to these troops on the ground. But patience is not a virtue in Washington, and is certainly not the way of high-minded reformers eager to "save a generation of kids" or "eliminate the achievement gap." So we try to short-circuit this organic process via carrots that start to feel a lot like sticks.

Obama and Duncan have enormous credibility on education, have used it to push the envelope on important issues like charter schooling and merit pay, and are striking promising chords.  It would be a waste for those efforts to be undermined. More importantly, we'd hate to see the potential for ideas like merit pay and charter schools compromised by the familiar cycle of overselling, over-prescription, disappointment, and backlash.     

What has eluded would-be reformers in both the Bush and Obama teams is the insight that not every good idea makes for a good federal policy, and that blowing too hurriedly on a flickering fire can extinguish it rather than fuel it. A touch of humility would have been helpful during the Bush years, and that's one lesson it would be good for the Obama team to learn sooner rather than later.

Frederick M. Hess is director of education policy studies at the American Enterprise Institute and co-host of the Education Gadfly Show Podcast.

Violent video games are no new addition to the world of electronic entertainment; it seems hardly a jump, skip, or hop, then, to find iPhone apps with those same violent tendencies. But not all material is acceptable for target practice. RetardedArts, developer of the iPhone app "Zombie School," was a bit slow on this realization. (We should have been tipped off by the name of said company, but alas, we were optimistic.) This game is what it sounds. Picture Night of the Living Dead and Robin Hood rolled into one, except the zombies and their potential killers are school children. Yep, that's right, this app is about school violence, specifically human students killing zombie students with bows and arrows. Hello, lawsuit! But RetardedArts still doesn't get it: "Zombie School is not promoting school shooting; it's rather promoting elimination of zombies to protect the humans," they explained in a statement after the app was yanked. Right, except the humans are age 12 and this repugnant game eerily reminds us (and Apple, apparently) of Columbine and Virginia Tech. Good riddance.

"School-Shooting iPhone Game Removed from App Store," by Brian X. Chen, GadgetLab blog, Wired Magazine, July 20, 2009

Gadfly would be the first to admit he's gotten his wings a bit sticky over at Flypaper; the best blogging is provocative, which sometimes provokes angry reactions. Michele Kerr, a recent graduate of the Stanford Teacher Education Program, learned that lesson the hard way. A mid-life career changer, Kerr knew she didn't see eye-to-eye with STEP's social justice bent ("philosophically out of sync with the program," was how she put it), but she figured it was a good opportunity. But that didn't mean she was going to sit by quietly. Indeed, she commented on her philosophical disagreements to a program instructor at a reception for accepted students in spring 2008. And that's when the trouble started. The program first tried to keep Kerr out on "legal grounds," something she discovered when an email, meant for someone else, was mistakenly sent to her. So Kerr got a lawyer and started a blog, "Surviving Stanford," on which she shared her thoughts on a variety of education-related topics, including her disagreements with STEP's progressivist tendencies. Suffice to say, program administers were less than pleased, accusing her of a host of transgressions. They went so far as to threaten Kerr's graduation from the program by deeming her unfit for teaching, though she had good grades and was doing well at her student-teaching stint. Kerr finally did graduate but as Jay Mathews, who narrated Kerr's saga in his Washington Post column, pointed out, "In her struggle with STEP, she exposed serious problems in the way Stanford and, I suspect, other education schools, treat independent thinkers, particularly those who blog." We know nothing more than what Mathews has told us, but we do know an education gadfly when we see one. And there, at least, we can relate.

"They Messed with the Wrong Blogger," by Jay Mathews, Washington Post, July 24, 2009

Gadfly couldn't be more pleased that Hunter College ed school dean David Steiner will be moving to Albany in October as the new New York state education commissioner. Steiner brings ample reform credentials to the table. He's probably best known for helping start Teacher U at Hunter, an innovative teacher preparation program run by leaders from three of the best charter school organizations around (Uncommon Schools, KIPP, and Achievement First). He's also a big proponent of preparing teachers to teach a strong liberal arts curriculum (read his chapter in Fordham's Beyond the Basics) and he was one of the first researchers to take a long hard look at the syllabi of top-ranked ed schools. (He found that most were unfortunately dominated by constructivist and progressive thinkers.) There's a new reform Sheriff in the Empire State, and we hope he shapes up those teacher prep scalawags or makes 'em skedaddle.

"A Hunter Dean is to Lead State's Education Dept.," by Jennifer Medina, New York Times, July 28, 2009

Government Accountability Office
June 2009

Never known for sugar-coating, the Government Accountability Office (GAO) made no exception in this review of education in Washington, D.C. This report was commissioned by Congress, which wanted to review the progress made since passage of the 2007 Public Education Reform Amendment Act. That act gave Mayor Adrian Fenty control over schools and heralded the arrival of his hard-hitting schools chancellor, Michelle Rhee. The GAO review used both quantitative measures (e.g., test scores) and qualitative data (e.g., lots of interviews). Analysts also looked at four other cities with mayoral control (Boston, Chicago, Cleveland, and New York City) for comparison purposes. The authors acknowledge that D.C.'s battle is steeply uphill and its myriad problems deeply ingrained. Yet they also conclude that the steps taken by the mayor and his chancellor have had mixed results. By launching so many different initiatives so quickly, Rhee may have overwhelmed the schools. And some of her reforms (especially regarding the financing and staffing of schools) have worked out to the detriment of at least a few schools. The good news is that DCPS has recognized these shortcomings and is moving to fix them. Of course it still has a long way still to go. One assumes that DCPS leaders will read it carefully. You can read it here.

Naomi Chudowsky and Victor Chudowsky
Center on Education Policy
July 2009

Move over, Mt. Everest! The Center on Education Policy has found no proof that the so-called "plateau effect" in testing is widespread. Originally used to describe Florida's 1977-1997 test score trajectory, the term "plateau effect" is now commonly used in policy discussions when increases in test scores appear to taper off after larger initial gains. CEP set out to expand on that lone Sunshine State study; they found 16 states with 55 test score proficiency "trend lines" between 1999 and 2008 that met their criteria: The test had to remain substantially unchanged; it had to be given for 6 to 10 years starting in 1999 or later; and the state had to have kept its cut scores unchanged over that period as well. Only 15 of the 55 trajectories (or 28 percent) showed conclusive signs of a plateau for the percentage of students who scored "proficient," while 21 showed steady increases and 19 zig-zagged. The most striking conclusion--appropriately labeled an "informed conjecture"--is that NCLB may have had a lot to do with gains in 20 of the 55 trend lines between 2003 and 2004--the year when NCLB really got rolling. It's important to remember that this report looks at trends for the percent of students reaching "proficient" every year, not average scores for all pupils. So it's still possible that students are, on average, hitting a plateau on the test but there are still more kids getting over the proficiency bar every year. Confused? Dig into it yourself here.

Executive Office of the President, Council of Economic Advisers
July 2009

Foreign capital. Trade deficits. High quality education. Inventory investment. As a question in a game of "which one of these is not like the other," that one's a cinch. Yet this report from the President's economic council finds education and economics to be deeply entwined. It examines how, after the worst recession since the Great Depression, America can and should prepare its workforce for the future. In particular, jobs will increasingly require critical thinking abilities, instead of specific technical skills. But the path to gaining these skills does not lie solely in the direction of what we now know as "21st Century" skill-based standards and their ilk, explains the report. Rather, high quality primary-secondary education must focus on "strong basic skills," "quality instruction," "high standards," "rigorous assessments," and "strong accountability," too. The report also finds that the education industry is "expected to contribute most substantially to job growth," as students look to become educators and administrators; we, too, found that science and math majors in our home state of Ohio were also interested in the education field. Though this report devotes the majority of its pages to post-high school training, the bottom line is that our economic crisis should be addressed from the bottom up--and the bottom starts young. It's an important reminder from a new administration that strong standards, teacher quality, and accountability reform are not only beneficial to the individual student, but to the nation as a whole. Read the report here.