The Education Gadfly Weekly

John Cronin, G. Gage Kingsbury, Martha S. McCall, and Branin Bowe
Northwest Evaluation Association
April 12, 2005

The invaluable Northwest Evaluation Association (NWEA) has a fascinating new update on student achievement and growth under No Child Left Behind. This study has already occasioned a sky-is-falling article in the New York Times, which was strange since the study has both good and bad news regarding achievement and its authors themselves note that their data set is "broad" but "not nationally representative." That said, the results are definitely mixed. Because of its unparalleled access to student-level data in a number of states (if only such data were ubiquitous), NWEA looks not only at absolute scores but also at gains over time, giving it a depth of analysis few other outfits can achieve. This report looks at both absolute scores before and after (two years of) NCLB, as well as student gains over the same time, with an eye toward predicting whether states will meet the law's proficiency-by-2014 goal, as well as whether they are closing achievement gaps. Turns out, math and (to a lesser extent) reading proficiency rates have improved over the past two years under NCLB, but achievement growth has actually faltered. More troubling, achievement growth of ethnic subgroups is falling relative to white students. Meanwhile, the long-term trajectory is not good: NWEA analysts conclude that "If change in achievement of the magnitude seen so far continues, it won't bring schools close to the requirement of 100 percent proficiency by 2014." These data are, of course, preliminary and mixed and have to be set against other analyses showing improvements in ethnic and economic subgroups, such as the recent report from the Council of the Great City Schools (see here). But they obviously raise troubling questions. Is there a way to ensure NCLB stays on track? The report notes that students in grades with state tests perform better in terms of absolute achievement and growth, so extending the NCLB testing regime may be called for. And, as sense would suggest, high-stakes testing spurs greater achievement and growth than low-stakes testing. Fascinating, important, and compelling reading all around, even if the New York Times has overdone it. You can read it here.

"Study finds shortcomings on new law in education," by Greg Winter, New York Times, April 13, 2005.

Lynn A. Karoly, James H. Bigelow
RAND Corporation
2005

A new RAND report finds that high-quality, universal preschool for 4-year-olds in California would eventually generate between $2 and $4 for every public dollar invested. It advocates a publicly financed, voluntary program that would place highly qualified teachers in small classrooms in adequate facilities. The price tag of $1.7 billion a year would be offset by future gains. "It's a simple equation. Investing in preschool leads to a stronger workforce, better jobs, reduced juvenile crime, and an increased standard of living for all Californians," said Lois Salisbury of the David and Lucile Packard Foundation, which funded the study and favors universal preschool. While such analyses invariably deploy a zillion variables and assumptions to achieve their silver bullet outcomes, this one offers decent evidence. The problems are legion, though: how do you implement such a program, what does academically challenging mean in the pre-school context, and can a cash-strapped state possibly front the cost of such a program? (Californians will be pleased, no doubt, to know that actor Rob Reiner, Meathead of "All in the Family" fame, has launched a statewide bus tour promoting universal preschool.) As the study notes, preschool programs across the country vary dramatically in quality, and this study bases its analysis on successful programs, specifically on results from the Chicago Child-Parent Centers program, which resembles what they envision for California. Clearly, sector-wide results including Head Start programs of all stripes evidence decidedly mixed results. Done right, one can envision pre-school programs that pay off as handsomely as RAND suggests they might. That it has only intermittently been done right is a fact not really engaged by the study's authors. You can find the study here.

David Salisbury and James Tooley, editors
Cato Institute
April 2005

Cato aficionados won't be surprised that this book offers much evidence in favor of expanded school choice - or, more precisely, of creating a true marketplace in education. In the U.S., we have experimented (albeit in small doses) with any number of market-style reforms: charter schools, vouchers, public school choice, tax credits, etc. However, these schemes have typically been constructed in ways that limit their competitive impacts; while they may help participating students, their limited scale and political and fiscal constraints typically mean they can't be counted on to stimulate much change in existing schools or to spark the creation of new schools. This interesting volume culls similar lessons from Canada, Sweden, Chile, and New Zealand in particular. Canada's experience suggests that "government funding of private education . . . may make private schools more like public schools, but it also seems to make public schools more like private schools." In Sweden, "vouchers work," yet they're in danger of being stifled by a host of new regulations. The study also updates the plethora of research on Chile, noting that vouchers there improve outcomes, even when controlling for demographic factors - so long as the voucher is fully funded. There's more, including research on special education at home and abroad, and chapters discussing the relevance of international programs to the U.S. It's an accessible read that manages to unpeel much research without being boring, and backs up its pleas for choice with useful evidence. Look for it soon here. We're also anxiously awaiting the publication of James Tooley's upcoming international study of school choice, which Fordham has helped to support and which promises to be a blockbuster.

Paul E. Barton
Educational Testing Service
January 2005

In this ETS policy report, veteran analyst Paul E. Barton analyzes numerous studies and concludes that, while most states have developed academic standards, too often their tests, curricula, and performance goals are not properly aligned with those standards. This, Barton notes, will render any assessment of progress meaningless. But this "unfinished business" isn't limited to alignment. He also finds error in basing assessments of progress on a single marker (such as "proficient" or "adequate") instead of looking at the entire achievement distribution. He advocates value-added assessments to take into account a host of other factors beyond "what happens in school." And he questions the timing of assessments - at the end of the year, far too late to assist instruction during the year. To measure the outcomes of standards-based reforms properly, he argues, much work still remains. To see for yourself, visit here.

And you thought textbooks caused problems in the U.S. (see The Mad, Mad World of Textbook Adoption). This week, violent protests erupted in China upon the release of new Japanese history books that Chinese authorities claim whitewash Japanese atrocities committed on the mainland before and during World War II. The textbooks ignore Japan's seizure of some 100,000 to 200,000 "comfort women" as prostitutes for the Japanese troops, the nation's treatment of Chinese prisoners, and the Rape of Nanjing, where tens of thousands of Chinese civilians were killed. The vice chairman of the organization responsible for the textbooks did little to quell the upset when he remarked, "In actuality, there is no evidence proving that Japanese war crimes were any worse than war crimes committed by other nations." With regard to "comfort women" taken in China and Korea, he said, "prostitution in itself is a tragedy, but there is no evidence to indicate that the women were forced into it by the Japanese military. If this had been the case, I am sure [their countrymen] would have been so outraged that they would have stood up to kill all Japanese, no matter what the consequences." South Korea is also upset over the new textbooks' treatment of a string of islets in the Sea of Japan (called Takeshima in Japan and Dokdo in Korea). In 1905, Japan took the uninhabited ocean specks when it was expanding its influence in Asia, but South Korea claims historical ownership because it has essentially governed the islands since the 1950s. A new Japanese textbook claims that South Korea illegally occupies the islands. Astonishingly, the controversy has apparently threatened the quest for permanent Japanese membership on the U.N. Security Council. Chinese Premier Wen Jiabao said, "Only a country that respects history, takes responsibility for its past, and wins over the trust of the people of Asia and the world at large can take greater responsibility in the international community." Truly, the textbook is mightier than the sword.

"Japanese minister defends use of textbooks," by Mari Yamaguchi, Associated Press, April 8, 2005.

"Japan's nationalism risks its power position," by Erich Marquardt, The Power and Interest News Report, April 13, 2005.

"Textbook controversy may cost Japan permanent U.N. seat," by Patrick Goodenough, CNSNews.com, April 13, 2005.

As students at Palm Springs Middle School were being let out for the day, they encountered a big blue fish with a simple message: Please don't eat me. Freda Fish (get it, free-da-fish?) was asking students to "Look not Hook" and handed out a pamphlet from People for the Ethical Treatment of Animals, "The Secret Lives of Fish." Unfortunately for Freda, the kids weren't buying it. "What the hell is that? Get that out of here," yelled eighth-grader Hart Martinez. "To me, they're dumb," said seventh-grader Julissa Arana. But Karin Robertson, a.k.a. Freda, claimed victory: "A lot of kids had seen 'Finding Nemo' and understand that fish are individuals." Others demurred. Eighth-grader Manny Rodriguez had a message for Freda in case she thought of coming back: "I'll eat you. You're nothing but sushi to me." No word yet on when Gunter, the talking pistol who delivers NRA safety manuals, will arrive at the school.

"Students have some other fish to fry," by Rebecca Dellagloria, Miami Herald, April 5, 2005. (registration required)

George Will examines an Arizona referendum called the "65 percent rule," which reallocates school district budgets from bureaucracy to classrooms. If passed, it would require that at least 65 percent of district operational budgets be spent directly on "in the classroom" instruction - a worthy goal. But the proposal (read more here) would allow "each school board . . . to decide for itself how to spend the additional funds for the classroom," and supporters have a few suggestions: more teachers, smaller class sizes, and computers for everyone! Bad as those ideas are, we shudder to think what activities or programs many local school boards would decide are "instructional." (For the list of NCES-defined "in the classroom" activities, click here.) As savvy Gadfly readers know, hiring more teachers, not more qualified teachers, is self-defeating (see "Teacher can't teach"). Of handing out laptops like backpacks, the less said the better. The sensible George Will acknowledges that "there is scant evidence that increasing financial inputs will by itself increase a school's cognitive outputs. . . . Or that adding thousands of new teachers would do as much good as firing thousands of tenured incompetents." Exactly right.

"One man's way to better schools," by George F. Will, Washington Post, April 10, 2005.

Even as negotiators announced the first concrete details of the new Denver pay for performance plan for teachers (see here for a profile of the program), its future is in jeopardy because of a looming conflict with the local union over pay, scheduling, and curricular issues. The union has already notified the district that it might consider a strike and the two sides are set to go to arbitration. Both say the conflict should not endanger the landmark scheme, called ProComp, which will pay bonuses for working in hard-to-staff schools and for increasing student test scores, satisfactory evaluations, and further education. But one thing might endanger it: a local referendum on a property tax increase to finance the plan. "If there's unrest about our current salary system, that makes it tougher" to win the election, said Brad Jupp, a union member who was instrumental in crafting a plan and organizing teacher support for it. 

"Dispute with teachers may threaten pay plan," by Julie Poppen, Rocky Mountain News, April 7, 2005.  

"Denver union unrest may cloud future of pay plan," by Bess Keller, Education Week, April 13, 2005.

With dozens of states throwing toddler-style tantrums vis-??-vis NCLB's rules and expectations, the Bush Administration is offering them a "new, common sense approach" to compliance.

But first, the tantrum of the week: Connecticut school chief Betty J. Sternberg has sent Margaret Spellings a three-page letter demanding an apology for comments the Education Secretary made on PBS. The comments concerned Connecticut's planned NCLB lawsuit and included the phrases "soft bigotry of low expectations" and "un-American." (You can read the transcript here.) Sternberg huffs that "Anyone knowledgeable about the track record I and this department have had on relentlessly pursuing higher expectations for all of our students is appalled by your characterization of this department and Connecticut's educators." She also references Rod Paige's comments last year about the NEA-as-terrorist-organization (see here).

Put aside the appalling grammar, which makes one wonder about the "high expectations" stuff. Ms. Sternberg still doth protest too much. Perhaps all this energy might be better spent trying to close Connecticut's large and persistent black-white achievement gaps.

As for the administration's olive branch to governors and chiefs, in a high-visibility meeting with some fifteen of the latter in the solemn air of Mt. Vernon, Spellings last week pledged "additional flexibility" for states that embrace NCLB's principles while making significant efforts "to reform their education systems as a whole." (She expanded on the new approach in the Wall Street Journal; see here.) The accompanying fact sheet asserted that

The Department will consider very carefully the extent to which States demonstrate progress and effective implementation. . . . We intend to reward innovative and effective reformers and to use what we've learned from research, and the field, over the last three years. . . . We are willing to show States a more workable and informed approach on other aspects of the law, such as how students with persistent academic disabilities will be assessment (sic) and included in accountability. Another example of such flexibility could include a request for the use of growth models; or States may have their own proposals for demonstrating progress and effective implementation.

Is this pot half full or half empty?

On the positive side, the suggested quid pro quo - flexible means in return for demonstrated results - is a proven management precept; the bargain that underlies charter schools; and recalls a once-vibrant ESEA reform strategy known as "Straight A's." (See here and here.)

We should also praise the Secretary and her team for being willing to learn from experience instead of stubbornly clinging to NCLB rigidities that may be wholly unworkable and interfere with standards-based reforms that were already succeeding in some states. (Florida comes to mind.)

And yet. While it will naturally take time before we can see how this newfound flexibility plays out, several caveats need to be registered.

First, though the example offered at Mt. Vernon was a new option for testing disabled youngsters - allowing states to use modified assessments for two percent more of them, besides the one percent previously granted - it appears that most of the new flexibility will take the form of plan modifications to be negotiated state by state. In other words, while the Department may bring new decision rules into those negotiations, states have no inherent right to do anything different. They must ask for waivers, exemptions, and modifications, and the feds can say yea or nay or jerk them around for months.

Second, since the President's re-election and Ms. Spellings's ascension, the agency's approach to state deviations and waiver requests has been strikingly uneven. The Department has been tough with Connecticut and, of all places, Texas (for the scoop there, see here). But it's been lax with North Dakota, alternated between stern and accommodating with Utah, and compromised with California. Perhaps such tailoring is warranted, given evidence from several recent studies that states differ greatly in their ardor for, and success in, reforming schools and closing learning gaps. But this approach also invites politics rather than the needs of kids to dominate such decisions (as happened under Goals 2000) and threatens to turn a nationwide reform into a mixed bag of disparate state schemes.

Third, NCLB contains some important core ideas that many states and districts find onerous to implement - or simply don't agree with. Public-school choice and highly-qualified teachers are prime examples. Does the "new flexibility" mean that adult interests and bureaucratic inertia will again trump the needs of children? Recent accommodations with New York City (on choice) and several states (beginning with North Dakota) on teachers are not encouraging. (For more, see "Searching the Attic" from the National Council on Teacher Quality.)

Fourth, while one shouldn't be surprised that this team - which has what psychologists delicately call "control issues" - is doing its utmost to keep all such decisions within the executive branch, most analysts have concluded that NCLB, like every complex and ambitious new federal statute, needs amendment. In other words, the key revisions will have to be legislative, not regulatory or administrative. Though few members of Congress seem interested in tackling this project before 2007 (the "regular" reauthorization cycle), that may be too late if by then the executive branch has, in effect, weakened the law despite ample advice that it begs for strengthening in key areas. (For a lengthy discussion of needed and potentially useful reforms, see the new Koret Task Force volume, Within Our Reach, edited by John Chubb.)

Chiefs and governors are predictably grateful for the promise of NCLB flexibility, but so are school boards and teacher unions. (See, for example, here and here.) If the kids had a Washington lobby, I doubt it would be as pleased. But of course nobody represents them.

Key members of Congress aren't thrilled, either. Messrs. John Boehner and George Miller, chairman and ranking Democrat on the House education committee and major NCLB architects, issued an unusual joint statement a few hours after Secretary Spellings's Mt. Vernon expedition. Though polite, its authors were notably wary of a "smorgasbord of different waivers." In fact, they deserve to be quoted at length:

Today, the Secretary unveiled an approach that, if carried out fairly and without favoritism, could help iron out some of the difficulties in implementing the law. We agree with her that every effort must be made to ensure smooth and effective implementation, but we firmly believe that the effort must be based on the law as it is written, not on a smorgasbord of different waivers for different states and districts. In many respects, what's good for a student in Ohio should also be good for a student in California, or anywhere else in the country.

If the law is implemented with too much variety from state to state, the progress we are making on boosting achievement and improving accountability will be cut short. We cannot allow our students to pay this price. It is imperative that the Department assess flexibility requests evenly, objectively, and fairly. We believe this is achievable under the law and we urge the Secretary to make it a priority.

With too much leeway, opponents of No Child Left Behind will lobby the Secretary to act outside of the bounds of this law. We read her remarks today as a refusal to grant such wide latitude, and we hope our reading is accurate. The law must be implemented fairly and evenly across the states if it is going to continue to succeed and make a difference in children's lives.

This week, the Broad Foundation announced the five finalists for its 2005 Prize for Urban Education, the "largest education award in the country given to a single school district." The nominees are: Aldine Independent School District (near Houston), Boston Public Schools, New York City Department of Education, Norfolk (Virginia) Public Schools, and the San Francisco Unified School District. That's the fourth nomination for Boston, the third for Norfolk, and the second for Aldine. Each finalist receives $125,000 in scholarships for graduating seniors, and the big winner will receive $500,000 in scholarships. The award-winning district will be announced on September 20 in D.C.

"Broad Foundation announces finalists for 2005 Broad Prize," Broad Foundation, April 12, 2005.

Several weeks ago, we echoed The Economist in worrying that "the new SAT, with its writing requirement and junking of the analogy section, might signal a return back to something like the old WASPocracy, since it will reward students who have been rigorously coached in essay-writing" (see here). The returns are still early but we might have been right. Test prep companies report that their prot??g??s are very, very happy with results from the first offering of the newly revised SAT, which were released to test-takers this week. Paul Kanarek of Princeton Review told the newspaper Inside Higher Ed that "people are getting absurdly good results, much better than we predicted, especially on the writing test. . . . We had long suspected that the first example of the new test would be reasonably easy and that scoring was going to be generous, and it looks like that's what happened." The College Board scoffs, claiming that scale scores have remained stable from the last administration of the old test, and since no official figures are available yet no one has any way of judging. And, of course, it's possible that this reaction from test-prep companies is more in the nature of advertising than analysis. But that won't stop us from speculating!
 
"Generous curve?" by Scott Jaschik, Inside Higher Ed, April 12, 2005.

David Salisbury and James Tooley, editors
Cato Institute
April 2005

Cato aficionados won't be surprised that this book offers much evidence in favor of expanded school choice - or, more precisely, of creating a true marketplace in education. In the U.S., we have experimented (albeit in small doses) with any number of market-style reforms: charter schools, vouchers, public school choice, tax credits, etc. However, these schemes have typically been constructed in ways that limit their competitive impacts; while they may help participating students, their limited scale and political and fiscal constraints typically mean they can't be counted on to stimulate much change in existing schools or to spark the creation of new schools. This interesting volume culls similar lessons from Canada, Sweden, Chile, and New Zealand in particular. Canada's experience suggests that "government funding of private education . . . may make private schools more like public schools, but it also seems to make public schools more like private schools." In Sweden, "vouchers work," yet they're in danger of being stifled by a host of new regulations. The study also updates the plethora of research on Chile, noting that vouchers there improve outcomes, even when controlling for demographic factors - so long as the voucher is fully funded. There's more, including research on special education at home and abroad, and chapters discussing the relevance of international programs to the U.S. It's an accessible read that manages to unpeel much research without being boring, and backs up its pleas for choice with useful evidence. Look for it soon here. We're also anxiously awaiting the publication of James Tooley's upcoming international study of school choice, which Fordham has helped to support and which promises to be a blockbuster.

John Cronin, G. Gage Kingsbury, Martha S. McCall, and Branin Bowe
Northwest Evaluation Association
April 12, 2005

The invaluable Northwest Evaluation Association (NWEA) has a fascinating new update on student achievement and growth under No Child Left Behind. This study has already occasioned a sky-is-falling article in the New York Times, which was strange since the study has both good and bad news regarding achievement and its authors themselves note that their data set is "broad" but "not nationally representative." That said, the results are definitely mixed. Because of its unparalleled access to student-level data in a number of states (if only such data were ubiquitous), NWEA looks not only at absolute scores but also at gains over time, giving it a depth of analysis few other outfits can achieve. This report looks at both absolute scores before and after (two years of) NCLB, as well as student gains over the same time, with an eye toward predicting whether states will meet the law's proficiency-by-2014 goal, as well as whether they are closing achievement gaps. Turns out, math and (to a lesser extent) reading proficiency rates have improved over the past two years under NCLB, but achievement growth has actually faltered. More troubling, achievement growth of ethnic subgroups is falling relative to white students. Meanwhile, the long-term trajectory is not good: NWEA analysts conclude that "If change in achievement of the magnitude seen so far continues, it won't bring schools close to the requirement of 100 percent proficiency by 2014." These data are, of course, preliminary and mixed and have to be set against other analyses showing improvements in ethnic and economic subgroups, such as the recent report from the Council of the Great City Schools (see here). But they obviously raise troubling questions. Is there a way to ensure NCLB stays on track? The report notes that students in grades with state tests perform better in terms of absolute achievement and growth, so extending the NCLB testing regime may be called for. And, as sense would suggest, high-stakes testing spurs greater achievement and growth than low-stakes testing. Fascinating, important, and compelling reading all around, even if the New York Times has overdone it. You can read it here.

"Study finds shortcomings on new law in education," by Greg Winter, New York Times, April 13, 2005.

Lynn A. Karoly, James H. Bigelow
RAND Corporation
2005

A new RAND report finds that high-quality, universal preschool for 4-year-olds in California would eventually generate between $2 and $4 for every public dollar invested. It advocates a publicly financed, voluntary program that would place highly qualified teachers in small classrooms in adequate facilities. The price tag of $1.7 billion a year would be offset by future gains. "It's a simple equation. Investing in preschool leads to a stronger workforce, better jobs, reduced juvenile crime, and an increased standard of living for all Californians," said Lois Salisbury of the David and Lucile Packard Foundation, which funded the study and favors universal preschool. While such analyses invariably deploy a zillion variables and assumptions to achieve their silver bullet outcomes, this one offers decent evidence. The problems are legion, though: how do you implement such a program, what does academically challenging mean in the pre-school context, and can a cash-strapped state possibly front the cost of such a program? (Californians will be pleased, no doubt, to know that actor Rob Reiner, Meathead of "All in the Family" fame, has launched a statewide bus tour promoting universal preschool.) As the study notes, preschool programs across the country vary dramatically in quality, and this study bases its analysis on successful programs, specifically on results from the Chicago Child-Parent Centers program, which resembles what they envision for California. Clearly, sector-wide results including Head Start programs of all stripes evidence decidedly mixed results. Done right, one can envision pre-school programs that pay off as handsomely as RAND suggests they might. That it has only intermittently been done right is a fact not really engaged by the study's authors. You can find the study here.

Paul E. Barton
Educational Testing Service
January 2005

In this ETS policy report, veteran analyst Paul E. Barton analyzes numerous studies and concludes that, while most states have developed academic standards, too often their tests, curricula, and performance goals are not properly aligned with those standards. This, Barton notes, will render any assessment of progress meaningless. But this "unfinished business" isn't limited to alignment. He also finds error in basing assessments of progress on a single marker (such as "proficient" or "adequate") instead of looking at the entire achievement distribution. He advocates value-added assessments to take into account a host of other factors beyond "what happens in school." And he questions the timing of assessments - at the end of the year, far too late to assist instruction during the year. To measure the outcomes of standards-based reforms properly, he argues, much work still remains. To see for yourself, visit here.