The Education Gadfly Weekly

Achieve, Inc.
April 2007

Achieve, Inc. deserves kudos for this "second annual" survey of states' progress "on the alignment of high school policies with the demands of college and work," an outgrowth of the American Diploma Project and the 2005 high school summit. But applaud softly, please, because the data presented here don't show huge progress and some of them indicate progress in directions that may not bear scrutiny. Get beyond the executive summary and you will encounter glum news about how few states are really aligning their high school exit and college entrance expectations (in the sense of common "cut scores," not shared aspirational standards); how few have continuous data systems that bridge the K-12 to postsecondary divide; how few hold their high schools to account for the subsequent performance of their graduates; and more. Consider, for example, that in just one of fifty states (New York) do "postsecondary institutions find the state's end-of-course high school tests... challenging enough to determine whether incoming students are prepared to enroll in credit-bearing courses." Yes I know, it's barely two years since the summit--but it's 24 years since A Nation at Risk, which cast most of its recommendations in terms of beefing up high school expectations and (vaguely) linking them to college requirements. Achieve does good work and we at Fordham are proud of our affiliation with the American Diploma Project, but the evidence presented in this report suggests mighty slow progress by states in long-overdue directions. Read it here.

Council of the Great City Schools
April 2007

The hopeful tone of this latest installment of Beating the Odds is much the same as it's been the last six years. Once again we learn that the country's 66 Great City School districts are raising achievement levels while narrowing achievement gaps. This year's highlights: the percentages of fourth- and eight-graders at or above "proficient" in math and reading have all jumped by at least 8 percentage points and as much as 15 points since 2002. About half of the GCS districts reduced achievement gaps for poor and minority eighth-graders in math and reading; between 60 and 75 percent did so for fourth-graders. Unfortunately, like its predecessors, this report also points out that most urban districts still score below statewide averages in all areas. And the authors preface their findings with numerous important caveats, such as non-comparability of state assessments and the impossibility of testing for statistical significance due to lack of data. (We'd add another one: it's possible that the state assessments themselves are getting easier, which might help to explain these rosy findings.) Still, the authors point out that "the overall direction of the state numbers is corroborated by the most recent estimates from the National Assessment of Education Progress." And they're optimistic that states and cities will continue to improve their reporting systems. The ever-increasing volume of the city-by-city data and individual city profiles (the sections are so large this year that the council is only making them available online) suggests how quickly they're progressing. Access that material and the report itself here.

Volume II, From a World at War to the Triumph of Freedom
William J. Bennett
Thomas Nelson Publishers
2007

Volume II of Bill Bennett's fine U.S. history is out this week. Clocking in at almost 600 pages, it recounts the country's saga from World War I through Ronald Reagan. As Aspen Institute president Walter Isaacson (himself the author of a swell new biography of Einstein) says on the dust jacket, "Bill Bennett's wonderfully readable book...puts our nation's triumphs, along with its lapses, into the context of a narrative about the progress of freedom." This is not only an excellent work for anyone wanting deeper familiarity with American history--it (together with volume I, which Thomas Nelson published last year) would also be a superb textbook for an ambitious high school or college course.

Education reformers have long argued that school choice is already widespread--among the well-to-do. Foes counter that choice programs are nothing but a "life raft" to save a few while letting the ship sink. Here's a new twist: St. Louis firefighters--union members all--are steamed that state law requires them to reside in the city of their employ. With its public schools in turmoil (see here), these men and women in uniform want a life raft of their own, namely the right to move to the suburbs and take advantage of their good public schools. "I'd be living out in Fenton and sending my kids to one of the best school districts in Missouri--and not paying for it," one of the firefighters explained. Fanning the flames is a 2005 decision allowing veteran police officers to escape from city to county schools. We're all for educational freedom, so give the firefighters what they want. But how unjust if the same state legislature that recently rejected school vouchers for the city's neediest families gives the green light to this form of school choice for city employees. What's good for the rescuers is also good for those who need rescuing.

"St. Louis firefighters are battling city schools," by Jake Wagman, St. Louis Post-Dispatch, April 17, 2007

Noam Scheiber, senior editor at The New Republic, is none too pleased about what he calls the "cleverness problem" bedeviling top economic graduate schools. According to him, today's students and professors are far less interested in using the dismal science to investigate important issues (poverty, inequality, etc.) than in finding cute, clever ways to show causation between situations that have little or no practical consequence.

Demonstrating compelling causation--called clean identifying--is the holy grail of economic studies. And Scheiber isn't necessarily against it. He thinks clean identifying is well and good when applied to areas that deserve study (he gives as an example this paper about the correlation between education and future wages). But he also thinks the clean identification fetish for showing causation between unimportant, everyday occurrences has gone too far. Freakonomics might be to blame.  

He's correct in a sense. The economics that garners headlines is that of the "cute" variety. Does it really matter if diplomats' parking tickets are correlated to their country's level of corruption (see here), or that Mexican men pay prostitutes a premium for unprotected sex (see here)? Economists are supposed to be solving problems, not noting largely worthless causations. 

But in another sense, Scheiber gets it wrong, as noted by MIT economics professor Joshua Angrist. Angrist, who coauthored the aforementioned study of education's correlation to earnings, writes that he is "especially pleased when students manage to come up with clean identification." He continues: "Clean identification is not a fetish; without it, little of value is learned."

That's especially true in education, an area that has too long been plagued by studies with shoddy methodologies. (Still true, alas; see here and here.) But a substantial push is underway to make education studies more rigorous, and thus, make their results more useful and conclusive. 

Randomized trials are the gold standard for controlling causality, and often they allow for clean identification. Some education problems, however, cannot be researched using randomized trials (see here). When Angrist and coauthor Alan Krueger decided to examine whether going to school longer increased future wages, they couldn't very well compare the salaries of Ph.Ds to those of dropouts--there's no way to establish causation with such a coarse comparison. But nor could they devise a randomized trial in which they selected a group of high school students, forced half to drop out and the other to attend college and grad schools. Besides being impossible, it would be unethical.

So they had to look for a "clever" way in which natural divisions would set up the functional equivalent of a randomized trial. What they found was that, because of school attendance rules, if student A was born in January and student B in December of the same year, and both dropped out as 16-year-olds, student B was in school nearly a year longer than student A. Thus, a natural, randomized trial could be studied. And the study was clever, if not cute.

If an urge to dig for natural clean identification yields more such studies in education, and if it raises the standards of education research as a whole, so much the better. Moreover, as Freakonomics demonstrated, clever clean identification piques people's interests. And if more Americans become interested in education policy, we're sure that bodes well for reformers.

Should we care whether drug dealers still live with their mothers? If it will eventually help us build better schools, absolutely.

A couple months ago, it looked like the Boston Archdiocese was actively cooperating with charter schools. No more. With enrollment in Catholic schools flagging (in part because charters are tuition free), Cardinal Sean P. O'Malley decided to take off the gloves. The Archdiocese now refuses to lease or sell school facilities to charters. Among the victims is high-achieving Boston Collegiate Charter School, which had been in negotiations to buy a church-owned building. That's over. To his credit, however, O'Malley isn't simply making life difficult for charters. He has also enlisted top business leaders to raise money for Boston-area parochial schools and is working to make those schools more efficient. Competition, we're confident, is a good thing. If the Catholic schools are able to pull off a, err, ahem, reformation, it's apt to benefit needy kids.

"The church vs. charters," by Steve Bailey, Boston Globe, April 13, 2007

Tomorrow, the House Education and Labor Committee will hold a hearing to consider the future of the much-discussed Reading First program, a key component of NCLB. While the hearing's title promises a focus on alleged "mismanagement and conflicts of interest" within the program, members of Congress would be better advised to concentrate on the future of federal policy in the domain of primary reading.

To consider the future, we're always well advised to start with a bit of history. Where did Reading First come from?

It came, above all, from mounting concern with the educational plight of far too many of our children. Despite honorable intentions, for years our governments, our educators, our media, and our scientists have been letting down kids, particularly poor kids and their families. When over 50 percent of underprivileged children keep failing in school and dropping out, something is not working.

Any effort to address that problem must begin with reading. An enormous proportion of young Americans cannot read well enough to learn about history, math, or science. Most such kids come from disadvantaged environments and many of their parents cannot read, either. Yet when it comes to educating these children, we continue to engage in practices and programs that have had no discernible effect on improving their reading capabilities.

For far too many years, the mainstays of instructional practices were superstition, tradition, and untested assumptions about how kids learn to read. Scientific research has rarely been applied to identifying effective instructional practices despite the fact that making responsible decisions about what is effective in classrooms and good for students requires scientific evidence. Until awfully recently, the practice of education resembled the practice of medicine a century back: virtually any treatment that could be thought up was tried out.

Though pioneering experts such as the late Jeanne Chall were on this case by the mid-sixties, it wasn't until the mid-1990s that the nation's leaders actually began to consider looking at different ways of addressing reading failure. When President Clinton included the rates of such failure in his 1996 State of the Union Address, it was the first time that reading instruction was recognized as a major issue by the Federal government. That mention provided the context for the development of the Reading Excellence Act, the antecedent of Reading First.

In 1997, I received a call from the office of the chairman of the House Committee on Education and the Work Force--the very same committee that is conducting tomorrow's hearing. Chairman Bill Goodling had learned that NIH's National Institute of Child Health and Human Development was conducting research on reading development, reading difficulties, and reading instruction, and had been doing so for years. He asked me to brief him on the findings. Goodling was surprised that NICHD had studied and supported research involving over 44,000 children and adults, many for more than 20 years, and that the findings of that research were validated and published in peer-reviewed journals. He was, frankly, shocked by the amount of evidence I showed him.

I explained that the phonics vs. whole-language debate was largely a waste of time, that reading was far too complex to place into such a binary straight jacket. In fact, I noted, reading requires the development of a number of complex skills that have to be integrated and practiced constantly.

After that meeting, the Reading Excellence Act was drafted and for the first time the phrase Scientifically Based Reading Research (SBRR) was introduced into law. Unfortunately, the 1998 Act provided little professional development or technical assistance and no systematic monitoring. Many states and local school districts received funds and spent them on whatever methods of reading instruction they were already using, scientific or not.

That federal funding should be contingent on evidence of effectiveness of a reading program or instructional strategy was not considered until 2001. It was then that my colleague Bob Sweet and I recommended such a framework to the new Bush administration. The idea caught on, and Reading First was born.

Unfortunately, as the program made its way through Congress, it was watered down. Rather than funding only programs with demonstrated effectiveness, Congress opted for the much broader and looser category of programs "based on scientific research." There are many possible reasons for this change including the fact that only a handful of programs had been "proven" effective. It's possible that implementing Reading First with so few programs would not have proved practical. However, we also knew that members of Congress were heavily lobbied by developers of non-proven programs who did not want to be excluded. Many were lured by the promise of federal dollars.

This dilution by Congress has had significant negative consequences. For example, some vendors of reading programs simply changed the language in their promotional materials to create the impression that they are "based on scientific research"--without making any real changes (see here).

Even with its flaws, Reading First remains incredibly important. It encourages reading instruction that is comprehensive, that is based on scientific research, and that is taught using direct and systematic instructional principles.

How to make it better? Congress should make two key changes. First, federal funds should only be used for those programs and instructional models that have been found to be effective using experimental research designs that can determine their causal impact on student achievement in reading. "Scientifically-based" programs should be replaced with "scientifically-proven" ones. 

Second, Congress should stop dancing around the "local control" issue and simply ask a federal agency to vet the reading research and to determine, on a regular basis, which reading programs make the "scientifically-proven" cut. In other words, Congress should create for reading (and perhaps other subjects where scientific research can be done) the equivalent of  an FDA for education to ensure that states and school districts only spend their Reading First funds on interventions that have been conclusively shown to work. (The "What Works Clearinghouse" might serve as a model.)

Five years after Reading First became law, evidence is beginning to show that it's starting to move publishing companies and diagnostic assessment creators towards a higher standard (see here and here). Moreover, the intent and the language of Reading First are now contained in discussions about reading in state departments of education, school districts, and individual schools. And the educational community has been energized by Reading First and its promise that all students should be provided with research-based instruction. It is difficult to visit any school and not hear administrators and teachers discussing whether particular instructional programs and strategies have sufficient evidence of effectiveness. Several publishing companies and program developers have now invested in research initiatives to test the effectiveness of their own programs. Five years back, few programs had done anything of the sort.

It would be irresponsible to allow medical practitioners to revert to superstition, anecdote, and snake oil in the 21st century. Why would we have teachers do that?  Rather than moving backwards to the day when federal dollars flowed to any program under the sun, regardless of effectiveness, we need to continue moving forward to the day when education is truly a research-based enterprise. The life prospects of millions of children depend on it.

Achieve, Inc.
April 2007

Achieve, Inc. deserves kudos for this "second annual" survey of states' progress "on the alignment of high school policies with the demands of college and work," an outgrowth of the American Diploma Project and the 2005 high school summit. But applaud softly, please, because the data presented here don't show huge progress and some of them indicate progress in directions that may not bear scrutiny. Get beyond the executive summary and you will encounter glum news about how few states are really aligning their high school exit and college entrance expectations (in the sense of common "cut scores," not shared aspirational standards); how few have continuous data systems that bridge the K-12 to postsecondary divide; how few hold their high schools to account for the subsequent performance of their graduates; and more. Consider, for example, that in just one of fifty states (New York) do "postsecondary institutions find the state's end-of-course high school tests... challenging enough to determine whether incoming students are prepared to enroll in credit-bearing courses." Yes I know, it's barely two years since the summit--but it's 24 years since A Nation at Risk, which cast most of its recommendations in terms of beefing up high school expectations and (vaguely) linking them to college requirements. Achieve does good work and we at Fordham are proud of our affiliation with the American Diploma Project, but the evidence presented in this report suggests mighty slow progress by states in long-overdue directions. Read it here.

Council of the Great City Schools
April 2007

The hopeful tone of this latest installment of Beating the Odds is much the same as it's been the last six years. Once again we learn that the country's 66 Great City School districts are raising achievement levels while narrowing achievement gaps. This year's highlights: the percentages of fourth- and eight-graders at or above "proficient" in math and reading have all jumped by at least 8 percentage points and as much as 15 points since 2002. About half of the GCS districts reduced achievement gaps for poor and minority eighth-graders in math and reading; between 60 and 75 percent did so for fourth-graders. Unfortunately, like its predecessors, this report also points out that most urban districts still score below statewide averages in all areas. And the authors preface their findings with numerous important caveats, such as non-comparability of state assessments and the impossibility of testing for statistical significance due to lack of data. (We'd add another one: it's possible that the state assessments themselves are getting easier, which might help to explain these rosy findings.) Still, the authors point out that "the overall direction of the state numbers is corroborated by the most recent estimates from the National Assessment of Education Progress." And they're optimistic that states and cities will continue to improve their reporting systems. The ever-increasing volume of the city-by-city data and individual city profiles (the sections are so large this year that the council is only making them available online) suggests how quickly they're progressing. Access that material and the report itself here.

Volume II, From a World at War to the Triumph of Freedom
William J. Bennett
Thomas Nelson Publishers
2007

Volume II of Bill Bennett's fine U.S. history is out this week. Clocking in at almost 600 pages, it recounts the country's saga from World War I through Ronald Reagan. As Aspen Institute president Walter Isaacson (himself the author of a swell new biography of Einstein) says on the dust jacket, "Bill Bennett's wonderfully readable book...puts our nation's triumphs, along with its lapses, into the context of a narrative about the progress of freedom." This is not only an excellent work for anyone wanting deeper familiarity with American history--it (together with volume I, which Thomas Nelson published last year) would also be a superb textbook for an ambitious high school or college course.