The Education Gadfly Weekly

David T. Gordon, Editor, Harvard Education Press
January 2003

This is the first of what will doubtless prove to be numerous efforts to observe the 20th anniversary of A Nation at Risk, with a stocktaking of what has and has not been accomplished. (The Hoover Institution's Koret Task Force presents its version on February 26.) Published by the Harvard Education Press and edited by the Harvard Education Letter's David T. Gordon, it contains a preface by Patricia A. Graham and ten essays by well-known education thinkers, most of them ed-school professors, as well as a reproduction of the text of the 1983 report. There is no collective view or group statement here, just a collection of individual pieces. On the whole, they are glum assessments, perceptive about why change comes slowly to U.S. education but not very helpful about what to do differently. Individual authors make their own pleas - the three-point formula suggested by the Efficacy Institute's Jeff Howard is the sagest of these - but it adds up to no roadmap or blueprint. Note, too, that this volume is deeply system-focused and school-centered. Next to nothing is said about empowering consumers or the potential of competition-style reform to do what traditional power relationships have been so sluggish in accomplishing. Still, an hour with this book will add to your understanding of why little has been achieved in the two decades since America was told to get serious about its K-12 education system. For ordering information, go to http://www.gse.harvard.edu/~hepg/nationreformed.html.

Ronald D. Ferguson, NCREL Policy Issue 13
December 2002

In a report released in December by the North Central Regional Education Lab, Harvard professor Ronald Ferguson evaluates findings from a recent survey of secondary school students in high performing suburban school districts to help understand the racial and ethnic achievement disparities in otherwise outstanding schools. He concentrates on findings that showed that, while African American and Hispanic students were more likely than whites to respond to teacher encouragement, they were less likely to have high demands placed on them. This finding, says Ferguson, "focuses attention on the possibility that effective teacher-student relationships may be especially important resources for motivating black and Hispanic students" and helping to reduce achievement gaps within high performing schools. To narrow those gaps, Ferguson encourages schools to focus professional development strategies on improving teacher-student relationships in addition to improving content and pedagogy. He also observes that, for black and Hispanic students to achieve at the levels of their white counterparts, they will have to work harder. "After all, no runner ever came from behind by running the same speed as race leaders." To read Ferguson's analysis, go to http://www.ncrel.org/policy/pubs/html/pivol13/dec2002b.htm.

Luis Benveniste, Martin Carnoy, Richard Rothstein, RoutledgeFalmer
November 2002

Do private schools really do a better job than public? Not necessarily, say Luis Benveniste of the World Bank, Martin Carnoy of the Stanford ed school and Richard Rothstein of the Economic Policy Institute. Based on a miniscule sample of sixteen (elementary and middle) schools, all in California, they contend that private schools are very similar to public schools and that observable differences have more to do with the kinds of communities they're located in and the kids they serve than with the auspices under which they operate. "Within particular communities, similarities between schools and the problems that they confronted overwhelmed the differences." They conjecture that parents don't really value unconventional schools and that public-private differences don't correspond to differences in parent behavior or expectations. Note, though, that this book is based entirely on interviews and observations - no hard data on pupil achievement, for example - and that the authors are known for their animus toward vouchers and others policies that would ease the access of more children into private schools. My sense is that they found exactly what they sought by way of "evidence" to buttress their own dispositions. One interesting (and worrying) point does come through, however: the authors claim that it's as hard (and rare) to remove an unsatisfactory teacher in the Catholic schools they visited as in the public schools. If you want to see for yourself, the ISBN is 0415931975, the publisher is RoutledgeFalmer and you can get more information at http://www.semcoop.com/detail/0415931975.

Sara Bolt, Jane Krentz, Matha Thurlow, University of Minnesota, National Center on Educational Outcomes.
November 2002

This report by the University of Minnesota's National Center on Educational Outcomes examines state accountability systems to determine the degree to which publicly available documents clearly show whether students with disabilities are included in accountability calculations. The authors found that, for many states, publicly available information does not make clear whether disabled pupils were included or the extent to which their data are used in the state accountability systems. They note that special schools serving students with disabilities are often held to different requirements for accountability purposes. For more information on these findings, which the authors admit will likely change as states adjust to NCLB's accountability rules (as well as the upcoming reauthorization of IDEA), go to http://education.umn.edu/nceo/OnlinePubs/Technical33.htm.

Morrison Institute
January 2003

Arizona State University's Morrison Institute is the publisher of this multi-authored look at teacher supply and demand in Arizona, which concludes that the state presently has a "delicate balance" between demand and supply but no "overall shortage of teachers," though problems exist in particular fields and regions. Because shortages may worsen in the years ahead, however, the authors suggest a series of policy actions to boost supply and retention. The most interesting of these involve streamlining (or removing) certification requirements and paying teachers differentially. Into these 36 pages are crammed much data, expert analysis and sensible policy ideas. Every state would do well to produce such a study. Have a look by surfing to http://www.asu.edu/copp/morrison/TSfinal.pdf.

George L. Wimberly, ACT Policy Report
2002

Like Ronald Ferguson's piece [see "Addressing Racial Disparities in High Achieving Suburban Schools" above], this report focuses on student relationships as key to the success of African American students. Unlike Ferguson's analysis, however, this ACT report places a higher value on personal relationships that help students "see the importance of education" than on raising standards and teacher expectations for black students. While it acknowledges "a positive relationship between teachers' expectations and student achievement," the author believes that teachers' expectations should be used to "influence the type of information they convey, the opportunities they create for their students, and the values they help perpetuate" - fuzzier stuff than simply raising standards and expectations. And while the ACT study admits that "schools appear to help African Americans develop educational goals and expectations," it suggests that a lack of access to information, rather than low teacher expectations and less demanding curricula, is what keeps African American students out of college. For those interested in learning how high expectations and hard work can help reduce the black-white achievement gap, read Ferguson's piece. For those more interested in cultivating warm relationships between teachers and students, see http://www.act.org/research/policy/pdf/school_relation.pdf.

Jay P. Greene, Marcus Winters, Greg Forster, The Manhattan Institute for Public Policy
February 11, 2003

In this new Manhattan Institute study, Jay Greene and two associates conclude that high-stakes tests are accurate measures of student proficiency whose reliability is not undermined by "teaching to the test" or other strategies intended to inflate or manipulate the scores. Their method is to compare results from high and low stakes tests in nine school systems (two states, seven districts), the point being that nobody has an incentive to fudge low-stakes test scores and, therefore, if low- and high-stakes instruments yield similar results, we need not worry overmuch about the accuracy of the high-stakes kind. Their closest results come from Florida, where they found "a 0.96 correlation between high and low stakes test score levels, and a 0.71 correlation between the year-to-year gains on high and low stakes tests." Across all nine sites, they found a robust average correlation of 0.88 between achievement levels shown on the low and high stakes tests and a weaker but decent (0.45) correlation between yearly score changes on the two kinds of instruments. These findings contradict recent claims by David Berliner and Audrey Amrein that high-stakes test results are distorted. [See http://www.edexcellence.net/gadfly/issue.cfm?issue=63#924 and http://www.edexcellence.net/gadfly/issue.cfm?issue=8#372 for more on the Berliner and Amrein reports.] You can find the new Manhattan Institute study at http://www.manhattan-institute.org/cr_33.pdf.

U.S. Department of Education, Office of the Under Secretary
Prepared by Mathematica Policy Research, Inc. and Decision Information Resources, Inc.
February 2003

The Department of Education commissioned this report - the first of a trio over the next few years - to evaluate the 21st Century Learning Centers program, the primary source of federal dollars for after-school programs in 7,500 schools across the country. It examines a sample of these programs in their second and third years after the federal initiative refocused on academics in 1998. Given the size of the program - it received a $1 billion appropriation in 2002 - it's certainly worth asking whether or not it's working. So far, the results are discouraging. The test scores of participants are no better than those of similar non-participants, with a few exceptions. (Black and Hispanic middle school students did benefit slightly from the program, for example). Perhaps more surprising, student behavior, such as drug-use and study habits, did not improve and, in some areas, actually worsened. For example, participants were more likely to have tried marijuana and to sell drugs. Critics of this study will contend that an examination of a single year is not sufficient and that student behavior is difficult to measure, which may be true. Still, it's important to evaluate these initiatives and it's crucial to figure out what works and what doesn't in after-school services. Meanwhile, the Bush administration has proposed a big ($400 million) reduction in the program's 2004 budget. To read this important piece of research for yourself, visit http://www.ed.gov/pubs/21cent/firstyear/.

A review of world history textbooks used in U.S. classrooms found that they routinely sanitize the problems of Islam while treating events in Western history and Christianity more critically. The report, released last week by the American Textbook Council, notes that topics such as jihad, the advocacy of violence among militant Islamists, the record of Muslim enslavement, and the harsh subjection of women are glossed over in U.S. textbooks. The study suggests that the rosy treatment of Islam may arise from the lobbying of the Council on Islamic Education, which has sent publishers guidelines and definitions for textbooks and has protested against those that it says offend Muslims.

"Textbooks said to hide problems with Islam," by Larry Witham, The Washington Times, February 7, 2003

To download a copy of the study, Islam and the Textbooks, surf to http://www.historytextbooks.org/islamreport.pdf

Even before the No Child Left Behind (NCLB) act shot across the sky, many districts and states had embarked upon heroic efforts to identify failing schools and set them right. The underlying assumption is that external, top-down interventions can correct the shortcomings of unsuccessful schools and transform them into places where children successfully learn.

If that assumption doesn't withstand scrutiny, a lot of school-accountability schemes are in trouble and NCLB won't work very well. Such efforts presuppose that state-mandated changes can fix dysfunctional school systems and that district interventions can repair broken schools. NCLB itself sets forth a complex, seven-stage cascade of 31 separate interventions. Some involve "sunshine" - gauging pupil achievement against state standards and yearly progress goals and publishing lists of which schools do and don't measure up. Some involve freeing students to find better schools or re-deploying a portion of the federal Title I money for private tutoring and the like. Others entail school-devised "improvement plans," coupled with technical assistance from districts. After two years "under improvement," still-failing schools get designated for involuntary "corrective action" by their districts. This may mean replacing staff, restructuring schools, installing new curricula, outsourcing the schools' management, etc. The balkier the school, the more severe the intervention. And a similar cascade of top-down correctives is supposed to be undertaken by states when entire districts fail to make adequate progress.

How likely is all of this to work? That's the question Ronald C. Brady examines in a fine new report from the Thomas B. Fordham Foundation. Though NCLB as a whole is new, it turns out that just about all of its elements have been tried before, so Brady found a good deal of data spanning seventeen different interventions undertaken by at least thirty jurisdictions in 22 states, some dating back as far as 1989.

What he learned is sobering, though not catastrophic: "The intervention experience is marked more by valiant effort than by notable success. A...'success rate' of 50 percent is high and most interventions yield positive outcomes at lower rates." Sometimes, moreover, those positive outcomes are meager. In New York, for example, about half of the schools placed "under registration review" have improved enough to leave that list - but "over 80 percent of their students still 'need extra help' to meet state standards or have 'serious academic deficiencies' in the two core subjects being monitored under NCLB." Of six schools "reconstituted" by Prince George's County, Maryland, just one "was able to 'catch up' with its peer schools by virtue of the strong performance gains it made, one...appeared to be on a path to catching up...while the remaining four remained far behind the state average."

Just as troubling as the modest impact of these intervention schemes is their almost random nature. Among the seventeen strategies that Brady examined, none "resulted in compelling evidence that it is superior to other interventions in terms of effectiveness." Moreover, they're hard to implement (and the stronger the intervention, the more difficult to effect), costly, controversial, and challenging to sustain.

Brady derives a number of policy implications from this review, including the fear that NCLB expects "too much too fast" and the likelihood that "some children will still need more" than NCLB offers by way of cures for failing schools. Allowing the kids to leave is one promising option, albeit one that didn't work very well this year. Brady's report suggests that districts ought to focus on making that part work better since it seems unlikely that they'll be able to fix a lot of their faltering schools.

Which is not to say they shouldn't try. Top-down interventions sometimes manage to turn around failing schools. But we dare not take for granted that this approach will work most of the time. Thus Brady's perceptive report stands as a rebuke to those who oppose school choice on grounds that what we really need to do is focus on fixing the public schools we already have. This analysis makes plain that we don't yet know how to do that very reliably.

If you have not yet downloaded your own copy of Brady's report, you can find it on the web at http://www.edexcellence.net/doc/failing_schools.pdf. To see what Education Week had to say about the report, go to http://edweek.org/ew/ewstory.cfm?slug=21fed.h22 and scroll to the bottom of the page.

President Bush's 2004 budget previews many worthy education policy reforms, though in most cases the fine print remains to be written. Last week, I applauded the Administration's excellent Head Start initiative (http://www.edexcellence.net/gadfly/issue.cfm?issue=10#350). Let me now note one that's drawn less attention but could prove equally momentous. In the portion of the budget dealing with the Education Department's Office of Vocational and Adult Education, you will find a preview of "The Secondary and Technical Education Excellence Act of 2003," which is the administration's plan to overhaul the "Perkins Act," Washington's main vocational-education law, last revised in 1998. Uncle Sam's involvement with voc ed goes back to 1917, however, and the existing program is creaky as well as old. It still assumes that high schools should prepare some kids directly for the workforce, via a job-centered and not-very-academic curriculum, while the rest get an academic education and head toward college. Perhaps that kind of curricular tracking made sense two or three decades back, but not today. The fact is that nearly all of today's young people need a proper secondary education to equip them for a life that will include multiple jobs requiring ever more sophisticated skills, as well as one or more bouts with postsecondary education (maybe right after high school, perhaps later) and successful citizenship in a complex modern society. To the dismay of traditional vocational educators, the Bush administration's "Sec-Tech" proposal, quietly crafted by assistant secretary Carol D'Amico, picks up where NCLB leaves off. It says that U.S. high schools need to equip ALL their students with core academic competencies and that job-related training should take place AFTER high school, when it should be supplied mainly by community and technical colleges. That seems to me exactly right and fully consistent with education reformers' quickening interesting in America's barely-changed high schools. Expect fireworks on Capitol Hill from unreconstructed defenders of old-fashioned voc ed, but Dr. D'Amico and her colleagues have pointed to the future that we need to reach. You can see for yourself at http://www.ed.gov/offices/OVAE/CTE/actespeech.doc.

If you missed the national conference on teacher compensation and evaluation sponsored by CPRE (Consortium for Policy Research in Education) in November 2002, you can now access most of its presentations online. Among the highlights: economist Dan Goldhaber on the relationship between teacher labor markets and teacher quality, union leader Brad Jupp on what's new with the performance-pay project in Denver, and a CPRE paper on possible changes in pension programs to increase teacher mobility and retention. Presentations are linked to the conference agenda at www.wcer.wisc.edu/cpre/conference/conference/Nov02/agenda.asp

The CPRE-UW (CPRE at the University of Wisconsin) newsletter [available at http://www.wcer.wisc.edu/cpre/news/newsletter/feb2003.asp] also identifies some new teacher compensation projects: Kentucky is accepting RFP's from districts for 10 pilots for differentiated pay programs; Philadelphia will pilot a performance-based pay system this spring (though it's tied to teacher standards, not student performance); the Colonial School District in Pennsylvania has changed its bonus program for individual teachers into a group/school-based performance award program; several state-funded pilot programs are underway in Minnesota, where the new governor has indicated that performance pay is a high priority; and both Idaho and the Arlington, Virginia school district are looking at possible changes to teacher compensation.

Parents who call school district offices in New York City to try to transfer their children out of failing schools have a nearly 1 in 2 chance of getting the wrong information, two reporters from the New York Daily News found. They made 69 calls to school districts with failing schools--schools out of which parents are now eligible to transfer their children under the No Child Left Behind Act. In response to their queries, various district staffers claimed (incorrectly) that the failing school in question was not on the state's No Child Left Behind list for transfers, gave callers erroneous guidance ("you have to have a clear reason to move the student from the school"), told callers there was no place to transfer in the overcrowded district, suggested that the caller check back in July or next Fall (when the next round of transfers would actually be over), tried to talk callers out of seeking a transfer, or simply disconnected the callers or transferred them to extensions that no one ever answered.

"They fail to connect," by Jose Martinez and Joe Williams, New York Daily News, February 10, 2003

Anthony Bryk, Paul Peterson, and E.D. Hirsch have won the first annual Fordham prizes for excellence in education, the Thomas B. Fordham Foundation announced this week. Bryk and Peterson will split a $25,000 prize for distinguished scholarship while Hirsch will receive his own $25,000 prize for valor. The winners were selected by an independent committee from dozens of nominations received in response to a public solicitation. For details, including what makes the prize winners so outstanding, surf to http://www.edexcellence.net/template/page.cfm?id=251. And please note that nominations are now welcome for next year's prizes. (Information about submitting them can be found at the same place.)

Anthony Alvarado, brought in by Superintendent Alan Bersin to lead a curriculum overhaul in San Diego, will leave the district by September. Bersin called his departure a mutual decision. The reforms that Alvarado designed and put into place, which have included controversial new curricula, lots of professional development and "teaching coaches," have had their critics, but they seem to have gained a lot of traction in the elementary schools, where academic performance is looking up. (San Diego's high school challenge remains ahead.) Alvarado has served as "chancellor of instruction," the # 2 position in the 141,000-student district, since 1998. The combination of this veteran educator with attorney Bersin has been one of the more innovative leadership arrangements in U.S. public education.

"Chief S.D. school reformer to leave," by Chris Moran, San Diego Union-Tribune, February 5, 2003

David T. Gordon, Editor, Harvard Education Press
January 2003

This is the first of what will doubtless prove to be numerous efforts to observe the 20th anniversary of A Nation at Risk, with a stocktaking of what has and has not been accomplished. (The Hoover Institution's Koret Task Force presents its version on February 26.) Published by the Harvard Education Press and edited by the Harvard Education Letter's David T. Gordon, it contains a preface by Patricia A. Graham and ten essays by well-known education thinkers, most of them ed-school professors, as well as a reproduction of the text of the 1983 report. There is no collective view or group statement here, just a collection of individual pieces. On the whole, they are glum assessments, perceptive about why change comes slowly to U.S. education but not very helpful about what to do differently. Individual authors make their own pleas - the three-point formula suggested by the Efficacy Institute's Jeff Howard is the sagest of these - but it adds up to no roadmap or blueprint. Note, too, that this volume is deeply system-focused and school-centered. Next to nothing is said about empowering consumers or the potential of competition-style reform to do what traditional power relationships have been so sluggish in accomplishing. Still, an hour with this book will add to your understanding of why little has been achieved in the two decades since America was told to get serious about its K-12 education system. For ordering information, go to http://www.gse.harvard.edu/~hepg/nationreformed.html.

George L. Wimberly, ACT Policy Report
2002

Like Ronald Ferguson's piece [see "Addressing Racial Disparities in High Achieving Suburban Schools" above], this report focuses on student relationships as key to the success of African American students. Unlike Ferguson's analysis, however, this ACT report places a higher value on personal relationships that help students "see the importance of education" than on raising standards and teacher expectations for black students. While it acknowledges "a positive relationship between teachers' expectations and student achievement," the author believes that teachers' expectations should be used to "influence the type of information they convey, the opportunities they create for their students, and the values they help perpetuate" - fuzzier stuff than simply raising standards and expectations. And while the ACT study admits that "schools appear to help African Americans develop educational goals and expectations," it suggests that a lack of access to information, rather than low teacher expectations and less demanding curricula, is what keeps African American students out of college. For those interested in learning how high expectations and hard work can help reduce the black-white achievement gap, read Ferguson's piece. For those more interested in cultivating warm relationships between teachers and students, see http://www.act.org/research/policy/pdf/school_relation.pdf.

Jay P. Greene, Marcus Winters, Greg Forster, The Manhattan Institute for Public Policy
February 11, 2003

In this new Manhattan Institute study, Jay Greene and two associates conclude that high-stakes tests are accurate measures of student proficiency whose reliability is not undermined by "teaching to the test" or other strategies intended to inflate or manipulate the scores. Their method is to compare results from high and low stakes tests in nine school systems (two states, seven districts), the point being that nobody has an incentive to fudge low-stakes test scores and, therefore, if low- and high-stakes instruments yield similar results, we need not worry overmuch about the accuracy of the high-stakes kind. Their closest results come from Florida, where they found "a 0.96 correlation between high and low stakes test score levels, and a 0.71 correlation between the year-to-year gains on high and low stakes tests." Across all nine sites, they found a robust average correlation of 0.88 between achievement levels shown on the low and high stakes tests and a weaker but decent (0.45) correlation between yearly score changes on the two kinds of instruments. These findings contradict recent claims by David Berliner and Audrey Amrein that high-stakes test results are distorted. [See http://www.edexcellence.net/gadfly/issue.cfm?issue=63#924 and http://www.edexcellence.net/gadfly/issue.cfm?issue=8#372 for more on the Berliner and Amrein reports.] You can find the new Manhattan Institute study at http://www.manhattan-institute.org/cr_33.pdf.

Luis Benveniste, Martin Carnoy, Richard Rothstein, RoutledgeFalmer
November 2002

Do private schools really do a better job than public? Not necessarily, say Luis Benveniste of the World Bank, Martin Carnoy of the Stanford ed school and Richard Rothstein of the Economic Policy Institute. Based on a miniscule sample of sixteen (elementary and middle) schools, all in California, they contend that private schools are very similar to public schools and that observable differences have more to do with the kinds of communities they're located in and the kids they serve than with the auspices under which they operate. "Within particular communities, similarities between schools and the problems that they confronted overwhelmed the differences." They conjecture that parents don't really value unconventional schools and that public-private differences don't correspond to differences in parent behavior or expectations. Note, though, that this book is based entirely on interviews and observations - no hard data on pupil achievement, for example - and that the authors are known for their animus toward vouchers and others policies that would ease the access of more children into private schools. My sense is that they found exactly what they sought by way of "evidence" to buttress their own dispositions. One interesting (and worrying) point does come through, however: the authors claim that it's as hard (and rare) to remove an unsatisfactory teacher in the Catholic schools they visited as in the public schools. If you want to see for yourself, the ISBN is 0415931975, the publisher is RoutledgeFalmer and you can get more information at http://www.semcoop.com/detail/0415931975.

Morrison Institute
January 2003

Arizona State University's Morrison Institute is the publisher of this multi-authored look at teacher supply and demand in Arizona, which concludes that the state presently has a "delicate balance" between demand and supply but no "overall shortage of teachers," though problems exist in particular fields and regions. Because shortages may worsen in the years ahead, however, the authors suggest a series of policy actions to boost supply and retention. The most interesting of these involve streamlining (or removing) certification requirements and paying teachers differentially. Into these 36 pages are crammed much data, expert analysis and sensible policy ideas. Every state would do well to produce such a study. Have a look by surfing to http://www.asu.edu/copp/morrison/TSfinal.pdf.

Ronald D. Ferguson, NCREL Policy Issue 13
December 2002

In a report released in December by the North Central Regional Education Lab, Harvard professor Ronald Ferguson evaluates findings from a recent survey of secondary school students in high performing suburban school districts to help understand the racial and ethnic achievement disparities in otherwise outstanding schools. He concentrates on findings that showed that, while African American and Hispanic students were more likely than whites to respond to teacher encouragement, they were less likely to have high demands placed on them. This finding, says Ferguson, "focuses attention on the possibility that effective teacher-student relationships may be especially important resources for motivating black and Hispanic students" and helping to reduce achievement gaps within high performing schools. To narrow those gaps, Ferguson encourages schools to focus professional development strategies on improving teacher-student relationships in addition to improving content and pedagogy. He also observes that, for black and Hispanic students to achieve at the levels of their white counterparts, they will have to work harder. "After all, no runner ever came from behind by running the same speed as race leaders." To read Ferguson's analysis, go to http://www.ncrel.org/policy/pubs/html/pivol13/dec2002b.htm.

Sara Bolt, Jane Krentz, Matha Thurlow, University of Minnesota, National Center on Educational Outcomes.
November 2002

This report by the University of Minnesota's National Center on Educational Outcomes examines state accountability systems to determine the degree to which publicly available documents clearly show whether students with disabilities are included in accountability calculations. The authors found that, for many states, publicly available information does not make clear whether disabled pupils were included or the extent to which their data are used in the state accountability systems. They note that special schools serving students with disabilities are often held to different requirements for accountability purposes. For more information on these findings, which the authors admit will likely change as states adjust to NCLB's accountability rules (as well as the upcoming reauthorization of IDEA), go to http://education.umn.edu/nceo/OnlinePubs/Technical33.htm.

U.S. Department of Education, Office of the Under Secretary
Prepared by Mathematica Policy Research, Inc. and Decision Information Resources, Inc.
February 2003

The Department of Education commissioned this report - the first of a trio over the next few years - to evaluate the 21st Century Learning Centers program, the primary source of federal dollars for after-school programs in 7,500 schools across the country. It examines a sample of these programs in their second and third years after the federal initiative refocused on academics in 1998. Given the size of the program - it received a $1 billion appropriation in 2002 - it's certainly worth asking whether or not it's working. So far, the results are discouraging. The test scores of participants are no better than those of similar non-participants, with a few exceptions. (Black and Hispanic middle school students did benefit slightly from the program, for example). Perhaps more surprising, student behavior, such as drug-use and study habits, did not improve and, in some areas, actually worsened. For example, participants were more likely to have tried marijuana and to sell drugs. Critics of this study will contend that an examination of a single year is not sufficient and that student behavior is difficult to measure, which may be true. Still, it's important to evaluate these initiatives and it's crucial to figure out what works and what doesn't in after-school services. Meanwhile, the Bush administration has proposed a big ($400 million) reduction in the program's 2004 budget. To read this important piece of research for yourself, visit http://www.ed.gov/pubs/21cent/firstyear/.