The Education Gadfly Weekly

Sanjiv Jaggia and Vidisha Vachharajani, with the assistance of Joseph McCarthyBeacon Hill Institute at Suffolk UniversityMay 2004

This short but insightful paper analyzes Massachusetts district-by-district school performance (as measured by scores on the Massachusetts Comprehensive Assessment System, or MCAS) since the 1993 changes in the state's funding formula, and finds no relationship between increased spending and education results. This is no simplistic study. It takes a value-added approach, comparing changes in school spending-not absolute levels-to students' performance on the MCAS. It also evaluates four separate gauges of spending-per-pupil expenditures, teachers' average salaries, per-pupil administrative expenditures, and the student-teacher ratio. For each, however, it finds the same results: spending increases were not associated with improved performance, and in many cases were associated with worsened performance. This holds true even for class size-actually, increasing the pupil-teacher ratio had a positive effect on MCAS scores. So what does improve student performance? Unfortunately, this analysis points only to socioeconomic status. However, the authors do (briefly) highlight three districts whose MCAS performance was higher than their models would predict (given the socioeconomic status of their student bodies). This appraisal suggests that the districts fare well due to high standards and hard work. A closer look at those districts would be interesting. As would comparable studies nationwide, so that policy makers might be encouraged to spend more effectively rather than just more abundantly.  To get a copy for yourself click here.

Cynthia G. Brown, Citizens??? Commission on Civil RightsMay 2004

The "Citizens' Commission on Civil Rights" is a twenty-year-old private organization supported by the Spencer, Ford, and Hewlett foundations. Nominally bipartisan, it consists primarily of prominent Democrats and is dedicated to "monitor[ing] the civil rights policies of the federal government" and advancing the cause of civil rights. The primary author of this 130-page report is Cynthia (Cindy) Brown, a veteran civil rights advocate who during the Carter Administration held top civil-rights posts in H.E.W. and then the Education Department. This study examines implementation of the Title I public-school choice provision of NCLB in 2002-3 and 2003-4, based primarily on data from a dozen states that responded conscientiously to the Commission's request for information. Its conclusions are surprisingly bullish - surprising both because of the widely reported implementation challenges that this new program element has been encountering and because of what might have been expected to be this Commission's view of school choice. The authors found widespread parent enthusiasm for choice, albeit greater demand than supply; some communities in which sizable numbers of children have been able to opt into better-performing schools thanks to this provision; and some evidence that these choices are leading to greater racial integration. They note, however, that many "advantaged" school districts resist taking in kids from neighboring districts with many schools "in need of improvement" and lament (as do I) the weakness of NCLB's "flabby" provision for inter-district transfers. (In a word, it's optional.) A whole raft of recommendations follows for making these provisions of NCLB work better and just about all of them seem sound. In sum, a thoughtful, encouraging and helpful piece of work. You can find it online by clicking here.

Jay P. Greene and Marcus A. Winters, Manhattan InstituteMay 2004

Critics of so-called "high-stakes" high school exit exams frequently argue that making a diploma depend on passing a standardized test will increase dropout rates by squeezing out students who will find it difficult to pass. But according to Jay Greene and Marcus Winters of the Manhattan Institute, there is little hard evidence to support this theory. In their study of the impact of exit exams on graduation rates, Greene and Winters use two "of the most widely respected" methods for arriving at notoriously difficult-to-calculate graduation rates - one method developed by Greene himself, the other by the National Center for Education Statistics. According to their findings, "Both analyses show that implementing a high school exit exam has no significant effect on a state's graduation rate." Though this analysis research doesn't indicate why, they conjecture that the number of students who fail to graduate because they cannot pass the exam might be small, since "passing exit exams might require very low levels of proficiency." They also propose that, since students have several chances to pass the exams before being denied a diploma - and are generally offered additional help once they've failed - any student who is serious about graduating should be able to. These findings are certainly good news to defenders of exit exams and should give (rational) critics of "high stakes" testing a reason to reconsider. To read the report, click here.

Education WeekMay 2004

The first half of this plump and packed report focuses on the role and availability of technology globally, with in-depth discussions of countries on five continents. As you doubtless surmised, the use of technology around the world is all over the map, so to speak, and gap-riddled. An opening table compiles a range of factors into one comprehensive picture. The second half of this report focuses on the fifty states and D.C. Because those data are more readily available, this section is more complete. Nine pages of tables supply state-by-state data breakdowns. We learn, for example, that 98 percent of U.S. schools now have Internet access. But just ten states require teachers to participate in technology training, meaning that all this access to technology could lead to lessons in last year's programs instead of preparing teachers to instruct effectively using tomorrow's technology. To access a copy, surf to http://www.edweek.com/sreports/tc04/article.cfm?slug=35exec.h23.

Three bills before the California legislature would reform that state's infamous textbook adoption process. Assembly Bill 2455 aims to curb the ever-escalating cost of textbooks. Unfortunately, rather than allowing the market to determine the price of textbooks by letting schools choose for themselves what is the best value and what texts best meet their needs, this bill would add yet another layer of bureaucracy to the process. It would compel the state board of education to consider what publishers spend to research and develop new texts and to compare California textbook prices with other states to help ensure that the Golden State is getting a fair price. Senate Bill 1380 aims to give schools more flexibility to purchase a greater variety of instructional materials. Specifically, it would allow schools to seek waivers from the state board to buy multimedia tools in lieu of approved texts. Finally, SB 1510 goes a bit further to give schools the flexibility to buy texts when they deem necessary, rather than having to purchase them on a fixed schedule - English this year, math next, etc. These underwhelming changes would not go nearly far enough to reform the appalling process of textbook adoption. (For more on that substantial problem, see Diane Ravitch's A Consumer's Guide to High School History Textbooks, and The Language Police.) Still, better to see some movement on this front than total paralysis. Perhaps these cracks will in time lead to an overdo shattering of the textbook monopoly that California has done so much to create.

"State aims to reduce textbook expenses," by Jessica Portner, Mercury News, May, 23, 2004 (registration required)

The Chronicle of Higher Education has an important article on pushback from the education research community concerning the use of randomized studies of education interventions. (We learn inter alia that less than 10 percent of members of the American Education Research Association "are knowledgeable about randomized trials.") The complaints are familiar: education is too complex to study in a rigorous way, turning research into policy is difficult to do well, etc. At heart, it's the old argument: does one dare to boil something as complex and subtle as education down to grades, scores, and numbers? The problem with this argument is that the kernel of truth it contains makes the falsehood that pervades it more difficult to combat, and all the more pernicious. Of course education is different from medicine or finance. Of course psychometrics is inherently imperfect, since it deals with humans, who will often act against their own interests narrowly conceived. And of course it's hard to translate data into policy. No one but the most extreme quantifiers has ever said differently. That is no reason not to avail ourselves of the tools that rigorous research and verifiable data put at our disposal in crafting effective interventions. Moreover, the stated objections are not the real heart of the matter. Bluntly stated: Those who oppose rigorous social science research in education do so not because of their concerns about methods but because of their concerns about what that research is revealing about student performance - and what those revelations imply about programs and policies the selfsame opponents have long advocated.

"No classroom left unstudied," by David Glenn, Chronicle of Higher Education, May 28, 2004

Gadfly caused a stir in Nebraska when it criticized the state for "doing just enough to keep its federal funds while skirting the spirit of the accountability provisions" of NCLB - and the U.S. Department of Education for playing along. In "Nebraska: Mo' money, less accountability," we took issue with the state's plan to allow its 517 districts to design their own systems of standards and assessments based on "a portfolio of teachers' classroom assessments, district tests that measure how well children are meeting locally developed learning standards, a state writing test, and at least one nationally standardized test included as a reality check." As we argued at the time, this approach is not only expensive and time-consuming, but, by focusing primarily on subjective measures of student achievement, it makes comparisons across districts (a critical component of NCLB) all but impossible. (See http://www.edexcellence.net/gadfly/issue.cfm?issue=144#1771 for more.) Certainly it violates the spirit of NCLB's call for uniform statewide testing aligned with uniform statewide standards.

Taking forceful issue was Douglas Christenson, Nebraska's commissioner of education, who argued that "It's clear that you know nothing about our system of assessment and accountability" and insisted that "There are other and better ways to judge the performance of students and schools" than comparing one school to another. (See http://www.edexcellence.net/gadfly/issue.cfm?issue=148#1824.) A local superintendent also chimed in with a vigorous letter disputing both our interpretation of Nebraska developments and our concept of standards-based reform. We surmise that more than a few of his colleagues have also grumped to each other over their Omaha strip steaks.

Let us be clear. The basic disagreement between us and Nebraska education officials is not the result of our misunderstanding of the workings of the approved NCLB accountability system in the Cornhusker state. Rather, it stems from a fundamental difference of opinion over whether NCLB should serve as just another system for assessing student performance or should fulfill the more ambitious goal of ushering in an era wherein teachers and school administrators are also held accountable for how well they educate their pupils, particularly in comparison to those in other schools, districts, and states.

No Child Left Behind was designed with the express purpose of holding schools and teachers accountable for student learning. It expects each state to create an accountability plan that provides an external audit that obliges every school to demonstrate in objective fashion that it is adequately educating its students vis-??-vis statewide academic standards. That external audit is, in our view, the centerpiece of this plan. Pre- NCLB, most "successful" schools were so judged merely because they had high average test scores, or because a large number of their students went on to college, etc. Those are fine things to do, of course, but high averages can also mask serious achievement gaps within and among schools and districts. NCLB sought to right this wrong by judging schools not only by how well their students do on average, but by using an objective and consistent measure of achievement to judge how all subgroups of students within each of the state's schools are doing.

NCLB further requires states to make public information about each school's performance in order to make clear when a school is or isn't living up to the expectations enshrined in the state's academic standards. By making such information public, NCLB seeks to help parents better judge the quality of their children's school and ultimately to hold educators and elected officials responsible for the quality of education delivered to, and acquired by, their students. (Given that most Americans erroneously think their own child's school is OK, providing such information is essential to eradicating misconceptions and improving the overall quality of education.)

Recall, too, that as part of the grand NCLB bargain between Washington and the states, Title I spending (the dollars attached to NCLB compliance) has risen by 43 percent since 2001. Therefore, it seems to us a bit underhanded for states to defy the spirit of the law and yet continue to receive that money.

Yes, "defy the spirit of the law." That's what we mean to say about Nebraska. While the state has set academic standards, they are intended to serve only as a guide for local districts, which remain free to develop their own so long as these are deemed by the state department of education "equal to or more rigorous" than the state's. If we've learned anything from NCLB implementation, it's that approval processes like these are shaky and exemptions (or loose interpretations) can be counted upon to undermine the effort to guarantee that all students are being held to equivalent - and rigorous - standards.

Nebraska school districts are also able to develop their own assessment systems, which can include norm-referenced tests, criterion-referenced tests, or locally developed classroom assessments (such as student portfolios). That means students across the state are subject to very different types of assessment, some more rigorous and objective than others. This makes comparisons among schools and districts next to impossible. That's music to the ears of Nebraska educators, who, according to their "Questions and answers about Nebraska's assessment system: STARS," feel "it is not appropriate to compare school districts because they are very different." Sorry about that. One purpose of NCLB is to compare schools and districts via objective measures. Such comparisons can provide useful information about which schools are doing better, and which schools need to improve.

By allowing districts to use their own standards and subjective measures of student achievement, Nebraska is basing its system of accountability on the premise that all teachers can be counted upon to do right by their students and be trusted to judge for themselves whether they are providing an adequate education to their students. (As Dr. Christenson puts it, the system requires one "to regard the work of teachers and to trust their judgment and professionalism.") The abiding problem with such a system is that, when a teacher (or principal or superintendent) does not do his/her job, no external audit illuminates that failure.

In the real world of teaching, it's been my experience, we find good teachers and weak teachers as well as good principals and inept principals. Even among the weak and inept, however, many sincerely believe they're doing a good (or good enough) job. External, objective evaluations (from someone with no vested interest in your success or failure) can highlight inadequacies that otherwise might slip under the radar. While it's true that means watching over the shoulders of excellent, hard-working teachers and administrators, too, this seems a modest price to pay to ensure that all of a state's students get an equal chance to succeed in an increasingly competitive world.

Kathleen Porter-Magee, a former middle and high school teacher, is associate research director at the Thomas B. Fordham Foundation.

Uncle Sam isn't the only one who wants to see evidence that schools are adequately educating their students. In Boston, a group of 8th graders caused a stir by creating a guide to the city's high schools based on their average test scores, dropout rates, and attendance records. According to the Boston Globe, several students have rethought where they will enroll in the fall based on information in this guide. Fourteen-year-old Christie Andre, for example, was planning to go to Madison Park Technical Vocational High School, where most of her friends were headed, but has since decided to enroll at Boston Arts Academy. Evidently, Madison's lofty dropout rate turned Christie off. "How would I turn out going to that school?" she asked. "I didn't want to drop out." Of course, some administrators say the portrayals of their schools in this guide are unfair. Karen Daniels, headmaster of Excel High School in South Boston, whose school has MCAS scores well below the citywide average as well as high dropout rates, complains that "the data doesn't [sic] tell the story. It . . . doesn't speak to how we're trying to individualize the experience for kids." It sounds like even the kids aren't buying that anymore.

"8th-graders draft handbook for choosing Hub high schools," by Anand Vaishnav, Boston Globe, May 24, 2004 (subscription required)

Virginia, a mostly conservative state, would seem a natural environment for school choice and, in fact, polls show that many Virginians support choice as a means of injecting competition into the education system. Yet the Old Dominion has been inhospitable territory for this reform strategy, boasting one of the weakest charter laws on the books. Adam Schaeffer faults state policymakers for lacking courage and failing to present cogent policy options to Virginians. (The relative excellence of public schools in Northern Virginia's densely populated suburbs may also be a factor in the Commonwealth's disinclination to tinker with a system that serves prosperous white people reasonably well.) Schaeffer also suggests that Virginia choice advocates focus on tax credits to get around a Blaine Amendment that the state Supreme Court has interpreted strictly. An excellent example of the need for such advocates to take their arguments to the grassroots and develop strategies that respond to local/ state political realities.

"No, Virginia, there is no such thing as school choice," by Adam B. Schaeffer, Doublethink magazine, Spring 2004

New data available this week from Standard & Poor's shows that charter schools sponsored by Central Michigan University have made strong academic gains over the past three years. CMU is the largest university charter sponsor in the country, overseeing a quarter of Michigan's 202 charter schools. The data-rich reports show that 40 percent of CMU-authorized charter schools outperformed their local districts on state tests this year, up from 25 percent last year. And 70 percent of those schools saw MEAP improvement in half or more subject areas over the past year. Two lessons: 1) good authorizing (CMU is rightfully considered among the best charter sponsors in the land) really matters. 2) Good trend data - skillfully analyzed, easily accessible, and rigorously verified - is essential to supporting all such claims.

"Standard & Poor's reports demonstrate growth at schools chartered by CMU," press release from CMU Charter Schools Office, May 26, 2004

San Diego City Council member Rocky Chavez is in hot water for the unusual pitch he makes to students considering enrolling in his charter School of Business and Technology. While on a May 17th recruiting trip to Oceanside High School, according to the San Diego Union Tribune, he told students that "if they want to flip burgers and drive their mother's old car," they should stay at Oceanside. But if they want "to drive a Lexus, live in a $900,000 house, and have a boat in the harbor," they should transfer to his school. That sounds like an executable verbal contract to us; we're thinking of enrolling.

"Charter school can resume recruiting, but director can't," by Lola Sherman, San Diego Union Tribune, May 25, 2004

Cynthia G. Brown, Citizens??? Commission on Civil RightsMay 2004

The "Citizens' Commission on Civil Rights" is a twenty-year-old private organization supported by the Spencer, Ford, and Hewlett foundations. Nominally bipartisan, it consists primarily of prominent Democrats and is dedicated to "monitor[ing] the civil rights policies of the federal government" and advancing the cause of civil rights. The primary author of this 130-page report is Cynthia (Cindy) Brown, a veteran civil rights advocate who during the Carter Administration held top civil-rights posts in H.E.W. and then the Education Department. This study examines implementation of the Title I public-school choice provision of NCLB in 2002-3 and 2003-4, based primarily on data from a dozen states that responded conscientiously to the Commission's request for information. Its conclusions are surprisingly bullish - surprising both because of the widely reported implementation challenges that this new program element has been encountering and because of what might have been expected to be this Commission's view of school choice. The authors found widespread parent enthusiasm for choice, albeit greater demand than supply; some communities in which sizable numbers of children have been able to opt into better-performing schools thanks to this provision; and some evidence that these choices are leading to greater racial integration. They note, however, that many "advantaged" school districts resist taking in kids from neighboring districts with many schools "in need of improvement" and lament (as do I) the weakness of NCLB's "flabby" provision for inter-district transfers. (In a word, it's optional.) A whole raft of recommendations follows for making these provisions of NCLB work better and just about all of them seem sound. In sum, a thoughtful, encouraging and helpful piece of work. You can find it online by clicking here.

Education WeekMay 2004

The first half of this plump and packed report focuses on the role and availability of technology globally, with in-depth discussions of countries on five continents. As you doubtless surmised, the use of technology around the world is all over the map, so to speak, and gap-riddled. An opening table compiles a range of factors into one comprehensive picture. The second half of this report focuses on the fifty states and D.C. Because those data are more readily available, this section is more complete. Nine pages of tables supply state-by-state data breakdowns. We learn, for example, that 98 percent of U.S. schools now have Internet access. But just ten states require teachers to participate in technology training, meaning that all this access to technology could lead to lessons in last year's programs instead of preparing teachers to instruct effectively using tomorrow's technology. To access a copy, surf to http://www.edweek.com/sreports/tc04/article.cfm?slug=35exec.h23.

Jay P. Greene and Marcus A. Winters, Manhattan InstituteMay 2004

Critics of so-called "high-stakes" high school exit exams frequently argue that making a diploma depend on passing a standardized test will increase dropout rates by squeezing out students who will find it difficult to pass. But according to Jay Greene and Marcus Winters of the Manhattan Institute, there is little hard evidence to support this theory. In their study of the impact of exit exams on graduation rates, Greene and Winters use two "of the most widely respected" methods for arriving at notoriously difficult-to-calculate graduation rates - one method developed by Greene himself, the other by the National Center for Education Statistics. According to their findings, "Both analyses show that implementing a high school exit exam has no significant effect on a state's graduation rate." Though this analysis research doesn't indicate why, they conjecture that the number of students who fail to graduate because they cannot pass the exam might be small, since "passing exit exams might require very low levels of proficiency." They also propose that, since students have several chances to pass the exams before being denied a diploma - and are generally offered additional help once they've failed - any student who is serious about graduating should be able to. These findings are certainly good news to defenders of exit exams and should give (rational) critics of "high stakes" testing a reason to reconsider. To read the report, click here.

Sanjiv Jaggia and Vidisha Vachharajani, with the assistance of Joseph McCarthyBeacon Hill Institute at Suffolk UniversityMay 2004

This short but insightful paper analyzes Massachusetts district-by-district school performance (as measured by scores on the Massachusetts Comprehensive Assessment System, or MCAS) since the 1993 changes in the state's funding formula, and finds no relationship between increased spending and education results. This is no simplistic study. It takes a value-added approach, comparing changes in school spending-not absolute levels-to students' performance on the MCAS. It also evaluates four separate gauges of spending-per-pupil expenditures, teachers' average salaries, per-pupil administrative expenditures, and the student-teacher ratio. For each, however, it finds the same results: spending increases were not associated with improved performance, and in many cases were associated with worsened performance. This holds true even for class size-actually, increasing the pupil-teacher ratio had a positive effect on MCAS scores. So what does improve student performance? Unfortunately, this analysis points only to socioeconomic status. However, the authors do (briefly) highlight three districts whose MCAS performance was higher than their models would predict (given the socioeconomic status of their student bodies). This appraisal suggests that the districts fare well due to high standards and hard work. A closer look at those districts would be interesting. As would comparable studies nationwide, so that policy makers might be encouraged to spend more effectively rather than just more abundantly.  To get a copy for yourself click here.