The Education Gadfly Weekly

Lee D. Mitgang, The Wallace Foundation
2003

The Wallace Foundation went to a lot of bother and expense to produce and publicize this 11-page policy brief, which is interesting as far as it goes - but doesn't go nearly far enough. Based on three separate studies commissioned by the foundation, it mainly concludes that we don't have an overall national shortage of people willing to take principals' jobs (though some places have a supply problem), but that the current pipeline doesn't do a good job of delivering first rate leaders into manageable jobs that enable them to boost student achievement. The implications, says author Lee Mitgang, are (1) to adjust incentives and working conditions, especially in places that aren't getting the principals they need; (2) to adjust recruitment and hiring practices to conform to "heightened expectations about principal performance"; and (3) to redefine the principal's job itself. So far, so good, and it may even remind you of the new Fordham-Broad "manifesto" called "Better Leaders for America's Schools." (If you haven't seen - and signed - that document, please go immediately to http://www.edexcellence.net/detail/news.cfm?news_id=1 and do so!) What's disappointing about the Wallace report is that it barely hints at the two touchy domains where the biggest and most contentious changes need to be made if schools are truly to get the kinds of leaders they need: the training and certification of elementary and secondary principals. Why so timid? You can get to the report at http://www.wallacefunds.org/frames/framesetnews.htm. Perhaps more valuable, you can obtain the three Wallace-funded studies that fed into it. The first is "A Matter of Definition: Is There Truly a Shortage of School Principals?" at http://www.crpe.org/pubs/pdf/mroza_princshortagewb.pdf. Second, "Who Is Leading Our Schools?: An Overview of School Administrators and Their Careers," at http://www.rand.org/publications/MR/MR1679. Finally, look up "The Attributes and Career Paths of Principals: Implications for Improving Policy" at http://www.teacherpolicyresearch.org/Career%20Paths%20of%20Principals.PDF.

Paul E. Peterson and William G. Howell, Harvard University
June 12, 2003

It's round three of the ongoing Harvard-Princeton debate over the effects of a privately funded New York City voucher program. In the first round, Harvard researchers Paul Peterson, William Howell, and others conducted an original study on the effects of the voucher program on low-income children who changed schools with its assistance, compared with a control group of similar youngsters who did not change schools. Based on their analysis of the data, Peterson et al. concluded that participation in the voucher program had positive effects on the test scores of African Americans but no effects, positive or negative, on the scores of Hispanic and white students. These initial conclusions offered tentative support to the idea that vouchers might provide poor students, particularly African Americans, with a better alterative to public schools. Princeton researchers Alan B. Krueger and Pei Zhu reanalyzed the data and concluded that the initial study was methodologically flawed because of the way it determined whether a student was black and because the researchers failed to include students for whom there was no baseline data. In their reanalysis, Krueger and Zhu found that that "the safest conclusion is probably that the provision of vouchers did not lower the scores of African American students." After the New York Times made a big deal of this, Peterson and Howell undertook their own reanalysis of the data. This time, they employed both their initial methodology and Krueger and Zhu's methodology. They (again) found that "all the standard ways of estimating voucher effects showed significantly positive effects" for African American students and (again) that "no effects, positive or negative, were observed for Hispanic students or for students who were members of other ethnic groups." Besides confirming Peterson and Howell's initial findings, this report explains the shortcomings in Krueger and Zhu's methodology and conclusions. The study is being released today at the National Press Club, and will be available online at http://www.ksg.harvard.edu/pepg/.

One more area in which American schools and districts are less than transparent: the budget. In this working paper, presented at a recent Brookings conference on "The Teachers We Need," the University of Washington's Marguerite Roza and Paul Hill work from the bottom up to construct real school-by-school teacher salary figures in four districts, rather than relying on district-wide averages. They conclude that there are wide variations in per-school spending within districts that are masked by these averages, with higher-achieving schools in wealthier areas taking a disproportionate bite of the total teacher salary budget line. This hurts students in low-achieving schools by ensuring that they are taught mostly by younger, less experienced teachers who are apt to move on as soon as they accrue the requisite seniority. Roza and Hill call for more transparent budget practices and accurate reporting of per-school spending, but fall just a bit shy of urging the obvious solution: real incentive pay for teachers, experienced or not, who choose to work in tough schools. However, they do hint - in a masterpiece of understatement - that tackling this problem "would not be politically neutral." Among other reasons, this paper is worth reading for the unintended comedy in the introduction, which recounts some of the difficulties reform-minded superintendents face in trying to obtain accurate information on per-school spending in their very own districts. To check it out, go to http://www.crpe.org/workingpapers/pdf/Roza-Hill.pdf.

Jay Greene and Greg Forster, Manhattan Institute's Center for Civic Innovation
June 2003

Jay Greene and Greg Forster authored this new report from the Manhattan Institute's Center for Civic Innovation. It's timely because the Florida program that they analyzed is the nearest thing in America to a voucher program for disabled youngsters and, despite recent rejection of this approach by the U.S. House of Representatives, it remains a topic of intense interest as Congress reauthorizes the Individuals with Disabilities Education Act (IDEA). With 9,200 Florida youngsters using McKay Scholarships to attend private schools (and several hundred thousand more eligible for them), how this program is working is a matter of no little significance. According to Greene and Forster's data (based on surveys of parents of current and past scholarship users), it's working pretty well. "Virtually all measurements," they say, "showed higher levels of satisfaction, better provision of services, and better student environments at McKay schools than at public schools, and in almost all cases members of different racial, income, and disability groups showed no significant differences in their experiences." Left for another day is the question that special-ed evaluators find toughest to tackle: how much academic progress are the children making? You can find this 40-page report at http://www.manhattan-institute.org/html/cr_32.htm. - Chester E. Finn, Jr.

"New report finds significant benefits, parental satisfaction with Florida's innovative McKay scholarships," Press release of the House Committee on Education and the Workforce, June 11, 2003, http://edworkforce.house.gov/press/press108/06jun/manhattan061103.htm.

This month and last, as also happened during May-June 2002 in what threatens to become a new seasonal ritual, efforts to adopt state graduation testing are under fire. Nevada has announced that 12 percent of its 12th graders may not graduate. Florida reports that 13,000 students may be denied diplomas, while the number in Massachusetts is said to be 5,000. Parents are reportedly furious. Civil rights groups are threatening boycotts. In Massachusetts, seven communities have announced their intention to ignore state guidelines governing the issuance of diplomas. Everywhere, media outlets are highlighting heart-wrenching tales of "B" students who can't pass the math graduation exam, with suggestions that the exam systems are themselves flawed.

In fact, these exit exams are pitched at a rather modest level and offer students multiple chances to pass, which leads to an entirely different explanation: perhaps a nontrivial number of students actually lack mastery of essential knowledge and skills. If "B" students are failing, either their school grades are too high or they have not learned basic content. Either way, merely packing them off into the adult world of college or work does them no favors and ensures that our schools will continue to shortchange a new generation of students.

These high-profile complaints coexist with polls showing that more than 80 percent of Americans support high-school graduation testing. The result has been the inevitable call for appropriate "refinements," spearheaded by "temperate" critics.

This is a siren's song. Standards-based reform is alluring because it promises that all graduates will master critical knowledge and skills. Setting bona fide performance standards makes it inevitable that some students (and schools) will fail to meet them. This poses a daunting political challenge in a democratic society where those who fail have specific incentives to challenge the legitimacy of the system, while the larger community will little notice the diffuse benefits of better-prepared graduates and more demanding schools.

Simply put, there are two kinds of accountability: coercive and suggestive. Coercive accountability uses incentives and sanctions (like withholding diplomas) to ensure that students master specified content and that educators teach that content. In such a system, school performance no longer rests upon fond wishes and good intentions.

Suggestive accountability seeks to improve schooling through informal social pressures and norms, by using tests as a diagnostic device, and by increasing coordination across schools and classrooms. These changes produce educational benefits, but they tend to be modest, uneven, and dependent upon individual volition. Relying on these is tantamount to asserting that General Electric's famed six-sigma initiative would produce the same results if divorced from employee evaluation, promotion, compensation, or termination.

In practice, these rival visions constitute two ends of a continuum. Many accountability programs begin with a rhetorical commitment to the transformative power of coercive accountability and then are eroded by opposition into something more like suggestions.

Let's face it. The details of accountability - governing the content to be tested, the assessments used, what constitutes adequate mastery, and how to deal with students who fail-are inherently arbitrary. Neither developmental psychologists nor psychometricians can prove that specified content must be taught at a particular grade level. Such decisions are imperfect judgments about the needs and capacities of children.

Accountability advocates have difficulty standing firm on program details precisely because decisions regarding what students need to know, when they need to know it, and how well they need to know are reasonable approximations.

Loath to concede that graduation testing can never be perfect, such advocates try to placate critics with one "refinement" after another to the standards, the tests or the system of consequences. Adjusting required scores, giving students additional chances, adding essays, or delaying implementation can be appropriate. But no amount of tweaking will yield a perfect instrument, so the result is most apt to be a series of compromises that leave the fa??ade of accountability but eventually strip it of its power to motivate reform.

Ultimately, the choice is between an imperfect system and none at all. Absent coercive accountability, we have seen how easy it is to graduate ill-equipped students and excuse inadequate school performance - especially among the most disadvantaged students. In the end, standards are a useful and essential artifice.

For those who endorse graduation testing, there is promise on the horizon. While the push for coercive accountability initially generates fierce opposition, once these systems are in place for a while they become part of the "grammar of schooling" for parents, voters, and educators.

Eventually, critics find themselves in the unenviable position of attacking established benchmarks that help ensure that students are learning, teachers are teaching, and schools are serving their public purpose. The question confronting state officials in Nevada, Florida, Massachusetts, and many other places is whether they will use graduation tests to raise the bar for educational achievement or permit them to become another hollow rite of spring.

"States high school exit exams have become political minefields," by June Kronholz, Wall Street Journal, June 6, 2003

"Success in the schools," editorial, Boston Globe, June 9, 2003

Frederick Hess is a resident scholar at the American Enterprise Institute

The level of combat in the teacher certification wars escalated this week as the American Board for Certification of Teacher Excellence (ABCTE) announced that the field test of its "Passport to Teaching" certification exam had been compromised by leaks to ABCTE opponents. (This new test is being developed as an alternative to traditional education school certification. A future test is also being developed as an alternative to the National Board for Professional Teaching Standards super-teacher certification process.) ABCTE president Kathleen Madigan charged that David Imig, president of the American Association of Colleges of Teacher Education, passed around confidential field test questions at a March 17 meeting, thereby invalidating $1.2 million in test development and several months worth of work. Imig told the Washington Post that he received the test from someone working on the project for ACT, which was developing the test for ABCTE, and only gave it to education professionals who understand the sensitive nature of the questions. (ABCTE has since parted ways with ACT and engaged a new test development partner, Promissor.) But Lisa Graham Keegan of the Education Leaders Council, which helped form ABCTE, shot back that "it is hard to conclude that there is not some sort of plan here to try to keep [the test] out of the market." In a later message to ABCTE supporters, she wrote, "We expected much of what occurred - from the printing of blatantly false editorials about the American Board to the dissemination of misleading material to state officials. Far less predictable and just plain shocking is the fact that some individuals have fraudulently obtained and widely circulated exam questions from the first American Board field test."

"Stolen test used against school initiative," by George Archibald, The Washington Times, June 11, 2003

"Education effort meets resistance," by Jay Mathews, Washington Post, June 10, 2003

"Teachers group leaks test, undermines Bush initiative," by George Archibald, Washington Times, June 10, 2003

Controversy is brewing over President Bush's choice of sociologist Robert Lerner as the next Commissioner of Education Statistics and the Senate may well fuss about him during the confirmation process. He happens to be a first rate scholar with two decades of distinguished work under his belt, much of it in education and much of it relying on - believe it or not - federal education statistics. This would make him the first commissioner of education statistics in memory who is a serious consumer of the very product that he will, in this new role, be responsible for gathering, analyzing, and disseminating. One might, therefore, think him superbly qualified for the post. Rather than nominating a civil servant or practicing educator, the Bush administration has opted for a bona fide expert. The problem is that some of Lerner's scholarship hints at what Education Daily terms "conservative stances" and allegations are flying that his confirmation would undermine the mandate of the National Center for Education Statistics (as re-stated in the recent Education Sciences Reform Act) to produce information "in a manner that is objective, secular, neutral, nonideological," etc. Yet the only statutory requirement for the commissioner him/herself is that such a person "shall be highly qualified and have substantial knowledge of statistical methodologies and activities undertaken by the Statistics Center." Lerner fits that description better than any of his (modern) predecessors in this key role.

"Intended nominee may fail neutrality proviso," Education Daily, June 6, 2003 (subscription required)

"Ed. Dept's No. 2 official announces resignation," by Erik W. Robelen, Sean Cavanagh, and Michelle R. David, Education Week, June 6, 2003

Having introduced sweeping legislation to revamp the Head Start program on May 23 (the School Readiness Act of 2003), House Republicans have spent the past three weeks slowly rolling back its most important reforms. [For more information on this bill, go to http://www.edexcellence.net/gadfly/issue.cfm?issue=23#91.] Since its introduction, debate over the new bill has been furious. Head Start champions insist, inter alia, that the proposal to devolve control from Washington to the states will lead to an overall reduction in federal funding and will place an increasing financial burden on the states. (In fact, the GOP proposal would not reduce the amount of federal funding. Rather, it would distribute this money to states as block grants.) Despite attempts by the Bush Administration to support this reform effort - including a new report showing that the current program does not successfully bridge the gap between Head Start's low-income participants and their middle-class - defenders of the status quo have caused House Republicans to ease the most controversial portion of their bill. Instead of affording all states the opportunity to assume control over Head Start, the School Readiness Act allows no more than eight states to do so, and then only as a demonstration project. The full House is expected to vote soon on this new version of the bill.

"House Republicans rewrite Head Start provision," by Diana Jean Schemo, New York Times, June 11, 2003

"Strengthening Head Start: What the evidence shows," Department of Health and Human Services, June 2003

Tuesday, President Bush announced that all 50 states, the District of Columbia, and Puerto Rico now have federally approved accountability plans in place and thus are in formal compliance with the No Child Left Behind Act. (These are the plans that outline how each state will meet adequate yearly progress requirements toward the ultimate goal of bringing all children to proficiency by 2012.) The Bush Administration is heralding this achievement as a milestone for education reform. It's certainly a procedural milestone, considering how dilatory were state responses to Clinton-era federal education laws. The issue is whether these new plans are for real, whether they will actually guide changes in education practice (as opposed to promises on paper), and whether they suitably walk the line between rigorous and flexible. Consider Iowa, whose newly approved NCLB plan includes the same old off-the-shelf norm-referenced tests that Iowa districts have always used and does not include any statewide academic standards or benchmarks. Supposedly Iowa will be able to show adequate yearly progress without demonstrating content-area mastery in reading, math, or science. Seems odd. Is this praiseworthy flexibility in action or is it an evasion of NCLB's (admittedly onerous) requirements?.

"Approval of Iowa plan sets bar for federal flexibility," Education Daily, June 10, 2003 (subscription required)

"President Bush, Secretary Paige celebrate approval of every state accountability plan under NCLB," U.S. Department of Education press release, June 10, 2003

One more area in which American schools and districts are less than transparent: the budget. In this working paper, presented at a recent Brookings conference on "The Teachers We Need," the University of Washington's Marguerite Roza and Paul Hill work from the bottom up to construct real school-by-school teacher salary figures in four districts, rather than relying on district-wide averages. They conclude that there are wide variations in per-school spending within districts that are masked by these averages, with higher-achieving schools in wealthier areas taking a disproportionate bite of the total teacher salary budget line. This hurts students in low-achieving schools by ensuring that they are taught mostly by younger, less experienced teachers who are apt to move on as soon as they accrue the requisite seniority. Roza and Hill call for more transparent budget practices and accurate reporting of per-school spending, but fall just a bit shy of urging the obvious solution: real incentive pay for teachers, experienced or not, who choose to work in tough schools. However, they do hint - in a masterpiece of understatement - that tackling this problem "would not be politically neutral." Among other reasons, this paper is worth reading for the unintended comedy in the introduction, which recounts some of the difficulties reform-minded superintendents face in trying to obtain accurate information on per-school spending in their very own districts. To check it out, go to http://www.crpe.org/workingpapers/pdf/Roza-Hill.pdf.

Jay Greene and Greg Forster, Manhattan Institute's Center for Civic Innovation
June 2003

Jay Greene and Greg Forster authored this new report from the Manhattan Institute's Center for Civic Innovation. It's timely because the Florida program that they analyzed is the nearest thing in America to a voucher program for disabled youngsters and, despite recent rejection of this approach by the U.S. House of Representatives, it remains a topic of intense interest as Congress reauthorizes the Individuals with Disabilities Education Act (IDEA). With 9,200 Florida youngsters using McKay Scholarships to attend private schools (and several hundred thousand more eligible for them), how this program is working is a matter of no little significance. According to Greene and Forster's data (based on surveys of parents of current and past scholarship users), it's working pretty well. "Virtually all measurements," they say, "showed higher levels of satisfaction, better provision of services, and better student environments at McKay schools than at public schools, and in almost all cases members of different racial, income, and disability groups showed no significant differences in their experiences." Left for another day is the question that special-ed evaluators find toughest to tackle: how much academic progress are the children making? You can find this 40-page report at http://www.manhattan-institute.org/html/cr_32.htm. - Chester E. Finn, Jr.

"New report finds significant benefits, parental satisfaction with Florida's innovative McKay scholarships," Press release of the House Committee on Education and the Workforce, June 11, 2003, http://edworkforce.house.gov/press/press108/06jun/manhattan061103.htm.

Lee D. Mitgang, The Wallace Foundation
2003

The Wallace Foundation went to a lot of bother and expense to produce and publicize this 11-page policy brief, which is interesting as far as it goes - but doesn't go nearly far enough. Based on three separate studies commissioned by the foundation, it mainly concludes that we don't have an overall national shortage of people willing to take principals' jobs (though some places have a supply problem), but that the current pipeline doesn't do a good job of delivering first rate leaders into manageable jobs that enable them to boost student achievement. The implications, says author Lee Mitgang, are (1) to adjust incentives and working conditions, especially in places that aren't getting the principals they need; (2) to adjust recruitment and hiring practices to conform to "heightened expectations about principal performance"; and (3) to redefine the principal's job itself. So far, so good, and it may even remind you of the new Fordham-Broad "manifesto" called "Better Leaders for America's Schools." (If you haven't seen - and signed - that document, please go immediately to http://www.edexcellence.net/detail/news.cfm?news_id=1 and do so!) What's disappointing about the Wallace report is that it barely hints at the two touchy domains where the biggest and most contentious changes need to be made if schools are truly to get the kinds of leaders they need: the training and certification of elementary and secondary principals. Why so timid? You can get to the report at http://www.wallacefunds.org/frames/framesetnews.htm. Perhaps more valuable, you can obtain the three Wallace-funded studies that fed into it. The first is "A Matter of Definition: Is There Truly a Shortage of School Principals?" at http://www.crpe.org/pubs/pdf/mroza_princshortagewb.pdf. Second, "Who Is Leading Our Schools?: An Overview of School Administrators and Their Careers," at http://www.rand.org/publications/MR/MR1679. Finally, look up "The Attributes and Career Paths of Principals: Implications for Improving Policy" at http://www.teacherpolicyresearch.org/Career%20Paths%20of%20Principals.PDF.

Paul E. Peterson and William G. Howell, Harvard University
June 12, 2003

It's round three of the ongoing Harvard-Princeton debate over the effects of a privately funded New York City voucher program. In the first round, Harvard researchers Paul Peterson, William Howell, and others conducted an original study on the effects of the voucher program on low-income children who changed schools with its assistance, compared with a control group of similar youngsters who did not change schools. Based on their analysis of the data, Peterson et al. concluded that participation in the voucher program had positive effects on the test scores of African Americans but no effects, positive or negative, on the scores of Hispanic and white students. These initial conclusions offered tentative support to the idea that vouchers might provide poor students, particularly African Americans, with a better alterative to public schools. Princeton researchers Alan B. Krueger and Pei Zhu reanalyzed the data and concluded that the initial study was methodologically flawed because of the way it determined whether a student was black and because the researchers failed to include students for whom there was no baseline data. In their reanalysis, Krueger and Zhu found that that "the safest conclusion is probably that the provision of vouchers did not lower the scores of African American students." After the New York Times made a big deal of this, Peterson and Howell undertook their own reanalysis of the data. This time, they employed both their initial methodology and Krueger and Zhu's methodology. They (again) found that "all the standard ways of estimating voucher effects showed significantly positive effects" for African American students and (again) that "no effects, positive or negative, were observed for Hispanic students or for students who were members of other ethnic groups." Besides confirming Peterson and Howell's initial findings, this report explains the shortcomings in Krueger and Zhu's methodology and conclusions. The study is being released today at the National Press Club, and will be available online at http://www.ksg.harvard.edu/pepg/.