Getting Honest About Grad Rates: How States Play the Numbers and Students Lose
Daria Hall, Education TrustJune 2005
Daria Hall, Education TrustJune 2005
Daria Hall, Education Trust
June 2005
No doubt about it: this Education Trust report is a devastating indictment of state reporting of high-school graduation rates. No Child Left Behind requires, inter alia, that states report their graduation rates to the U.S. Department of Education. The latest round of those reports, due by January 31, 2005, covered the 2002-3 school year. (Baseline data for 2001-2 were reported in September 2003; to see what EdTrust and Gadfly thought of those, go here.)
The results are dismaying, to say the least, the more so at a time when high-school reform is supposedly the top education priority of the Bush administration and the National Governors Association. "[I]n far too many cases," says Ms. Hall,
the information [the states] reported is of little value to school-improvement efforts. In fact, three states [Alabama, Louisiana, Massachusetts, plus D.C.] reported no graduation-rate data at all. Another seven did not report data broken down by students' race, ethnicity, or socioeconomic status. Of the states that did provide graduation-rate information, most reported rates look dubiously high when compared to the results of multiple independent analyses of state graduation rates.
Those independent analyses include reports by the Manhattan Institute, the Urban Institute and the ETS Policy Information Center, among others. EdTrust settles on the Urban Institute's "cumulative promotion index" as an external audit of state graduation rates, and finds that it is lower than the state-reported results in every single case: by as little as 1-3 percent (Idaho, Washington, Alaska, New Jersey) and by as much as 27-33 percent (New Mexico and the Carolinas). North Carolina, for example, reported to the feds a graduation rate of 97 percent while the Urban Institute estimates that the state's true graduation rate is more like 64 percent.
In the middle of the pack, with 13-14 point discrepancies, are states like Wisconsin, Florida and Ohio, which reported graduation rates of 92 percent, 66 percent, and 84 percent respectively vs. Urban Institute estimates of 65, 53, and 71 percent.
Some states, moreover, reported incredible gains in graduation rates from spring 2002 to spring 2003. If you believe the data, for example, Nevada rose from 64 to 75 percent in a single year while Oklahoma's rate soared from 69 to 86 percent.
A few states, EdTrust says, deserve plaudits for tackling the challenge of honest graduation rate reporting - they cite Alaska and Washington in particular - and others are working on it. Meanwhile, however, many jurisdictions (which have wide latitude in this regard) have set outrageously low NCLB graduation targets - in 34 cases lower than their current graduation rates. And, asserts EdTrust, the slack wording of NCLB in this regard plus vagueness, inconsistency and excessive deference by the U.S. Department of Education have allowed these outrages to occur and to continue.
Get this report and read it. You can find it online here.
"North Carolina's near perfect graduation rate, and other fables," by Jay Mathews, Washington Post, June 28, 2005
"Report: States nationwide inflate graduation rates," by Bonnie Eslinger, San Francisco Examiner, June 26, 2005
Center on Education Policy
June 2005
This moderately interesting 27-page pair of case studies of two anonymous school systems, one in Maryland and one in Virginia, tracks how they're changing in response to mandatory statewide "end of course" testing as a prerequisite to high school graduation. (Such a scheme is in place in Virginia, being phased in in Maryland.) Most significant is that non-trivial changes are occurring in what schools do, how teachers teach, and (to a lesser extent) how kids respond. This suggests that high-stakes testing applied to students does serve the intended purpose of altering institutional behavior on the part of their schools and educators. But not all of these alterations are salubrious, at least in the authors' eyes. An example of a positive change would be that "the district was playing a critical function in brokering services and supports to help prepare teachers and students for the exams." A less desirable change: "Some students and teachers complain that instruction has become too focused on reviewing discrete facts, with little time for discussion . . . and that some students are being left behind as teachers push ahead to cover all the topics. . . ." See for yourself. It is only, the authors acknowledge, a "brief, first glimpse," but it's informative and, for the most part, encouraging. You can find it here.
Michael Podgursky, Lawrence Mishel, and Sean Corcoran, National Council on Teacher Quality
June 29, 2005
A few weeks ago (see here), we reviewed an article by Rick Hess, noting, "Ad hominem attacks, overblown rhetoric, and the imputation of sinister motives are failings of both sides [in the education debates] and have made compromise next to impossible." The new "Square-off" series from the National Council on Teacher Quality, which made its debut this week, may be the antidote to that shrillness. In reasoned tones, Podgursky (author of an important Education Next article on the topic; see here) and Mishel and Corcoran (who responded to Podgursky in How Does Teacher Pay Compare?; see here) discuss how to measure teacher compensation (through an hourly or weekly wage measure? And what about benefits?), the limitations of data sources, and how best to attract highly qualified candidates to the teaching pool. A refreshingly polite, yet bracing exchange of views on an important topic. We look forward to future installments. You can get to it here.
Center on Educational Governance, University of Southern California
2005
How can charter schools most effectively partner with nonprofits, for-profits, government agencies, and other groups? This brief paperback serves as a handy primer on that topic for both newbies and old hands, delivering a nuts-and-bolts breakdown of managing effective collaborations. The authors studied 22 charter schools that had entered into such partnerships, boiling down their collective experiences into an octet of "lessons for success," several of them obvious and predictable: identifying needs, picking the right partner, defining the scope of the relationship, and setting accountability measures. Overall, the book explains the basic character of a charter school-partner collaboration, outlining its elements and stages while also advising on how to break up if things don't go as planned. Nothing earth-shattering here, but worth a peek. Get it here.
A recent article in Education Week notes that NEA members will be celebrating growth in the union's membership rolls at the union's annual shindig, opening later this week in Los Angeles. After last year's drop in "active teacher" membership figures - the first such drop in 28 years - the NEA looks to gain members this year on the strength of a new recruitment campaign. But a closer examination shows that the 2.8-million-strong union has less to celebrate than it claims. Recent mergers with local and state affiliates of the American Federation of Teachers brought new members to the group, but without raising total number of unionized teachers. Rates of annual membership growth have dropped to one percent. Ninety percent of schoolteachers in the U.S. are members of either the NEA or the AFT, and it appears that real recruitment - that is, the recruiting of non-union teachers - has hit a brick wall, with half of the NEA's 38,000 new members being drawn from the ranks of school support staff. The regions where NEA membership is weakest - mainly in the South - have traditionally resisted unionization. Recruitment efforts in these areas, though seriously beefed up as part of a major nationwide recruitment campaign, have lagged significantly. Most ominous, union-backed candidates have fared only moderately well in the last three election cycles despite an enormous get-out-the-vote effort and the expenditure of buckets of teachers' dues. Is the ride over? (And if you'd like the inside scoop, blow by blow, from the NEA convention, check out the excellent daily first-hand reports of Mike Antonucci, see here.)
"NEA grows more strategic about membership," by Bess Keller, Education Week, June 22, 2005
John Walton's tragic and untimely death earlier this week in a small-plane crash at the age of 58 deprives low-income American children of one of their best friends and most generous benefactors. Soft-spoken, unassuming, and selfless, this mega-billionaire, described by one admirer as "the very soul of the choice movement," gave tens of millions of dollars to create better educational opportunities for disadvantaged youngsters. Unlike many philanthropists who pick one path or the other, John was both strategic with respect to policy and open-handed with respect to kids in need. Thousands of youngsters in dozens of communities (including Dayton's PACE program) benefited directly from the Children's Scholarship Fund that he and Theodore Forstmann founded and funded, enabling them to transfer with the help of these private vouchers from dire urban public schools into decent private-school alternatives.
At the same time, John and the Walton Family Foundation underwrote much of the charter-school movement, helping individual schools, state charter associations, and national ventures. Nor did he simply write checks; he served on boards, he checked people out, he made calls, he asked probing questions - and he displayed plenty of grit and determination (as he had in Vietnam decades earlier) when it was necessary to make tough decisions, pull plugs, and reinvent things.
John was also one of the least pretentious and kindest people you'd ever want to know. He drove himself hard - I once watched him pedal a bike up a mountainside in Colorado (as I cruised along the riverbank) - and relished physical as well as policy challenges.
I cannot improve upon the words of Howard Fuller in a Monday-night email to friends (to read his remarks in their entirety, visit http://www.baeo.org): "John was one of the nicest people I have ever met. He was a staunch, unbending warrior in the battle for parental choice. I once asked John why he supported this movement. He said that poor children were getting a raw deal and he wanted to do something about it and he was in a position to do so. Here was a man who could have lived a life of leisure. Instead, he put his money, his intellect, and his energy into the effort to empower low income and working class families to be able to choose the best learning environment for their children. Because of him there are thousands of our children who have a better future. Because of him there is a movement out here that must honor his memory by working harder to advance our cause. I will never forget this kind and decent man. I thank God for allowing him to be in our midst." Indeed. Rest in peace.
"Wal-Mart heir John Walton dies in Wyoming plane crash," by Melissa Nelson, Associated Press, June 30, 2005
"A champion for children: BAEO mourns the loss of John T. Walton, 58," by Howard Fuller
Teachers and staff at a Little Rock elementary school recently received bonuses totaling $134,800 after they improved student test scores by a whopping 17 percent in one year. The privately funded pilot bonus-pay program used a value-added scale, with teachers receiving $100 for a 4 percent improvement in a student's score, $200 for a 5-9 percent improvement, $300 for a 10-14 percent improvement, and $400 for a 15 percent or higher improvement. Teachers screamed and cried as bonuses were announced, which ranged from $1,800 to $8,600 apiece. Lisa Black, executive director of the Public Education Foundation and the program's sponsor, said, "We are here to show that the quality of teaching impacts the education of children, and that the clear measurement of data is important and necessary." While the program was only a small-scale experiment, it demonstrates that, contrary to teacher union claims, teachers can be motivated by incentives like bonuses (hey, who can't?) to produce better results. Also, as first-grade teacher Kathy Thomas noted, testing pinpoints the areas in which students need extra help and shows the positive results of a year of hard work: "I've taught since 1970 and have never had the opportunity to look at what my students knew and how they changed. The money is nice; the tools are better."
"LR elementary scores bonuses for test gains," by Cynthia Howell, Arkansas Democrat-Gazette, June 21, 2005
In the past several weeks, as the Ohio legislature crafted the Buckeye State's biennial budget, each line item had its own constituency. Some of the most contentious debates were those centered on the state's system of funding public education.
One piece of pervasive misinformation is the supposed cost that charter schools exact on school districts. The rhetoric on this issue is completely overblown and deeply misleading.
At the margin, the state provides school districts with at least $5,169 in basic aid for each additional student enrolled. Districts also receive transportation aid as well as funding for a host of categorical items such as teacher training, gifted education, special education, etc.
While the state goes through the exacting process of setting its biennial budget, school districts set their own budgets for the next school year. Each district budget is based on estimates of revenue and expenditure, which in turn are partly based on enrollment estimates that are supposed to account for a variety of changes over the course of a year - families moving in and out, new construction, and enrollment at private and charter schools, as well as home schooling. The pressure to get these estimates correct is very high; once set, school budgets are difficult to change.
Take this example: Assume that a school district estimates it will begin the next year with 10,000 students and that the average cost per student is $9,000, which yields a $90 million budget.
What happens if 100 extra students enroll next year? At the margin, it is likely that each additional student will cost the district very little. A bit more paper, water, and floor wax might be used, but most costs are fixed. It's unlikely that the district will have to hire additional staff or purchase additional buses for these students. Most costs will simply be absorbed. And the district will realize revenues of at least $516,900 in additional basic aid ($5,169 per student x 100) in the first year. (Yes, it may need to hire another teacher or two the following year.)
The opposite happens when a district begins its school year with 100 fewer students than estimated. Here the marginal reduction in state aid per student is $5,169, leaving the actual revenue $516,900 less than predicted and planned around. Once again, the district cannot reduce expenditures much for that year, but in the next year the reduced need for expenditures such as books, paper, and other supplies resulting from fewer students will be captured in the new budget.
So, in the end, a two-year budget cycle can wash out the benefit or penalty of move-ins and move-outs - and even in the first year, these are hardly the enormous sums most districts contend they are. The only result of more or fewer students than estimated is actual revenue that is either more or less than estimated.
The key word here is estimated. A fundamental task of the school budget process, and one of the primary jobs of a district accounting office and local board, is estimating next year's student count. School districts in Ohio know - and have known for ages - that they face stiff competition from other education providers (charters for sure but also home schooling, virtual schooling, and private schooling). While they wail to the press about these pressures, district continue to make enrollment projections that pretend that charters don't exist, or aren't the popular alternative they are, and they continue to overestimate district enrollment. That's not the fault of charter schools. It's the fault of na??ve enrollment projections from people who ought to know better.
In Olentangy, a fast growing suburban district (adding 1000+ students per year), we employ a consulting firm to provide enrollment estimates, which are then reviewed and massaged by the board's development committee and district administration, taking into account such factors as the housing market and charter school demand. Due to a spike in housing starts this past school year (1,732 new homes instead of the predicted 1,640), the district enrolled 100 more students than predicted. These extra students provided additional state funding which the district recorded as a financial benefit in its state-mandated Five-Year Forecast.
Though Olentangy benefited this one year by a missed projection, neighboring districts have suffered due to overly-rosy enrollment projections. The Columbus Dispatch recently rapped the city district for what appears to be an under-estimation of the number of students expected to flee to charter schools. By under-estimating future charter enrollment, the city was able to balance its books based on "expected" state funding that will disappear into vapor once the school year begins and the count comes in lower than projected. Guess who will get the blame? Charter operators.
What to conclude from this? It's not charter school enrollments (or any other demographic factor) that causes revenue shortfalls, but school districts' inability to accurately gauge student enrollment and plan accordingly. If districts correctly estimate their student count (including but not limited to the effects of charter attendance) revenue and expenditures will be the same as budgeted. Charter students and move-outs affect districts in the same manner, but neither is a cause of a school district's financial woes. Flawed enrollment assumptions are the real problem.
James Fedako ([email protected]) is a member of the Olentangy, Ohio, school board and president of Decision Outlook, a consulting firm.
Last week, David Broder and George Will - high priests of the Washington fourth estate - published side-by-side columns on education in the Washington Post. Both say important and useful things without, finally, coming to any real conclusions or recommendations. Broder noted that teachers and the general public are miles apart on how they view education reform, as revealed by a recent poll from ETS. A large plurality of parents support NCLB, but teachers overwhelmingly oppose it; a strong majority of parents say high school academic expectations are too low, while an equal percentage of teachers disagree. Most distressing, when asked whether all students should be held to a single high standard rather than differing standards for different types of students, "[m]ore than half of the parents favored the single standard, but only one-quarter of the high school teachers agreed." Broder notes that this disconnect is "a real barrier to progress." Well, yes, if these polling data really do represent the opinions of average teachers, we think that is a real barrier to progress. In a paired column, George Will noted how states are balking at NCLB's requirements, and seemed to agree with Education Secretary Margaret Spellings that they do so mostly out of shame at the poor performance that NCLB has uncovered. But embarrassment, Will notes, is of "limited power. . . . Spellings expects NCLB's high expectations to be substantially self-fulfilling because she 'thinks the best' of people - parents and school officials - at the local level. But if they really are as vigilant, diligent and susceptible to embarrassment as she assumes, why do we need the law?" It's hard to know, finally, what to make of either essay. Is Broder not willing to suggest which side - teachers or parents - is right about demanding a high, uniform standard for everyone? Is Will suggesting that the right incentives cannot overcome the resistance to raise standards? In both cases, if that's the conclusion, we respectfully dissent.
"Split over schools. . . ," by David Broder, Washington Post, June 23, 2005
". . . And responsibility," by George Will, Washington Post, June 23, 2005
The Texas House, called back into session by school funding issues, has passed a bill (HB 2) that increases teacher pay, lowers school property taxes, and allows districts to purchase digital technology instead of traditional textbooks. This last part is a second attempt at an earlier bill that failed to pass (HB 4), which we wrote about in April (see here). As before, all references to "textbooks" are replaced with "instructional materials," and the funds available to purchase them are increased. Currently, Texas spends $30 per student on technology and $65 per student on textbooks; under the new plan, that amount will jump to $125 by 2006, of which $50 must be spent on technology, and $150 by 2007, of which $60 must be spent on technology. The bill also creates a rolling adoption process rather than the strict timetables in place. The new approach doesn't do everything we advocated in The Mad, Mad World of Textbook Adoption - including getting rid of textbook adoption completely! - but it's a start, and a welcome one at that.
"Texas House approves education bill," AP, June 29, 2005
Michael Podgursky, Lawrence Mishel, and Sean Corcoran, National Council on Teacher Quality
June 29, 2005
A few weeks ago (see here), we reviewed an article by Rick Hess, noting, "Ad hominem attacks, overblown rhetoric, and the imputation of sinister motives are failings of both sides [in the education debates] and have made compromise next to impossible." The new "Square-off" series from the National Council on Teacher Quality, which made its debut this week, may be the antidote to that shrillness. In reasoned tones, Podgursky (author of an important Education Next article on the topic; see here) and Mishel and Corcoran (who responded to Podgursky in How Does Teacher Pay Compare?; see here) discuss how to measure teacher compensation (through an hourly or weekly wage measure? And what about benefits?), the limitations of data sources, and how best to attract highly qualified candidates to the teaching pool. A refreshingly polite, yet bracing exchange of views on an important topic. We look forward to future installments. You can get to it here.
Center on Education Policy
June 2005
This moderately interesting 27-page pair of case studies of two anonymous school systems, one in Maryland and one in Virginia, tracks how they're changing in response to mandatory statewide "end of course" testing as a prerequisite to high school graduation. (Such a scheme is in place in Virginia, being phased in in Maryland.) Most significant is that non-trivial changes are occurring in what schools do, how teachers teach, and (to a lesser extent) how kids respond. This suggests that high-stakes testing applied to students does serve the intended purpose of altering institutional behavior on the part of their schools and educators. But not all of these alterations are salubrious, at least in the authors' eyes. An example of a positive change would be that "the district was playing a critical function in brokering services and supports to help prepare teachers and students for the exams." A less desirable change: "Some students and teachers complain that instruction has become too focused on reviewing discrete facts, with little time for discussion . . . and that some students are being left behind as teachers push ahead to cover all the topics. . . ." See for yourself. It is only, the authors acknowledge, a "brief, first glimpse," but it's informative and, for the most part, encouraging. You can find it here.
Center on Educational Governance, University of Southern California
2005
How can charter schools most effectively partner with nonprofits, for-profits, government agencies, and other groups? This brief paperback serves as a handy primer on that topic for both newbies and old hands, delivering a nuts-and-bolts breakdown of managing effective collaborations. The authors studied 22 charter schools that had entered into such partnerships, boiling down their collective experiences into an octet of "lessons for success," several of them obvious and predictable: identifying needs, picking the right partner, defining the scope of the relationship, and setting accountability measures. Overall, the book explains the basic character of a charter school-partner collaboration, outlining its elements and stages while also advising on how to break up if things don't go as planned. Nothing earth-shattering here, but worth a peek. Get it here.
Daria Hall, Education Trust
June 2005
No doubt about it: this Education Trust report is a devastating indictment of state reporting of high-school graduation rates. No Child Left Behind requires, inter alia, that states report their graduation rates to the U.S. Department of Education. The latest round of those reports, due by January 31, 2005, covered the 2002-3 school year. (Baseline data for 2001-2 were reported in September 2003; to see what EdTrust and Gadfly thought of those, go here.)
The results are dismaying, to say the least, the more so at a time when high-school reform is supposedly the top education priority of the Bush administration and the National Governors Association. "[I]n far too many cases," says Ms. Hall,
the information [the states] reported is of little value to school-improvement efforts. In fact, three states [Alabama, Louisiana, Massachusetts, plus D.C.] reported no graduation-rate data at all. Another seven did not report data broken down by students' race, ethnicity, or socioeconomic status. Of the states that did provide graduation-rate information, most reported rates look dubiously high when compared to the results of multiple independent analyses of state graduation rates.
Those independent analyses include reports by the Manhattan Institute, the Urban Institute and the ETS Policy Information Center, among others. EdTrust settles on the Urban Institute's "cumulative promotion index" as an external audit of state graduation rates, and finds that it is lower than the state-reported results in every single case: by as little as 1-3 percent (Idaho, Washington, Alaska, New Jersey) and by as much as 27-33 percent (New Mexico and the Carolinas). North Carolina, for example, reported to the feds a graduation rate of 97 percent while the Urban Institute estimates that the state's true graduation rate is more like 64 percent.
In the middle of the pack, with 13-14 point discrepancies, are states like Wisconsin, Florida and Ohio, which reported graduation rates of 92 percent, 66 percent, and 84 percent respectively vs. Urban Institute estimates of 65, 53, and 71 percent.
Some states, moreover, reported incredible gains in graduation rates from spring 2002 to spring 2003. If you believe the data, for example, Nevada rose from 64 to 75 percent in a single year while Oklahoma's rate soared from 69 to 86 percent.
A few states, EdTrust says, deserve plaudits for tackling the challenge of honest graduation rate reporting - they cite Alaska and Washington in particular - and others are working on it. Meanwhile, however, many jurisdictions (which have wide latitude in this regard) have set outrageously low NCLB graduation targets - in 34 cases lower than their current graduation rates. And, asserts EdTrust, the slack wording of NCLB in this regard plus vagueness, inconsistency and excessive deference by the U.S. Department of Education have allowed these outrages to occur and to continue.
Get this report and read it. You can find it online here.
"North Carolina's near perfect graduation rate, and other fables," by Jay Mathews, Washington Post, June 28, 2005
"Report: States nationwide inflate graduation rates," by Bonnie Eslinger, San Francisco Examiner, June 26, 2005