Keeping Count and Losing Count: Calculating Graduation Rates for all Students Under NCLB Accountability
Christopher B. Swanson, Education Policy Center, The Urban InstituteAugust 2003
Christopher B. Swanson, Education Policy Center, The Urban InstituteAugust 2003
Christopher B. Swanson, Education Policy Center, The Urban Institute
August 2003
This clear and concise report, initially presented at a conference sponsored by the Harvard Civil Rights Project (and one of the few interesting reports to emerge from that ill-starred venture), offers a method for calculating graduation rates that states may find useful now that NCLB requires them to include such measures in their accountability plans. Calculating a graduation rate may seem simple, but is actually fraught with difficulty. Thus the "conventional wisdom" from the National Center for Education Statistics (NCES) - that U.S. graduation rates hover between 85 and 90 percent - is deeply misleading. Superior methods show that the true rate is, sadly, closer to two-thirds. Here, Swanson presents one of these methods, which he developed, and recaps the second, developed earlier by Manhattan's Jay Greene. [see http://www.edexcellence.net/gadfly/issue.cfm?issue=27#153] Both methods consider how many students actually graduate; NCES, on the other hand, relies on drop-out rates, which are notoriously flawed. This is no mere academic quarrel. States need a sound methodology to meet the spirit of NCLB, but unfortunately too many states are instead opting for the NCES approach, which meets the barest letter of the law but surely violates its intent. [For each state's plans see Swanson's separate NCLB Implementation Report] Both Swanson's and Greene's methods rely on more readily available data. Greene's may be slightly more accurate while Swanson's is easier to calculate, but both represent a quantum improvement over the NCES approach in terms of accuracy and data availability. States would be remiss to ignore them. And everyone should note the troubling results they reveal - particularly that barely half of minority youngsters graduate from high school in America today. And that in some states the average student has only a 50-50 chance of graduating. To learn more, click here.
Christopher Mazzeo, National Governors Association's Center for Best Practices
September 2003.
This constructive ten-pager discusses what states can do to strengthen the leadership (principals, mainly) of their public schools. It tackles three big issues - licensure, preparation, and professional development - and offers valuable insights and sage advice on all three. This includes removing "barriers for talented individuals to enter the profession," establishing "alternative principal preparation programs," and tailoring the professional development of school leaders to the expectations of NCLB. You can find it online here.
Julian R. Betts and PPIC's Andrew C. Zau and Lorien A. Rice, Public Policy Institute of California (PPIC)
2003
This short book from the Public Policy Institute of California (PPIC) provides a close analysis of data from San Diego and seeks to illumine equalities and inequalities in school resources; trends in pupil achievement and achievement gaps; and, especially, which factors in school and classroom have the most influence on changes in student performance. On the first point, the authors found that the main resource discrepancy between affluent and non-affluent elementary schools in San Diego is teacher experience and preparation. On the second point, they unearthed "shocking" achievement gaps in all the usual directions - but also found nearly all of those gaps narrowing between 1998 and 2000. (That's before the onset of the much-publicized "Blueprint for Student Success" reform strategy of Alan Bersin and Tony Alvarado.) What turns out to influence student achievement from school to school and year to year? There's much technical stuff here, though Betts & Co. do an exceptionally nice job of making it accessible to ordinary mortals. It turns out - no big surprise - that attending school actually matters, i.e. kids who are absent a lot learn less. Peer group matters, too, but more at the grade level than the classroom level: the higher achieving a child's grade-mates, the better he/she is apt to do. Class size matters some, although only in reading in the early grades. And teachers matter, but the usual ways of defining their "qualifications" matter only intermittently. In what may turn out to be their most controversial conclusion, they say, "In very few cases did we find any statistically significant difference between the effectiveness of fully credentialed teachers with ten or more years of experience and teachers with less experience, regardless of whether they held a full or emergency credential or an internship." (The big exception: high-school math, where "subject authorization level" matters greatly.) Perhaps most important: schools don't generally cause achievement gaps. They're present when kids arrive at school. Policy makers may therefore want to give greater emphasis to pre-school. Though it's a case study of a single district, this book is so careful and perceptive that you may well want a copy. It's hard to believe these findings don't apply elsewhere. The ISBN is 1582130442 and you can get more information at http://www.ppic.org/main/publication.asp?i=321.
YMCA, Dartmouth Medical School and the Institute for American Values
2003
It's never been easy to get your mind around what James Coleman called "social capital" and what the Commission on Children at Risk, in this provocative new report, terms "connectedness." It says the deteriorating mental and behavioral health of U.S. children is a real problem but we're barking up the wrong tree when we seek to solve it primarily through medication, therapy, and special programs for "at-risk" children. What's needed instead - or in addition - say the 33 members of this commission (a joint venture of the YMCA, Dartmouth Medical School, and the Institute for American Values), is greater attention to the "broad environmental conditions that are contributing to growing numbers of suffering children." "In large measure," they find, "what's causing this crisis of American childhood is a lack of . . . close connections to other people and deep connections to moral and spiritual meaning." Their antidote: children should live in "authoritative communities." Many pages are devoted to setting forth the essential characteristics of such communities, which include: establishing "clear limits and expectations," transmitting to community members "a shared understanding of what it means to be a good person," and encouraging "spiritual and religious development." It's a bit like reinventing "civil society" at the community level and urging that we raise our children within that society. Of particular merit is the scientific evidence undergirding the commission's diagnosis and prescription. The report is 82 pages (not counting 18 commissioned papers) and you can find it at several places, including http://www.americanvalues.org/html/hardwired.html.
Karl T. Kurtz, Alan Rosenthal, and Cliff Zukin, National Conference of State Legislatures
September 2003
This report publishes the results of a survey conducted by the National Conference of State Legislatures, the Center for Civic Education, and Indiana University's Center on Congress. The survey focused on civic education and the awareness of the American public, with questions concerning citizen's political knowledge as well as levels of participation and involvement in politics and community. By comparing the results of different generations, it shows a significant generation gap in civic knowledge. There is a clear decline in understanding and civic participation among our nation's youth - dubbed "DotNets," aged 15-26, - which arguably contributes to decreased engagement in local, state, or national politics. Only 47 percent of DotNets vote, compared to 77 percent of older generations. Just 48 percent knew the political party of their current governor and only 40 percent knew which party controlled Congress. Interestingly, DotNets who had taken civics classes were more than twice as likely to follow politics. This report is informative and helpful to those designing civics programs for young people. Check it out at http://www.ncsl.org/public/trust/citizenship.pdf.
We welcome a new player on the education-choice team, the Hispanic Council for Reform and Educational Options, or Hispanic CREO. The group was launched today at the National Press Club, with a follow-up conference in Washington and the release of a new study on Hispanic students and choice, authored by Jay Greene of the Manhattan Institute. (We'll have more to say on the study in a future issue.) Hispanic CREO will concentrate its initial efforts in four states - Colorado, Texas, New Jersey, and Florida - and will seek to build grassroots support for all forms of choice, including charter schools and vouchers.
The latest issue of National Review contains a special section on education, featuring Victor Davis Hanson (a contributor to Fordham's recent publication Terrorists, Despots, and Democracy: What Our Children Need to Know) on the Iraq War and college campuses. But our attention was caught by Princeton student Carlos Ramos-Mrosovsky's piece on the hypocrisy of student service. As competition for spots in elite colleges becomes fierce - and post-college prospects grow more straitened - many students are turning to volunteer service to differentiate themselves from the pack. Problem is, this service has now become just one more box to check off on the application, which lends a workaday attitude to what ought to be spontaneous service to one's fellow men. We guess we'll take service-for-reward over no service at all, but in San Diego the unions don't want either. The local paper recently reported that parents who tried to organize community cleanups of weed-ridden schools in response to cuts in maintenance budgets were turned away after unions complained that volunteers were displacing union workers. "What happens when the district gets in better financial shape? Why rehire the landscape crews when the work is being done free? If people really want to help, they should be writing their elected officials about the budget," said a union official.
"Special education section," National Review, October 13, 2003,
"Schools caught between weedy yards and union jobs," by Maureen Magee, San Diego Union-Tribune, October 5, 2003,
Four years ago in two St. Clair County, Michigan school districts, officials started a highly controversial new accountability program whereby students who were not reading at grade level at the end of third grade would be held back. Critics of retention blasted the program, citing dropout statistics that show that students who are held back are more likely to drop out than students who are pushed along through social promotion. The two district superintendents stood their ground, however, despite cries from individual schools, teachers, principals and parents asking to delay the consequences of the program to another year. "Why should we wait?" Superintendent Dennis Guiser said. "If what we're doing is the right thing, why not affect this year's third-graders?" Besides, the superintendents maintained, there is "other data [that] shows that kids who are not on grade level in reading by the end of third grade are less likely to graduate." Turns out the commitment to the program was exactly what the students of St. Clair County needed. In both districts, "83 percent of fourth-graders met the state's expectations in reading, up from 55 percent in Algonac Community Schools and 65 percent in Yale Public Schools in 1999. They also came in ahead of the state average, 75 percent."
"MEAP success: Holding kids back seems to pay off," by Peggy Walsh-Sarnecki,
Detroit Free Press, October 8, 2003
Standardized tests may be under attack in America but they turn out to be a godsend for Russian parents. That nation's college entrance exams, relics of the Communist era, are specific to each university and usually involve professors drilling applicants in an oral exam. The system is rife with corruption, with professors charging fat sums to serve as "tutors" who give students the exact questions they will face, or simply taking outright bribes for letting students pass. Those who don't pay face a barrage of unanswerable questions and have little chance of admittance, while those who can't afford the bribes are often forced to pay in other ways. (One woman went to work for free at the university her daughter studied at, a kind of latter-day indentured servitude that many American parents can at least sympathize with.) Now a new standardized entrance test, the Unified State Exam, is leveling the playing field and tamping down the corruption of Russian college entry.
"Admissions fee," by Masha Gessen, The New Republic, October 13, 2003 (subscription required)
In August, the Marysville, Washington school superintendent refused to comply with 30 union demands, including across-the-board raises that would have cost the district $14 million. And so, on September 1st - the first day of school, chosen to cause maximum chaos - Marysville teachers began what is about to become the longest strike in the state's history. It's gone on so long that even children are fed up with their unexpected vacation, especially seniors who saw college application deadlines looming and graduation slipping into late summer. So, a group of high-school students requested that the teachers go back to work. Request denied. The resourceful students took matters into their own hands and held an all-night sit-in to protest the strike, to no avail. They begged the governor to intervene. He wouldn't. Finally, they enlisted the help of the Evergreen Freedom Foundation, which helped the kids call a press conference to dramatize their cause. The Washington Education Association responded by calling EFF "an evil band of zealots." The EFF has decided to live up to the name, so they "covered up [their] horns," as spokeswoman Marsha Richards put it, and distributed information about district rules and school funding to parents, teachers, and students. As it turns out, the strike is actually illegal under state law, but the Marysville superintendent has yet to go to court to enforce the no-strike provision, though she has indicated that "perhaps, one of these days" she might. A group of parents has beaten her to the punch.
"Kids these days," by Heather Roscoe, American Spectator, October 7, 2003,
"The paramount duty of the state is . . . high school football?" Evergreen Freedom Foundation, October 3, 2003,
"Parents file suit in Marysville teacher strike," by Leslie Knopp, October 6, 2003,
"If men were angels," Madison wrote in Federalist #51, "no government would be necessary."
Thus too with schools and other education institutions. If children were angels, nobody would need to check on whether they did their homework or test them to see whether they learned the week's spelling words. If educators were angels, they would spontaneously leave no child behind and we wouldn't need elaborate mechanisms to hold them and their schools "accountable" for performance.
"Accountability" means the arrangements by which others can determine whether we're doing what angels would do unbidden—and the means by which those others can influence us to become more angel-like. Its two vital ingredients are not angelic wings, however, but, transparency and intervention (or its equivalent: incentives, sanctions, and rewards whose influence is similar to intervention).
Transparency and intervention (and its avoidance) occur throughout our lives. Cars have speedometers and police have radar—and the ability to write tickets—to keep us accountable to speed limits. The physician has a scale. A device beeps if you take an item from the store that you didn't pay for. The monthly bank statement asks to be reconciled with one's checkbook. Dinner guests smile or frown after taking a bite of the meal you've cooked. The remote lets you "punish" a bad television show by opting for another. (If every program were produced by angels, who would need options?) The auditor checks the company treasurer's books and reports his findings to shareholders and the public. And on and on.
In education, we've grown accustomed to accountability at the individual and institutional levels. Has Mathilda passed the 4th grade proficiency test or must she attend summer school? Is the Lincoln School making "adequate yearly progress" or must the district intervene? Do enough children want to attend the Einstein Charter School to enable it to meet its budget? Is Montana making headway on its NAEP math results or should voters replace the governor?
But what of organizations that seek to influence education? To whom is Fair Test accountable? The American Educational Research Association? The Harvard Educational Review? The Thomas B. Fordham Foundation? The American Association of School Administrators? The teachers' unions? Are they transparent? Does anyone have authority to intervene to make them more angelic? Such questions rarely get asked, yet some of these outfits, particularly the unions, are immensely powerful.
This grew vivid indeed in recent days as Barbara A. Bullock, former head of the Washington Teachers Union, was charged with (as the Washington Post put it) "plundering more than $2.5 million from union coffers so she and two fellow union leaders could finance lavish lifestyles complete with fur coats, catered meals, and luxury cars." Bullock has since pleaded guilty and agreed to serve up to 10 years in prison and pay a $500,000 fine for her shopping spree.
Besides law enforcers, to whom is a union's leader accountable? To the members, one assumes, who pay its dues and elect its officers. But what happens when someone messes up badly—and seeks to conceal the evidence? How transparent are teacher unions? Who has the authority to intervene? How effective are the internal controls? Does membership accountability work?
The eight-page "charging papers" in the Bullock case describe what the Post termed "a long unchecked conspiracy" in which she and two henchmen sought "to enrich themselves and others by stealing millions of dollars from [the union] and its individual members."
A similar case is under investigation in Miami, Florida where the longtime head of that teachers' union siphoned large sums out of its coffers and into his own.
How does a membership organization hold its officers and staff to account in the event they turn out not to be angels? Supposedly in the way that voters and taxpayers hold government accountable: democratic politics, the capacity to elect leaders to run the place, to observe their performance, and then to retain or replace them as needed. (Consider what just happened in California.) Government by consent of the governed is supposed to work in membership organizations, too. (This is different from publicly traded corporations, which are accountable to their shareholders, and nonprofits such as think tanks, which are accountable to their trustees and, sometimes, their funders.)
Nearly half a century ago, famed sociologists Seymour Martin Lipset, Martin Trow, and James Coleman wrote Union Democracy, based on a case study of the International Typographical Union. They concluded that lively internal politics—the existence of factions, rivals for office, and the equivalent of political parties—had kept democracy vibrant in the I.T.U. But they didn't find it in most unions and attributed its absence to the potent forces of bureaucracy and incumbency:
[A]n incumbent administrator [has] great power and advantage over the rank and file. . . . This advantage takes such forms as control over financial resources and internal communications. . . . The normal position of the trade-union member in modern urban society makes it likely that few individuals will ordinarily be actively interested in the affairs of the union. . . . The absence of membership participation facilitates the existence of one-party oligarchy. . . . [U]nion leaders possess great power to do things which would never be approved if a democratic choice were available.
In other words, non-angelic union leaders can get away with misdeeds due to the absence of viable choices for their members. And that's exactly what happened in the nation's capital. Barbara Bullock "ruled by fear," says the Post, "and proved so effective at stifling dissent that during her last two years in office, membership meetings rarely drew the 100 people needed for a quorum. . . . When [a longtime union activist] would ask a question about union finances, most of Bullock's supporters would leave the meeting. Then someone would call for a quorum and, seeing none, Bullock would declare the meeting over. Bullock ran the union as a political patronage system. . . ."
One might say she ran it precisely the way the unions say schools would be run if it weren't for teacher unions!
The absence of choice is bad for democracy. It causes institutions to go wrong. It saps their accountability. It invites petty (and not so petty) despotism and disregard for performance. How is the plight of members of the Washington Teachers Union different from that of children in failing schools to which they have no alternative? Surely the uncle-knows-best-so-pipe-down-and-trust-us mindset that leads unions to block education choice for families must also habituate their members to expect no alternatives among union leaders, either.
When internal controls fail, outside oversight is called for. That's the theory of NCLB. That's what happened to Ms. Bullock and her confreres when the FBI swept in. And that's what is proposed for large unions by Labor Secretary Elaine Chao, who has proposed major changes in form LM-2, which unions file annually with the Labor Department to say how they've spent their money. Today, it's a nebulous document with big categories and little detail. Ms. Chao would have unions explain each outlay of $5000 or more. That's like disaggregating the NCLB test scores so that everyone can see how the kids in a school are doing. But guess who's opposed? The Washington Teachers' Union's parent AFT and its grandparent, the AFL-CIO. The latter's chief, John Sweeney, insists that the revised LM-2 is "craftily designed to weaken unions." But how, then, and to whom, would unions be accountable? Or aren't they?
Union chief led by quashing dissent, by Justin Blum, Washington Post, October 7, 2003,
Former leader of D.C. union charged in theft, by Carol D. Leonnig and Allan Lengel, Washington Post, October 4, 2003
Teachers union ex-chief pleads guilty, by Carol D. Leonnig, Washington Post, October 8, 2003
Union doozy, editorial, Wall Street Journal, October 8, 2003 (subscription required)
Christopher B. Swanson, Education Policy Center, The Urban Institute
August 2003
This clear and concise report, initially presented at a conference sponsored by the Harvard Civil Rights Project (and one of the few interesting reports to emerge from that ill-starred venture), offers a method for calculating graduation rates that states may find useful now that NCLB requires them to include such measures in their accountability plans. Calculating a graduation rate may seem simple, but is actually fraught with difficulty. Thus the "conventional wisdom" from the National Center for Education Statistics (NCES) - that U.S. graduation rates hover between 85 and 90 percent - is deeply misleading. Superior methods show that the true rate is, sadly, closer to two-thirds. Here, Swanson presents one of these methods, which he developed, and recaps the second, developed earlier by Manhattan's Jay Greene. [see http://www.edexcellence.net/gadfly/issue.cfm?issue=27#153] Both methods consider how many students actually graduate; NCES, on the other hand, relies on drop-out rates, which are notoriously flawed. This is no mere academic quarrel. States need a sound methodology to meet the spirit of NCLB, but unfortunately too many states are instead opting for the NCES approach, which meets the barest letter of the law but surely violates its intent. [For each state's plans see Swanson's separate NCLB Implementation Report] Both Swanson's and Greene's methods rely on more readily available data. Greene's may be slightly more accurate while Swanson's is easier to calculate, but both represent a quantum improvement over the NCES approach in terms of accuracy and data availability. States would be remiss to ignore them. And everyone should note the troubling results they reveal - particularly that barely half of minority youngsters graduate from high school in America today. And that in some states the average student has only a 50-50 chance of graduating. To learn more, click here.
Christopher Mazzeo, National Governors Association's Center for Best Practices
September 2003.
This constructive ten-pager discusses what states can do to strengthen the leadership (principals, mainly) of their public schools. It tackles three big issues - licensure, preparation, and professional development - and offers valuable insights and sage advice on all three. This includes removing "barriers for talented individuals to enter the profession," establishing "alternative principal preparation programs," and tailoring the professional development of school leaders to the expectations of NCLB. You can find it online here.
Julian R. Betts and PPIC's Andrew C. Zau and Lorien A. Rice, Public Policy Institute of California (PPIC)
2003
This short book from the Public Policy Institute of California (PPIC) provides a close analysis of data from San Diego and seeks to illumine equalities and inequalities in school resources; trends in pupil achievement and achievement gaps; and, especially, which factors in school and classroom have the most influence on changes in student performance. On the first point, the authors found that the main resource discrepancy between affluent and non-affluent elementary schools in San Diego is teacher experience and preparation. On the second point, they unearthed "shocking" achievement gaps in all the usual directions - but also found nearly all of those gaps narrowing between 1998 and 2000. (That's before the onset of the much-publicized "Blueprint for Student Success" reform strategy of Alan Bersin and Tony Alvarado.) What turns out to influence student achievement from school to school and year to year? There's much technical stuff here, though Betts & Co. do an exceptionally nice job of making it accessible to ordinary mortals. It turns out - no big surprise - that attending school actually matters, i.e. kids who are absent a lot learn less. Peer group matters, too, but more at the grade level than the classroom level: the higher achieving a child's grade-mates, the better he/she is apt to do. Class size matters some, although only in reading in the early grades. And teachers matter, but the usual ways of defining their "qualifications" matter only intermittently. In what may turn out to be their most controversial conclusion, they say, "In very few cases did we find any statistically significant difference between the effectiveness of fully credentialed teachers with ten or more years of experience and teachers with less experience, regardless of whether they held a full or emergency credential or an internship." (The big exception: high-school math, where "subject authorization level" matters greatly.) Perhaps most important: schools don't generally cause achievement gaps. They're present when kids arrive at school. Policy makers may therefore want to give greater emphasis to pre-school. Though it's a case study of a single district, this book is so careful and perceptive that you may well want a copy. It's hard to believe these findings don't apply elsewhere. The ISBN is 1582130442 and you can get more information at http://www.ppic.org/main/publication.asp?i=321.
Karl T. Kurtz, Alan Rosenthal, and Cliff Zukin, National Conference of State Legislatures
September 2003
This report publishes the results of a survey conducted by the National Conference of State Legislatures, the Center for Civic Education, and Indiana University's Center on Congress. The survey focused on civic education and the awareness of the American public, with questions concerning citizen's political knowledge as well as levels of participation and involvement in politics and community. By comparing the results of different generations, it shows a significant generation gap in civic knowledge. There is a clear decline in understanding and civic participation among our nation's youth - dubbed "DotNets," aged 15-26, - which arguably contributes to decreased engagement in local, state, or national politics. Only 47 percent of DotNets vote, compared to 77 percent of older generations. Just 48 percent knew the political party of their current governor and only 40 percent knew which party controlled Congress. Interestingly, DotNets who had taken civics classes were more than twice as likely to follow politics. This report is informative and helpful to those designing civics programs for young people. Check it out at http://www.ncsl.org/public/trust/citizenship.pdf.
YMCA, Dartmouth Medical School and the Institute for American Values
2003
It's never been easy to get your mind around what James Coleman called "social capital" and what the Commission on Children at Risk, in this provocative new report, terms "connectedness." It says the deteriorating mental and behavioral health of U.S. children is a real problem but we're barking up the wrong tree when we seek to solve it primarily through medication, therapy, and special programs for "at-risk" children. What's needed instead - or in addition - say the 33 members of this commission (a joint venture of the YMCA, Dartmouth Medical School, and the Institute for American Values), is greater attention to the "broad environmental conditions that are contributing to growing numbers of suffering children." "In large measure," they find, "what's causing this crisis of American childhood is a lack of . . . close connections to other people and deep connections to moral and spiritual meaning." Their antidote: children should live in "authoritative communities." Many pages are devoted to setting forth the essential characteristics of such communities, which include: establishing "clear limits and expectations," transmitting to community members "a shared understanding of what it means to be a good person," and encouraging "spiritual and religious development." It's a bit like reinventing "civil society" at the community level and urging that we raise our children within that society. Of particular merit is the scientific evidence undergirding the commission's diagnosis and prescription. The report is 82 pages (not counting 18 commissioned papers) and you can find it at several places, including http://www.americanvalues.org/html/hardwired.html.