For better or for worse, Education Week’s annual Quality Counts feature, now celebrating its thirteenth birthday, is the closest thing we have to a comprehensive annual report card on American K-12 education. Unfortunately, it’s fraught with methodological weaknesses.
For better or for worse, Education Week’s annual Quality Counts feature, now celebrating its thirteenth birthday, is the closest thing we have to a comprehensive annual report card on American K-12 education. Unfortunately, it’s fraught with methodological weaknesses. (Last year’s was too.) The state-by-state grades--which will inevitably grab most of the headlines--are actually the least reliable component. The grading is at times arbitrary (teacher experience is used to measure teacher quality, an unproven correlation), unfair (is the annual income of someone who graduated high school in 1965 really a good reflection of a state’s education system in 2010?), and redundant (NAEP scores are factored into both the Chance for Success and K-12 Achievement indices, and both children’s family income and statewide adult income are factored into Chance for Success). Such problems are nicely illuminated by CREDO’s Margaret Raymond in a new piece in Education Next. Raymond also recalculates some of the state grades on a fairer scale, with striking results. We’re glad to see people are beginning to cast a critical eye on QC. Still and all, the raw data that EPE gathers are valuable and reward analysis. Also deserving a look is this year’s focus theme--common standards, of course--and a new “Math Progress Index.” The report is available here--free to online subscribers, or in print for a small fee.
Editorial Projects in Education
January 2010
Finally, a strong, modern, quality-centered metric by which to judge the strength of state charter laws! The Center for Education Reform did pioneering and valuable work in this area but its metric is limited and somewhat archaic, having more to do with schools’ freedom from regulation than with their performance or their states’ accountability expectations. In this new report, the Alliance compares actual state charter laws with a model law that they developed, and ranks the forty states with such laws on elements like quality, accountability, and funding equity. “Quality control” measures (e.g., requiring performance-based contracts and having comprehensive charter school monitoring and data collection processes) are weighted the most heavily. Minnesota, DC, and California came out on top; Iowa, Alaska, and Maryland at the bottom. The indicators are presented separately so that state policyma kers, charter advocates, foes, and researchers alike can zero in on the elements that matter most to them. There’s a minor methodological quibble, however, which is more one of adequate explanation than analytical error; NAPCS fails to differentiate which scores derive specifically from state laws and which also derive from practice, which they took into consideration in three of the categories (e.g., on charter caps, they consider also if the state is at or approaching the cap). You can find the report itself and an accompanying online database of its contents here.
National Alliance for Public Charter Schools
January 2010
If you read only the coverage of this study, you’d come away with a vastly more negative view of TFA graduates than you should. So let’s put the record straight. The study asks whether the TFA experience “make[s] citizens”? The authors seek to compare the “civic engagement” of TFA “graduates” (those who finished two years of TFA), “dropouts” (those who began the program but did not finish it) and “non-matriculants” (those who were accepted but declined). They find that though graduates display the highest attitudinal scores on civic awareness, non-matriculants actually perform the most service. They attribute this difference to such factors as post-TFA burnout and disillusionment with TFA itself. That may be so, but TFA is an organization dedicated to recruiting and keeping talent in our nation’s classrooms, not creating Mother-Teresas-in-tr aining, even if Wendy Kopp commissioned the study herself. It is upon those metrics--and the effect that TFA teachers have on student achievement--that we should be evaluating the program. Even if you accept the premise of this inquiry, however, the findings are no black mark against TFA. For example, though “only” 89 percent of graduates said they voted in the last presidential election (compared to 92 percent for all three groups), that’s still 50 points higher than their similarly-aged peers. The lesson is that TFA doesn’t turn disengaged students into participatory citizens; rather, it tends to select the civic-minded from the get-go. You can order a copy here.
Doug McAdam and Cynthia Brandt, Stanford University
Social Forces
February 2010
This report adds to the pile of evidence that good charter schools elicit positive student achievement--and that something about New York City’s approach is worth understanding if not emulating. A follow-up to its June 2009 national study, CREDO undertook this one exclusively in Gotham. Lo and behold, it found that NYC charters have a significant positive effect on overall student achievement. In math, 51 percent of charter schools showed learning gains statistically larger than those shown by traditional district-operated schools, while 33 percent showed no difference and 16 percent fared worse. Gains in reading were less striking but still impressive. The study also disaggregated results by the number of years a student attended a charter, and found that, while first year charter pupils perform slightly worse than their district agemates in reading (and better in math), after three years their gains significantly outpace traditional school pupils in both subjects. Moreover, black and Hispanic charter school students perform significantly better than their traditional school counterparts. You can read it here.
Center for Research on Education Outcomes
January 2010
Human capital discussions in education nowadays typically start with the problem of “incompetent” teachers and what to do about them. The very notion of wholly-inept instructors in children’s classrooms inflames the emotions (think Steven Brill’s colorful New Yorker piece) and sometimes leads on to discussion of such other HR issues as tenure, compensation, hiring and firing rules, and evaluation. But “incompetents” are also a sort of distraction, as nobody thinks they comprise more than a tiny fraction of the total teacher workforce. When it comes to reforming HR practices, we might be better served to start with the challenges of mediocre-to-average teachers (and principals, too, for that matter) as they are far more numerous and cost taxpayers--and children--far more.
Teacher salaries and benefits typically comprise at least half of education budgets—and other school employees bring the total personnel cost to about three-fourths. At our “Penny Saved” conference on Monday, devoted to finding (and illustrating) ways that school systems can produce better results while spending less money, HR issues came up in virtually every discussion. It was evident that any serious budgetary reform in education is going to involve a serious rethinking and resizing of school staffing. So will any serious quality reforms. (You can access the conference papers here. They repay attention.)
Over the last forty years, we increased the size of the U.S. teaching profession both in absolute terms (47.9 percent between 1980 and 2006, for example) and relative to student enrollment numbers (the pupil-teacher ratio dropped from the high twenties to mid-teens since the mid-1950s). Though per-pupil spending rose dramatically, individual teacher salaries barely budged (in real terms). That’s because America opted for quantity over quality. (This point has been made before.) And at no point along the line did we--our public-school system, anyway--develop the kinds of evaluation systems we would need if we were winnowing the teacher corps according to performance or effectiveness. Almost no winnowing occurred, so why develop winnowing procedures, which, in any case, teachers and their powerful unions disliked?
But budgetary woes, Race to the Top, and technology are starting to turn the tide. Not only are teacher evaluations becoming methodologically stronger, but actually using them in personnel decisions is beginning to catch on. Not fast, to be sure, but there’s every reason to expect early moves to lead to more.
Two tactics are getting a bit of traction: removing seniority as a criterion of RIF-ing procedures and assignment practices, and breaking down firewalls between student performance data and teacher evaluations. On the first count, the Arizona legislature has forbidden school systems to use seniority and tenure in making firing decisions. This means districts can consider such factors as classroom effectiveness in making those choices and can (and perhaps must) renegotiate terms with local teacher unions on the subject. Rhode Island State Superintendent Deborah Gist didn’t go quite as far when she banned seniority bumping rights in the Ocean State but California Governor Arnold Schwarzenegger is currently drafting legislation that would loosen the state’s seniority-driven firing rules and eliminate a state-run appeals process that typically overturns a third of teacher dismissals.
As for linking student data and teacher evaluations, Arne Duncan has given this a considerable boost both by jawboning about it and by linking states’ Race to the Top eligibility to it (at least to not having laws that ban it). In November, Wisconsin received plaudits for knocking down the wall between student data and teacher evaluations; unfortunately, almost in the same breath, the state also banned the use of those data-driven teacher evaluations in teacher firing decisions. Perhaps Houston superintendent Terry Grier will have better luck. He hopes to implement a system where the value-added teacher evaluation model that the district currently uses to dole out performance bonuses will also be used in firing decisions. He sought to do that in his last post in San Diego but ran into a brick wall. Doing it well, of course, depends on having a well-engineered and accurate evaluation system, both in terms of fairly gauging teacher effectiveness and having the data infrastructure to back it up.
Performance evaluations of teachers have historically been infrequent and perfunctory, because the alternative w as costly and contentious. In a study of four states, The New Teacher Project found last year that an absurd 99 percent of teachers receive A’s or the equivalent on such evaluations. Last month we learned that Detroit teachers hadn’t been evaluated at all in more than a decade. Gist mandated earlier this year that Rhode Island districts revamp their teacher evaluation systems, according to a set of new standards developed by her office, and use them annually, a huge jump in frequency from previous practice.
We learned the other day that Teach For America has been collecting data on its (now) 7,300 corps members in various forms for the last twenty years. Since 2002, it’s been refining its recruitment and selection process to favor candidates who reflect a few “superstar” tendencies, such as setting big goals for themselves, constantly reevaluating their practice, meticulous and outcome-orientated planning, and relentlessness or “grit.” The real meat of TFA’s painstaking and highly-selective admissions process, which uses such data to predict potential corps members’ classroom success, will become public next month. And overhauls like Gist’s will likely inform other states’ attempts to make similar changes to their evaluation systems. The black box that is the classroom is now dimly illuminated by a sliver of light.
Opening the shades a bit wider, we find the AFT’s Randi Weingarten offering to reform teacher evaluations. In a National Press Club speech this week (see below), she said that our obsession with ineffective teachers “fails to recognize that we have a systems problem.” She went on to outline a four-part plan to overhaul teacher evaluations and dismissal procedures. Weingarten has thus given local affiliates cover to agree to more robust evaluation systems. Which doesn’t necessarily mean they will. But still and all, her words matter, because without giving principals and district leaders the ability to use evaluations in teacher staffing and firing decisions, they will remain largely meaningless.
We’re seeing slow progress. We have better evaluations, better data systems, state and local leaders somewhat more willing to use them, and a union whose chief says she’s willing to help make this happen. In the long run, changes in teacher evaluation could yield fiscal savings, too, as well as better use of the available personnel dollars. After all, if seismic changes in education finance are destined to force states and districts to rethink their HR budgets, shouldn’t we make sure that the teachers we engage, pay, and retain are the best ones?
In her Tuesday speech at the Press Club, AFT President Randi Weingarten attempted to take the teacher-policy steering wheel back from Arne Duncan, who’s been driving since the Race to the Top motoring began. The big news is her willingness to reconsider due process rules and to revamp teacher evaluations. Ms. Weingarten also recognized that her members value quality teaching above job protection. This is all welcome and deserves applause. But much of her speech was decidedly not news. Lots of longstanding union agenda items were restated, albeit with new rhetoric (“tools, time, and trust”) and some important reforms were conspicuously absent (e.g. tying these new evaluations to compensation). Also worth noting is Ms. Weingarten’s clever attempt to co-opt and redirect one of the most effective sallies against her organization’s agenda. For years, reformers have charged that union-backed uniform pay scales, swift tenure, and copious job protections were relics of the bygone “industrial age” in K-12 education. In the speech, however, she re-defined this “age” as one of standardized testing. Finally, it’s worth noting the timing. In the last few weeks, many unions have been scolded in local media outlets for opposing their states' RTT applications and rejecting important reforms. You don’t have to be a cynic to wonder whether this speech was about repositioning the AFT.
“A New Path Forward: Four Approaches to Quality Teaching and Better Schools,” speech given by Randi Weingarten, National Press Club, Washington, D.C., January 12, 2010
Some things are just not surprising: Mark McGwire’s steroid usage, the incompetency suit against Octomom’s fertility doctor, and states backing off graduation test requirements in response to political pressure. Twenty-six states now use passing some kind of test as a diploma requirement, and most if not all of them have faced political pressure that has led to weaker tests, lower standards, and myriad alternative routes to graduation. Some simply pushed off the implementation date, fearing plummeting graduation rates. The result, says one expert, is that “the exams are just challenging enough to reduce the graduation rate, but not challenging enough to have measurable consequences for how much students learn or for how prepared they are for life after high school.” Graduation rates are tricky things, of course, but revelation that the politics of graduation rates are all but unmanageable in state capitals makes one wonder about the upcoming use of the “common core” assessments that Secretary Duncan is about to pay for. They’re supposed to be aligned with standards that are supposed to be aligned with college readiness. But why should anyone suppose that states that sign on to use them will suddenly grow the backbone to actually hold students to college-readiness-level passing scores on them?
“As School Exit Test Prove Tough, States Ease Standards,” by Ian Urbina, New York Times, January 12, 2010
For better or for worse, Education Week’s annual Quality Counts feature, now celebrating its thirteenth birthday, is the closest thing we have to a comprehensive annual report card on American K-12 education. Unfortunately, it’s fraught with methodological weaknesses. (Last year’s was too.) The state-by-state grades--which will inevitably grab most of the headlines--are actually the least reliable component. The grading is at times arbitrary (teacher experience is used to measure teacher quality, an unproven correlation), unfair (is the annual income of someone who graduated high school in 1965 really a good reflection of a state’s education system in 2010?), and redundant (NAEP scores are factored into both the Chance for Success and K-12 Achievement indices, and both children’s family income and statewide adult income are factored into Chance for Success). Such problems are nicely illuminated by CREDO’s Margaret Raymond in a new piece in Education Next. Raymond also recalculates some of the state grades on a fairer scale, with striking results. We’re glad to see people are beginning to cast a critical eye on QC. Still and all, the raw data that EPE gathers are valuable and reward analysis. Also deserving a look is this year’s focus theme--common standards, of course--and a new “Math Progress Index.” The report is available here--free to online subscribers, or in print for a small fee.
Editorial Projects in Education
January 2010
Finally, a strong, modern, quality-centered metric by which to judge the strength of state charter laws! The Center for Education Reform did pioneering and valuable work in this area but its metric is limited and somewhat archaic, having more to do with schools’ freedom from regulation than with their performance or their states’ accountability expectations. In this new report, the Alliance compares actual state charter laws with a model law that they developed, and ranks the forty states with such laws on elements like quality, accountability, and funding equity. “Quality control” measures (e.g., requiring performance-based contracts and having comprehensive charter school monitoring and data collection processes) are weighted the most heavily. Minnesota, DC, and California came out on top; Iowa, Alaska, and Maryland at the bottom. The indicators are presented separately so that state policyma kers, charter advocates, foes, and researchers alike can zero in on the elements that matter most to them. There’s a minor methodological quibble, however, which is more one of adequate explanation than analytical error; NAPCS fails to differentiate which scores derive specifically from state laws and which also derive from practice, which they took into consideration in three of the categories (e.g., on charter caps, they consider also if the state is at or approaching the cap). You can find the report itself and an accompanying online database of its contents here.
National Alliance for Public Charter Schools
January 2010
If you read only the coverage of this study, you’d come away with a vastly more negative view of TFA graduates than you should. So let’s put the record straight. The study asks whether the TFA experience “make[s] citizens”? The authors seek to compare the “civic engagement” of TFA “graduates” (those who finished two years of TFA), “dropouts” (those who began the program but did not finish it) and “non-matriculants” (those who were accepted but declined). They find that though graduates display the highest attitudinal scores on civic awareness, non-matriculants actually perform the most service. They attribute this difference to such factors as post-TFA burnout and disillusionment with TFA itself. That may be so, but TFA is an organization dedicated to recruiting and keeping talent in our nation’s classrooms, not creating Mother-Teresas-in-tr aining, even if Wendy Kopp commissioned the study herself. It is upon those metrics--and the effect that TFA teachers have on student achievement--that we should be evaluating the program. Even if you accept the premise of this inquiry, however, the findings are no black mark against TFA. For example, though “only” 89 percent of graduates said they voted in the last presidential election (compared to 92 percent for all three groups), that’s still 50 points higher than their similarly-aged peers. The lesson is that TFA doesn’t turn disengaged students into participatory citizens; rather, it tends to select the civic-minded from the get-go. You can order a copy here.
Doug McAdam and Cynthia Brandt, Stanford University
Social Forces
February 2010
This report adds to the pile of evidence that good charter schools elicit positive student achievement--and that something about New York City’s approach is worth understanding if not emulating. A follow-up to its June 2009 national study, CREDO undertook this one exclusively in Gotham. Lo and behold, it found that NYC charters have a significant positive effect on overall student achievement. In math, 51 percent of charter schools showed learning gains statistically larger than those shown by traditional district-operated schools, while 33 percent showed no difference and 16 percent fared worse. Gains in reading were less striking but still impressive. The study also disaggregated results by the number of years a student attended a charter, and found that, while first year charter pupils perform slightly worse than their district agemates in reading (and better in math), after three years their gains significantly outpace traditional school pupils in both subjects. Moreover, black and Hispanic charter school students perform significantly better than their traditional school counterparts. You can read it here.
Center for Research on Education Outcomes
January 2010
