The Education Gadfly Weekly

Spanning a manageable 2,000 pages, this sixth edition of the National Council on Teacher Quality’s (NCTQ’s) annual teacher-policy yearbook focuses attention on states’ teacher-preparation policies (one of five areas tracked by NCTQ as part of this initiative). And, once again, NCTQ finds them wanting. Across the items investigated (including the rigor of admission requirements in teaching programs, student-teaching expectations, and accountability systems linked to the performance of prep programs’ alumni when they reach the classroom), the U.S. averages a D-plus. Only four states earn respectable marks (still a meager B-minus): Alabama, Florida, Indiana, and Tennessee. Three others (Alaska, Montana, and Wyoming) earn Fs. Looking closely at specific policies is even more depressing: Just three states (Indiana, Minnesota, and Tennessee) require high school teachers to pass content-area tests in their subjects without allowing loopholes (most of which are for math and science teachers). And Texas is the only state that norms its admissions exam to the general college-bound population (all others norm it to the prospective teaching population, setting a lower bar than for other college and university students). Still, NCTQ acknowledges that states are slowly moving in the right direction. In 2007, when the organization began scrutinizing these data, no state held its prep programs accountable for the quality of their graduates; today, eight do. And since 2011, fourteen states (including Ohio) have improved their teacher-preparation policies in some way. Kudos to NCTQ for continuing to spotlight one of education reform’s greatest ironies: that the work to improve the teaching profession often is done without thought to improving the quality of the folks we admit into it in the first place. Look for more from them in the spring with the release of their Teacher Prep Review. (Indeed, if you can’t wait, this year’s policy yearbook offers a few “sneak peeks.”)

SOURCE: National Council on Teacher Quality, 2012 State Teacher Policy Yearbook (Washington, D.C.: National Council on Teacher Quality, January 2013).

This report by Stanford’s Martin Carnoy and the Economic Policy Institute’s Richard Rothstein offers a catchy press-release headline: The U.S. Fares Better on International Assessments than Previously Thought. But that isn't actually true. Analyzing PISA data, Carnoy and Rothstein argue that the U.S. educates its disadvantaged students about as well as similar nations—and, for that, America should be praised. But the problems with the study are myriad. First, the authors use a “very approximate” index—the number of books in a student’s home—to determine social class. Others have explained the methodological flaws with this approach. Second, the authors engage in some dangerous statistical gymnastics to prove their point: Based on the assumption that students of low “social class” bring down average U.S. scores, Carnoy and Rothstein re-estimate PISA attainment (by using the books-in-the-home index) to norm the proportion of students in each class. They find that, if the U.S. had the same proportion of students in lower social classes as other nations, then it would rank fourth in reading (instead of fourteenth) and tenth in math (instead of twenty-fifth). The conclusions of this report only affirm the very significant education problem that it’s trying to downplay: We have a greater proportion—and a significantly greater number—of low-scoring and low-income students than other OECD countries. Carnoy’s and Rothstein’s flawed analysis and misleading primary conclusion is at best a diversionary ploy.

SOURCE:Martin Carnoy and Richard Rothstein, What do International Tests Really Show About U.S. Student Performance? (Washington, D.C.: Economic Policy Institute, January 15, 2013).

The teacher-evaluation debate follows a well-worn path: Traditional evaluation systems (in which upwards of 99 percent of educators are deemed “effective”) are meaningless, argue reformers. New models that rely heavily on value-added test-score data are unreliable and unfair, counter others. This new NBER working paper from Northwestern’s C. Kirabo Jackson provides the debate new turf on which to tread: Based on data from the 1988 National Educational Longitudinal Study, Jackson channels Paul Tough to argue that students’ “non-cognitive” abilities (adaptability, self-restraint, motivation) help explain their success. Noting this, teachers should be evaluated on them; yet they are rarely considered by current metrics. The report has two parts. First, Jackson shows that the “non-cognitive factor” (which he proxies with variables like absenteeism, suspensions, and grades) is predictive of college enrollment and lifetime earnings—more so, in fact, than cognitive ability. Jackson then evaluates whether teachers can affect this “non-cognitive” factor. Using 2005-10 North Carolina data, he finds that teachers’ impact on student test scores is only weakly associated with their impact on improving youngsters’ non-cognitive abilities. In other words, evaluations that rely exclusively on test scores fail to capture the full breadth of teachers’ contributions to student outcomes. Jackson concludes: Other variables that assess ability to improve students’ non-cognitive skills, variables such as student suspensions and absences, should also be used in teacher assessments. And the debate marches on.

SOURCE: C. Kirabo Jackson, "Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina" (Chicago, IL: Northwestern University, January 2013).

This book awakens an established but sparingly practiced and often unknown initiative in K–12 education: teacher autonomy. Authors profile eleven schools (seven of them charters and three of them in Minnesota, a pioneer in the "teacher-led-schools" initiative) that embrace teacher autonomy to differing degrees and study the policies and practices by which they operate. Ten criteria are used to judge the autonomy level of the teachers including their agency over: staff hiring and firing decisions, budget allocations, curriculum design, and school-wide discipline policies. Written for teachers, the book—a worthwhile primer on what teacher autonomy is and what its many forms look like—offers an illustrative blueprint for one manner in which teachers may be empowered, rather than alienated or demonized, by the reform movement. Still, the book unconvincingly handles one key component of a worthy teacher-autonomy policy: Before you give teachers the keys to the castle, make sure you have royalty in the profession.

SOURCE: Kim Farris-Berg and Edward Dirswager with Amy Junge, Trusting Teachers with School Success (Lanham, MD: Rowman & Littlefield Education, 2012).

Mike and Kathleen are skeptical about the President’s education agenda and newly released high school graduation rate data. Amber thinks about low-income high-flyers.

Amber's Research Minute

The Missing "One-Offs": The Hidden Supply of High-Achieving, Low Income Students by Caroline M. Hoxby and Christopher Avery, The National Bureau of Economic Research

The Common Core is coming, with forty-five states and the District of Columbia challenged to implement these new standards. Yet mystery surrounds how much this will cost states (and districts)—and whether the payoff will justify the price.

On May 30, the Fordham Institute will peek behind that curtain with a lively panel discussion of "Pricing the Common Core." Taking part will be former Florida Education Commissioner Eric J. Smith, Achieve President Mike Cohen, former Department of Education official Ze'ev Wurman, and University of San Francisco professor Patrick J. Murphy, who will present the findings of a new Fordham study that he co-authored. It estimates the dollar cost of the implementation process for each participating state—and shows how the pricetag varies depending on the approach a state selects.

Tests in use from Kindergarten through eleventh grade need to have cut scores that denote true readiness for the next grade and that culminate to "college and career readiness." Photo by albertogp123

As the U.S. education world eagerly awaits more information about the new assessments that two consortia of states are developing to accompany the Common Core standards, dozens of perplexing and important questions have arisen: Once the federal grants run out, how will these activities be financed? What will it cost states and districts to participate? Who will govern and manage these massive testing programs? What about the technology infrastructure? The list goes on.

The assessment questions that weigh most heavily on my mind these days, however, involve “cut scores.” For if the Common Core is truly intended to yield high school graduates who are college and career ready, its assessments must be calibrated to passing scores that colleges and employers will accept as the levels of skill and knowledge that their entrants truly need to possess. Adequately equipping young people cannot wait ‘til twelfth grade, nor can the assessment sequence. The tests in use from Kindergarten through eleventh grade need to have passing scores that denote true readiness for the next grade and that cumulate to “college and career readiness.”

That’s a daunting challenge for any test maker, but it’s further complicated by widespread fears of soaring failure rates and their political consequences, as well as by Arne Duncan’s stipulation (in the federal grants that underwrite the assessment-development process) that the states belonging to each consortium must reach consensus on those passing scores (in government jargon, “common achievement standards”). All this means, in effect, that Oregon and West Virginia (both members of the “Smarter Balanced” consortium) must agree on “how good is good enough” for their students, as must Arkansas and Massachusetts (both members of PARCC). Can that really happen?

The angst is palpable among state officials—especially the elected kind—over the threat of soaring failure rates, and not just among the poor and dispossessed. We already know from national assessment data that about half of eighth graders with college-educated parents fail to clear the “proficient” bar on NAEP. If (as mounting evidence suggests) “NAEP proficient” is roughly equivalent to “college ready,” and if the new assessments hew to that level of rigor and honesty, many millions of American youngsters will be found unready—and millions more will learn that they’re not on track toward readiness. Such a cold shower should benefit the nation over the long haul, but in the short run, it’s going to feel icy indeed.

Yet that’s only the start. Here are some other perplexing challenges in this realm:

• When will the chilly water hit? PARCC says it doesn’t intend to do any level-setting until after scores come in from the first test-administration in 2015, a classic example of psychometric considerations overriding real-world political considerations. How can states and districts possibly prep their students, their educators, and their publics for new standards (and heightened risk of failure) if nobody knows in advance what sort of performance will be deemed passable? The folks at Smarter Balanced say they’ll set cut scores in advance of that first administration but “confirm” them afterward. What exactly does that mean?

• A single cut-score or several? NCLB says states must set at least three cut scores on their assessments, but the Education Department’s RFP for the new Common Core assessments makes no such demand. It refers only to “college and career readiness.”

  I had the honor of chairing the National Assessment Governing Board (NAGB) twenty years back when it was struggling to set “achievement levels” for NAEP. Al Shanker (and others) warned that we’d be wrong to establish a single level. It would either be too high, he cautioned, in which case far too few kids would reach it, or too low, meaning it would amount to little improvement over state-set “minimum-competency” levels. In the end, NAGB set three achievement levels, familiar today as “basic,” “proficient,” and “advanced.” We declared “proficient” to be the level that every student ought to reach—but then, as now, fewer than two in five actually achieved that. “Basic” was intended to be a solid marker along the road to proficient and “advanced” was meant to represent “world class”. Will the Common Core assessments do something similar?

• Will these cut scores be static or will they rise over time? Some states (Texas comes to mind) have had good results by slowly elevating the cut scores on their own assessments. Doing something of the sort with the new Common Core assessments would ease the political backlash. On the other hand, the promise of “college and career readiness” would then remain hollow for years to come.

• Speaking of which, who is doing what to ensure that colleges and employers will actually accept these standards—and cut scores—as evidence of readiness? And how exactly is “readiness” being construed? (The federal procurement was nebulous and, while the Common Core standards themselves are substantively very ambitious, the “cut scores” on their assessments are not obliged to be.) For college-bound youngsters, I see “readiness” as the ability to enter credit-bearing courses in the appropriate subjects on non-selective-admission campuses—i.e., to avoid remediation. But I’ve no idea how to define “career readiness.” Who does?

• What happens in states (about half of them) that already have statewide graduation tests (e.g., Massachusetts Comprehensive Assessment System and the Ohio Graduation Test) with minimum passing scores? These have typically been pegged to levels of academic prowess around old-style ninth- or tenth-grade curricula and have been required (of almost all students) as prerequisite to getting a diploma. Are said states supposed to replace these with the new assessments pegged (presumably) to much tougher demands? Administer both? What about a two-tiered diploma system, at least for awhile? Set the new cut scores high to denote true “readiness,” get the colleges to accept them as such, and confer a “readiness diploma” on youngsters who meet that standard. But for some period of time also continue to hand out “regular” diplomas to those who meet the state’s existing graduation requirements, typically a mix of course completions, test scores, and the like. Phase out the latter if possible. (In the old days, New York State did something like this, distinguishing between an ordinary diploma and a “Regents’ Diploma.”) 

• As the “credit-recovery” industry grows, often abetted by online delivery of make-up courses and such, how will such arrangements intersect with the new standards, assessments, and cut scores? (The same question may fairly be asked of the G.E.D.) It may turn out that stiffer academic expectations drive more kids to flunk, drop out, etc., and that credit recovery becomes more important than ever. But will that path turn out to be a short cut to a meaningless diploma or an honorable avenue to meet the higher standards and demonstrate one’s “readiness” for job and college?

• If the ACT and College Board folks build the Common Core into their widely used college-admissions tests, as seems likely, will these new consortium-based assessments even be needed at the high school level? For that matter, what would a “cut score” on the SAT or ACT look like?

I hope some smart people are figuring all this out—but if they are, the answers haven’t yet reached my eyes or ears. Time is getting short.

In his second inaugural address, President Barack Obama mentioned two pieces of his K–12 policy agenda: his plans to train new math and science teachers and his plans to improve school safety. Politics K–12 notes that inaugural addresses are not typically policy-laden, so one can fairly infer that these two items top his second-term to-do list. In this week’s Education Gadfly Show, Mike Petrilli—self-professed “koala dad”—expresses unease over placing STEM education on a pedestal over all other subjects.

Last Friday, a federal appeals court upheld Wisconsin Governor Scott Walker’s public-sector-union reforms in full, rejecting the unions’ charges that the law violated the Equal Protection clause and the First Amendment. But according to the School Law blog, the practical effect of the ruling is “unclear” due to litigation in a separate state court. We will be watching.

A fresh batch of federal data shows that the U.S. public high school graduation rate rose to 78.2 percent in 2010—a thirty-five-year high. But before you bake Arne Duncan a cake and sing “For He’s A Jolly Good Fellow,” be sure to listen to this week’s Education Gadfly Show for a wee slice of humble pie. Has our fixation on graduation rates incentivized schools to cheapen the value of diplomas—say, with bogus credit-recovery programs?

After the ouster of Indiana State Superintendent Tony Bennett (who, subsequently, was snapped up by Florida), Hoosier Republicans began to push for the state to withdraw from the Common Core. Last week, Fordham’s Mike Petrilli visited the Indiana State Senate to make the case for staying the course with the standards. Read his full testimony on Flypaper.

The Common Core is coming, with forty-five states and the District of Columbia challenged to implement these new standards. Yet mystery surrounds how much this will cost states (and districts)—and whether the payoff will justify the price.

On May 30, the Fordham Institute will peek behind that curtain with a lively panel discussion of "Pricing the Common Core." Taking part will be former Florida Education Commissioner Eric J. Smith, Achieve President Mike Cohen, former Department of Education official Ze'ev Wurman, and University of San Francisco professor Patrick J. Murphy, who will present the findings of a new Fordham study that he co-authored. It estimates the dollar cost of the implementation process for each participating state—and shows how the pricetag varies depending on the approach a state selects.

The Common Core is coming, with forty-five states and the District of Columbia challenged to implement these new standards. Yet mystery surrounds how much this will cost states (and districts)—and whether the payoff will justify the price.

On May 30, the Fordham Institute will peek behind that curtain with a lively panel discussion of "Pricing the Common Core." Taking part will be former Florida Education Commissioner Eric J. Smith, Achieve President Mike Cohen, former Department of Education official Ze'ev Wurman, and University of San Francisco professor Patrick J. Murphy, who will present the findings of a new Fordham study that he co-authored. It estimates the dollar cost of the implementation process for each participating state—and shows how the pricetag varies depending on the approach a state selects.

Spanning a manageable 2,000 pages, this sixth edition of the National Council on Teacher Quality’s (NCTQ’s) annual teacher-policy yearbook focuses attention on states’ teacher-preparation policies (one of five areas tracked by NCTQ as part of this initiative). And, once again, NCTQ finds them wanting. Across the items investigated (including the rigor of admission requirements in teaching programs, student-teaching expectations, and accountability systems linked to the performance of prep programs’ alumni when they reach the classroom), the U.S. averages a D-plus. Only four states earn respectable marks (still a meager B-minus): Alabama, Florida, Indiana, and Tennessee. Three others (Alaska, Montana, and Wyoming) earn Fs. Looking closely at specific policies is even more depressing: Just three states (Indiana, Minnesota, and Tennessee) require high school teachers to pass content-area tests in their subjects without allowing loopholes (most of which are for math and science teachers). And Texas is the only state that norms its admissions exam to the general college-bound population (all others norm it to the prospective teaching population, setting a lower bar than for other college and university students). Still, NCTQ acknowledges that states are slowly moving in the right direction. In 2007, when the organization began scrutinizing these data, no state held its prep programs accountable for the quality of their graduates; today, eight do. And since 2011, fourteen states (including Ohio) have improved their teacher-preparation policies in some way. Kudos to NCTQ for continuing to spotlight one of education reform’s greatest ironies: that the work to improve the teaching profession often is done without thought to improving the quality of the folks we admit into it in the first place. Look for more from them in the spring with the release of their Teacher Prep Review. (Indeed, if you can’t wait, this year’s policy yearbook offers a few “sneak peeks.”)

SOURCE: National Council on Teacher Quality, 2012 State Teacher Policy Yearbook (Washington, D.C.: National Council on Teacher Quality, January 2013).

This report by Stanford’s Martin Carnoy and the Economic Policy Institute’s Richard Rothstein offers a catchy press-release headline: The U.S. Fares Better on International Assessments than Previously Thought. But that isn't actually true. Analyzing PISA data, Carnoy and Rothstein argue that the U.S. educates its disadvantaged students about as well as similar nations—and, for that, America should be praised. But the problems with the study are myriad. First, the authors use a “very approximate” index—the number of books in a student’s home—to determine social class. Others have explained the methodological flaws with this approach. Second, the authors engage in some dangerous statistical gymnastics to prove their point: Based on the assumption that students of low “social class” bring down average U.S. scores, Carnoy and Rothstein re-estimate PISA attainment (by using the books-in-the-home index) to norm the proportion of students in each class. They find that, if the U.S. had the same proportion of students in lower social classes as other nations, then it would rank fourth in reading (instead of fourteenth) and tenth in math (instead of twenty-fifth). The conclusions of this report only affirm the very significant education problem that it’s trying to downplay: We have a greater proportion—and a significantly greater number—of low-scoring and low-income students than other OECD countries. Carnoy’s and Rothstein’s flawed analysis and misleading primary conclusion is at best a diversionary ploy.

SOURCE:Martin Carnoy and Richard Rothstein, What do International Tests Really Show About U.S. Student Performance? (Washington, D.C.: Economic Policy Institute, January 15, 2013).

The teacher-evaluation debate follows a well-worn path: Traditional evaluation systems (in which upwards of 99 percent of educators are deemed “effective”) are meaningless, argue reformers. New models that rely heavily on value-added test-score data are unreliable and unfair, counter others. This new NBER working paper from Northwestern’s C. Kirabo Jackson provides the debate new turf on which to tread: Based on data from the 1988 National Educational Longitudinal Study, Jackson channels Paul Tough to argue that students’ “non-cognitive” abilities (adaptability, self-restraint, motivation) help explain their success. Noting this, teachers should be evaluated on them; yet they are rarely considered by current metrics. The report has two parts. First, Jackson shows that the “non-cognitive factor” (which he proxies with variables like absenteeism, suspensions, and grades) is predictive of college enrollment and lifetime earnings—more so, in fact, than cognitive ability. Jackson then evaluates whether teachers can affect this “non-cognitive” factor. Using 2005-10 North Carolina data, he finds that teachers’ impact on student test scores is only weakly associated with their impact on improving youngsters’ non-cognitive abilities. In other words, evaluations that rely exclusively on test scores fail to capture the full breadth of teachers’ contributions to student outcomes. Jackson concludes: Other variables that assess ability to improve students’ non-cognitive skills, variables such as student suspensions and absences, should also be used in teacher assessments. And the debate marches on.

SOURCE: C. Kirabo Jackson, "Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina" (Chicago, IL: Northwestern University, January 2013).

This book awakens an established but sparingly practiced and often unknown initiative in K–12 education: teacher autonomy. Authors profile eleven schools (seven of them charters and three of them in Minnesota, a pioneer in the "teacher-led-schools" initiative) that embrace teacher autonomy to differing degrees and study the policies and practices by which they operate. Ten criteria are used to judge the autonomy level of the teachers including their agency over: staff hiring and firing decisions, budget allocations, curriculum design, and school-wide discipline policies. Written for teachers, the book—a worthwhile primer on what teacher autonomy is and what its many forms look like—offers an illustrative blueprint for one manner in which teachers may be empowered, rather than alienated or demonized, by the reform movement. Still, the book unconvincingly handles one key component of a worthy teacher-autonomy policy: Before you give teachers the keys to the castle, make sure you have royalty in the profession.

SOURCE: Kim Farris-Berg and Edward Dirswager with Amy Junge, Trusting Teachers with School Success (Lanham, MD: Rowman & Littlefield Education, 2012).