Some parents in Michigan were none too pleased by the conclusions reached in Education Week's Quality Counts 2007: From Cradle to Career, especially by the report's "Chance-for-Success Index," which measures how likely are students to succeed in school by calculating the socioeconomic standing of adults. According to an article in the Detroit Free Press, some parents said, "the study... seems too defeatist. They said success for students cannot be measured so easily by looking at the accomplishments of parents." Demography is not destiny (see here).
They're right, of course. One wonders how the publishers, editors, and researchers who contributed to From Cradle to Career could be so wrong.
It's a radical about face for Quality Counts, which Education Week launched in 1997 following a call from the nation's governors for an "external, independent, nongovernmental effort" to measure if individual states were actually improving their schools and their students' academic achievement.
For the past decade, Quality Counts has examined the nation's focus on standards-based reform. Among other things it graded states on their standards and assessments (whether standards were clear and grounded in content, whether tests were aligned to those standards, etc.); on accountability (whether the state provided report cards for its schools, whether it rewarded good schools and sanctioned bad ones, whether student promotion hinged on exit exams, etc.); and on teacher quality. It looked at achievement trends in reading and math, as well as trends in state graduation rates, and then assigned grades based on an average of those indicators.
To be sure, Quality Counts hasn't been perfect (as our own Checker Finn made clear following the initial report). Still, it covered a lot of ground. Recent editions looked at more than 100 policy indicators and produced detailed, individual, and useful state reports. A legislator, for example, could use it to identify those policy areas in which her state was deficient, and then push to have those areas strengthened. And for those not enamored with the "conventional wisdom" as captured by Quality Counts, the report spawned imitators (including our own) that measured policies and indicators they felt important.
But with Quality Counts 2007, Education Week has abandoned standards-based reform and upended the series focus in the worst way. The editors have concluded, suddenly, that "children's chances for success don't just rest on what happens from kindergarten through high school," but that success is determined by who they are, where they live, what languages their parents do or don't speak, and how much money their parents make. Thus, readers are presented the "Chance-for-Success Index," by far the report's most publicized portion.
Here's how the editors describe it: "The Chance-for-Success Index...provides a state-focused perspective on the importance of education throughout a person's lifetime." They write that, "Since all states start at zero, the index can capture the cumulative effect of education experienced by residents of a state from birth to adulthood and pinpoint the chances of success at each stage."
Here's how I would describe it: "The Chance-for-Success Index divides states into those that are wealthy and white and those that are poor and minority. Students from wealthy, white states are determined to have a good chance to succeed; students from poor and minority states are not."
Let's start with the index's childhood indicators, which rightly acknowledge that education begins before students enter kindergarten classrooms. But most of the indicators for childhood success (such as "percent of children whose parents are fluent English-speakers") focus not on the quality of pre-K programs but on the economic and educational attainment of 3-year-olds' parents.
Children of richer or better-educated parents will probably arrive in kindergarten with more skills than their less-fortunate peers. But we already know that--the important question is how well a state's policies encourage the spread of early childhood advantages to more youngsters.
Defeatism permeates the Chance-for-Success Index because it evaluates demographic indicators that are beyond the purview of schools or policymakers. It should be named the Chance-for-Failure Index. According to their formulas, Quality Counts editors implicitly tell us that students born into poor or non-English speaking households are more likely to fail.
And maybe they are. But that has nothing to do with individual states or their educational policies. Shouldn't the Index have at least tried to evaluate states on things they can control?
At the other end of the spectrum, the adulthood indicators are unfortunate not only for their defeatism but also for their ridiculousness. They, too, are based on economic data--percent of adults with an income above the national median, percent of adults with a 2- or 4-year postsecondary degree, etc. The Quality Counts editors apparently hoped these economic statistics would yield information about education's "effects" within the states.
But they ignored the obvious: that many adults were not educated in the state where they now live; well over 30 percent of college-educated Americans received their K-12 educations in more than one state (see here). A 47-year-old New Jersey parent with a Ph.D. may have gone to high school in Georgia, received a B.A. in California, and done graduate work in Wisconsin. Her educational attainment reflects absolutely nothing about the Garden State--she may not even work there!
In the end, the results are typical. Virginia, Connecticut, Minnesota, New Jersey, Maryland, and Massachusetts are at the top; Alabama, Mississippi, Tennessee, Texas, Arizona, Louisiana, and New Mexico are at the bottom.
So what are state policymakers whose citizens are determined to have low chances of success supposed to learn? Is an Arizona legislator to conclude that his state's thriving charter schools are a waste, and that he should throw in the towel because children of non-English speakers and poor immigrants are largely doomed by birth?
The Chance-for-Success Index is defeatist, flawed, and largely ridiculous. It tells us nothing about the states it purportedly evaluates (Florida and California are ranked below Alaska--are we really supposed to buy that?). And states aren't the right unit of measure for this demographic information anyway; are we supposed to believe that folks who live in inner-city Baltimore or Newark are likelier to succeed than those in the Nashville or Houston suburbs?
The Chance-for Success Index is a mess. Evaluating states not by the education policies they implement or the student learning gains they achieve, but by coarse, demographic data only gives comfort to an education establishment desperate to blame "poverty" for its failings. It's the antithesis of standards-based reform. Let's hope the focus of Quality Counts 2007 is merely part of an off-year, not the start of a repudiation of today's most promising reform strategy.