Mapping 2005 State Proficiency Standards Onto the NAEP Scales
Institute of Education Sciences, Research and Development ReportJune 2007
Institute of Education Sciences, Research and Development ReportJune 2007
Institute of Education Sciences, Research and Development Report
June 2007
No Child Left Behind empowered states to set their own "proficiency" standards in reading and math. To keep them honest, NCLB also requires states to participate in the National Assessment of Educational Progress. Ideally, a student who scores at the proficient level on his state exam would perform equally well on NAEP. But the state exams are far from ideal. That's the working conclusion of this paper, which "mapped" student performance on fourth- and eighth-grade state exams in reading and math onto performance on the 2005 NAEP, using a methodology developed by H. I. Braun and J. Qian. (Nota bene: Not all states could be measured. The number of states mapped ranged from 32-36.) If you want the technical ins and outs, you'll need to read the report, but here's the bottom line. For fourth-grade reading, not a single state's proficiency standards rose to the level of NAEP's. And 22 states didn't even come up to NAEP's "basic" level. Results were slightly better for eighth- grade reading. The number of states below the basic level was nine. Still, none reached the proficient level. In math, the story is somewhat better. At the fourth-grade level, two states' proficiency standards matched the NAEP standards (kudos to Massachusetts and Wyoming), while all but six states ranked at or above the NAEP basic level. For eighth grade, three states make the proficiency mark (Missouri, South Carolina, and Massachusetts), but eight fail to reach the basic level. It's a great report, so long as one bears in mind its two inherent shortcomings. First, the content of NAEP is not well aligned to every state's academic standards, so one cannot know to what degree this factor is driving the discrepancies. And second, it measures only a single point in time (2005). We wonder if states, over time, have been progressively easing their tests to elevate the percentage of students making the grade. Stay tuned for our answer this fall.
Editorial Projects in Education
June 2007
The second iteration of the now-annual Diplomas Count report includes updated graduation data and an in-depth discussion of how states are tackling the tricky issue of "work and college readiness." Nationwide, the average graduation rate, which EPE calculates based on its own "Cumulative Promotion Index," has hardly budged since last year; it still hovers around 70 percent. The numbers for black and Hispanic males also remain depressingly low: 46 percent and 52 percent, respectively. This year's report also breaks down the nation's workforce into five income "zones" and analyzes each zone's average level of education. This produces sobering findings such as: "while 15.7 percent of the labor force in the District of Columbia occupies Job Zone 5--in which more than nine in 10 workers have at least some college and more than three quarters have a bachelor's degree--most of those jobs are inaccessible to Washington's public school students, more than four in 10 of whom fail to earn a high school diploma within four years." The report also features some solid commentary and unique data on how states are preparing their students for college and work. EPE concludes that only 11 states have adequately defined "college readiness" and 21 "work readiness," based on measures such as course requirements and curricular standards. (Compare this with a recent report from Achieve, which found that 13 states "require students to complete a college- and work-ready curriculum.") On the bright side, more than a dozen states claim to be developing such definitions. Finally, like last year's report, Diplomas Count 2007 features an interactive map that displays graduation rates in a colorful, easily digestible format, right down to the level of individual schools. Find all this and more here.
Research for Action
June 2007
Research for Action completes the trifecta with the publication of its third study on teacher quality in Philadelphia (see Gadfly's takes on the 2003 and 2005 installments here and here). Offering updates on both NCLB- and Paul Vallas-induced reforms, the latest report discusses how the district has upgraded teacher credentials, improved recruitment and retention, and more equitably distributed teachers across schools. The district has strengthened its classroom workforce, for example, by rewarding teachers who head to 24 "incentive schools" and relying on alternative certification programs. Four years of this campaign have yielded promising results. Yet teacher quality discrepancies between low- and high-poverty schools remain. And teachers continue to leave: only 30 percent of current teachers in the City of Brotherly Love have over six years' experience. Research for Action's fairly obvious recommendations include more incentives for teachers in high-poverty schools, more professional development, and recruitment of more minority and specialized teachers. Have a look here.
After an arduous process, it finally happened: Washington, D.C., Mayor Adrian Fenty took control of the city's schools. And his first act was ousting Superintendent Clifford Janey and appointing Michelle Rhee, founder of the New Teacher Project. In a symbolic act of renaming, Rhee will become the District's first "Schools Chancellor." Her lack of conventional experience has already come under fire from the usual suspects, though; Lee Glazer, founder of the demagogic Save Our Schools coalition, called Rhee a "nobody." (That just goes to show how out of touch Glazer is.) But in her previous job, Rhee worked extensively with large, urban districts (and their hulking bureaucracies) while still remaining outside of them. Such experience may actually help her come up with new and innovative solutions D.C. Public Schools desperately need. Of course, she faces a Herculean task that might very well be impossible--she is but one person against a backward system. For now, though, a tip of the fedora to Fenty for making such an inspired and politically-gutsy choice.
"Fenty to Oust Janey Today," by David Nakamura, Washington Post, June 12, 2007
In left-wing enclaves such as my current home of Takoma Park, Maryland, ridiculing the illogic of the Bush Administration (on Iraq, on global warming, etc.) is something of an official sport. As the only former member of the Bush Administration in town--if there are others, they stay closeted--I tend to stay in the bleachers. But last week, the President's domestic policy standard-bearer, Education Secretary Margaret Spellings, performed such an amazing feat of mental gymnastics around the issue of national standards that I can't help but step onto the field just this once.
My beef isn't that she dismissed the idea of national standards as a solution to the wide variability of state standards, a problem shown once again last week by a new government study. (See our review below.) While Gadfly readers know that we favor national standards, you also know that we share Spellings's concerns about its risks. She's right that the process of creating national standards could become "an exercise in lowest-common-denominator politics." That's why we need to think carefully about the best way to create national standards--a path that may not, probably should not, involve the federal government. (See our ideas here.)
No, my real concern is that she stubbornly refuses to acknowledge the downward pressure that NCLB is putting on state standards.
Consider these assertions from her Washington Post op-ed ("A National Test We Don't Need"). "Neighborhood schools deserve neighborhood leadership, not dictates from bureaucrats thousands of miles away," she writes. And: "Our goal is a public education system that is transparent and responsive to the needs of parents and children--not to the whims of Washington."
The dictates of bureaucrats? The whims of Washington? Forget national standards; these are excellent descriptors for much of No Child Left Behind today. Demanding universal proficiency by 2014. Expecting students with disabilities and English language learners to meet the same standards as their peers. What are these if not "dictates" and "whims"?
Is Spellings the last person in America who doesn't see a link between such mandates and lackluster state standards? Sure, those standards were low, many of them anyway, before NCLB burst onto the scene, but there's nothing in the act to inspire states to aim higher. On the contrary. If you really have to get 100 percent of students to "proficiency" by 2014 (including those who can't read or speak English and children with significant learning disabilities), you have little choice but to set the bar low. Ask Missouri, which set lofty, aspirational proficiency goals in the pre-NCLB era, then lowered its bar lest every school in the Show-Me state be listed as "needing improvement."
Even Education Trust, the hardest of NCLB hard-liners, admits that there's a problem here. It has offered a smart solution (see here), allowing states that set rigorous college-prep standards to move back their timelines and to aim for merely 80 percent proficiency. (Ninety-five percent would have to achieve a "new basic" level.) What an opening this gave the Secretary! Ed Trust gave her political cover to drop the "100% by 2014" rhetoric. And yet she passed.
States could use "anything and everything that could be considered raising standards" to justify "recalibrating the goalposts," she told Education Daily about the Ed Trust proposal. "I worry that that's a way to go backwards."
So let's get this straight: When it comes to defining proficiency--how much students are expected to know and be able to do--it's acceptable to move the goalposts. (No federal official gave Missouri any grief.) But when it comes to getting all students to proficiency, any delay is "going backwards."
And, Madame Secretary, you still don't see how your No Child Left Behind depresses state standards?
Alas, such a position isn't just illogical. It's also depressing.
Now back to my bleacher.
Last Sunday's New York Times Magazine was devoted to the income gap, the monetary expanse that separates the have-a-lots from the have-nots. One article in particular caught our eye: "The Poverty Platform." It was a detailed examination of John Edwards's current presidential campaign and its focus on eliminating poverty in America. Whether or not one agrees with Edwards's views, readers should be struck by his failure to speak substantially about improving education. Loads of research (and many of the other Times Magazine articles, such as this one) tell us that one of the surest ways to close the income gap is by improving education. Yet Edwards says next to nothing on this topic, preferring, it seems, to reinforce to his audiences just how dire and intractable their situation is. In fact, there's been much talk about inequality from all the Democratic candidates, but education has taken mostly a back seat (although Senator Hilary Clinton did roll out a national pre-K plan). Fixing the unequal economic system starts with fixing the unequal education one. Is any candidate willing to say that?
"The Poverty Platform," by Matt Bai, New York Times Magazine, June 10, 2007
Districts have long resisted plans to pay teachers based on their performance. So it's little surprise that Harvard economist Roland G. Fryer encountered flak when he proposed to pay students based on theirs. Fryer is pushing a student-pay pilot program for New York City that would give kids cash for test scores (at least $5 just for taking the city's mandatory exams, and as much as $25 in fourth grade and $50 in seventh grade for high-achievers). Mayor Bloomberg and Schools Chancellor Klein are receptive to the idea but others aren't convinced. Maggie Siene, principal of Public School 150 in Tribeca, said, "It makes me really nervous. I suspect paying kids for achievement in any way tends not to work." Tom Loveless of the Brookings Institution disagreed. Poor kids, he said, lack the motivating forces that push their better-off peers to succeed in school and thus "have a very hard time understanding that what they do today pays off decades from now." An experiment seems to be in order--with a really good evaluation attached. But may we suggest giving students Borders or Barnes & Noble gift cards for books rather than cold, hard cash? If we're going to be paternalistic, we might as well go all the way.
"A Plan to Pay for Top Scores on Some Tests Gains Ground," by Julie Bosman, New York Times, June 9, 2007
"Money on the Mind," by Alexander Nazaryan, The New Republic Online, June 11, 2007
"Earned autonomy" is an education-reform idea whose time has come--and should come to federal policy. Increasingly, superintendents (in places like Chicago, Las Vegas, New York City, etc.) are allowing schools with a track record of improving student achievement to gain more freedom from central office control. Everybody knows that effective principals need--and make shrewd use of--authority to run their schools as they think best and that kids end up benefiting (see here). Everyone also knows that not every principal is up to that challenge. So smart school systems look for evidence of successful leadership and then enhance it.
The charter-school domain is trending in a similar direction. While charters have always been about "accountability in return for autonomy," more and more of their sponsors (including Fordham's Ohio operation) understand that autonomy is something to be granted with care. Once upon a time, some of us thought we should plant as many charter seeds as possible as quickly as possible and let a thousand school flowers bloom, the theory being that we could always close them down. Well, it turns out that closing schools is bloody hard and, as we've witnessed, many charter schools founder (or worse) because their leaders weren't prepared to make effective use of the autonomy they had been given (see here, for instance).
As a consequence, conscientious charter sponsors are more apt today to screen applicants carefully, just as venture capital firms appraise prospective business start-ups. We also reward charter schools for good performance by granting longer charters, hassling them less, and encouraging their replication. (The leash is shorter for low-performing schools.)
The appeal of this idea is obvious: It encourages desirable behavior (especially stronger pupil achievement), it recognizes that some entities are better at handling autonomy than others, it keeps more support systems in place for those that aren't ready, and it reduces risk.
Unfortunately, "earned autonomy" has very little traction at the federal level, due mainly to Washington's proclivity to treat all states (and districts) exactly alike. The architects of No Child Left Behind looked upon states with distrust, assuming that, without strong oversight, they would fail to put in place serious accountability systems--or least systems focused on the needs of poor and minority students. Thus the law's many prescriptions and mandates. By so doing, they doubtless catalyzed activity in some laggard states but they also made life more difficult for those that already had well-functioning accountability (and choice) arrangements of their own. (Dual accountability systems--state and federal--continue today in confusing and contradictory ways in far too many places.)
Why not ease the regulatory burden on states that are clearly succeeding? Probably too simple and obvious an idea for federal policy to embrace. But we persevere. Here are some ways that "earned autonomy"might be imported into NCLB:
1. Grant greater flexibility to states that sign up for rigorous national standards and tests. Washington need not (indeed should not) create these standards and tests, but could encourage their use. Volunteer states might earn freedom from federal spending restrictions. Today, Uncle Sam's dollars pour into myriad silos, categorical programs that may or may not meet the needs of individual communities. NCLB's "transferability" provision began to address this problem by allowing states or districts to shift funds from one silo to another, or into Title I. But it set a cap at 50 percent. President Bush and Congressman McKeon have recommended expanding transferability to 100 percent, allowing states or districts to send all their ESEA dollars into the Title I program and then ignore all rules and regulations for the other programs. This would cut red tape while also driving more federal dollars toward needy students.
2. Let successful states devise their own "school improvement" timelines. Sure, it's important to take action when low-performing schools repeatedly fail to improve. What's not clear is whether NCLB's rigid sequence of prescribed annual interventions (including choice and tutoring, corrective action, restructuring) is any better than those that states might devise. In our view, such actions are likelier to succeed if decided as close as possible to the problem and on timetables that make sense to those who must implement them. Moreover, ample sunlight shining down on school/district/state performance vis-à-vis clearly specified national standards will give state and local officials (and voters, taxpayers, parents, etc.) good information by which to repair their own schools.
So lawmakers should let states where achievement is rising out from under the prescribed intervention sequence. And free them to distinguish between abysmal and mediocre schools--and treat them differently.
3. Allow schools that make AYP to ignore HQT. The impulse behind NCLB's "highly qualified teachers" provision is understandable. Teacher quality matters and most states have woeful standards for incoming teachers. Still, the HQT mandate has created undesirable consequences and just about everyone knows that its obsession with paper credentials makes a poor proxy for classroom effectiveness. So reward schools for getting great results by allowing them flexibility around staffing. To continue making AYP, schools will have to make good decisions around teachers--and should be free to do this without federal red tape. (This is especially important for high-performing charter schools, which are supposed to be freed from such regulation but in fact are tangled in NCLB's subject matter requirements like everyone else.)
These ideas would begin to align federal policy with a maxim from Management 101: Treat all of your units fairly, but don't treat them all the same, because not everyone needs the same level of oversight or autonomy. If Uncle Sam wants to keep playing a leadership role in education, he should learn to treat different states (and districts and schools) differently. Some may benefit from heavy-handed federal oversight. But those that produce great results should earn their way to greater autonomy. It's a pragmatic notion that both liberals and conservatives should love.
This essay is adapted from Finn's June 7 Congressional Testimony in front of the House Early Childhood, Elementary and Secondary Education Subcommittee.
United Teachers Los Angeles has decided that instead of fighting charter schools they'd rather chase their teachers. "We have come to the realization that we need to look at organizing teachers at charter schools," said UTLA President A.J. Duffy. Steve Barr, whom we admire for his gutsy leadership of Green Dot Public Schools, and whose teachers already belong to the California Teachers Association, sees this as a positive development. "As relationships start to come together between the unions and unionized charters, the people that will be left out of the equation are non-unionized charters," he said. "The charter movement is more stubborn about these kinds of relationships than the unions are." Stubborn for good and sufficient reason, we think. A big part of what separates charters from district schools is their freedom from red tape, including the encumbrances of a zillion-clause collective bargaining contract. Charter school principals can, among other things, hire and fire teachers as they see fit. It's hard to picture the unions agreeing to that. If UTLA wants to start its own charter schools, like New York City's United Federation of Teachers, fine. But it should leave the others alone.
"Union Targets Charter Schools," by Naush Boghossian, Los Angeles Daily News, June 10, 2007
Editorial Projects in Education
June 2007
The second iteration of the now-annual Diplomas Count report includes updated graduation data and an in-depth discussion of how states are tackling the tricky issue of "work and college readiness." Nationwide, the average graduation rate, which EPE calculates based on its own "Cumulative Promotion Index," has hardly budged since last year; it still hovers around 70 percent. The numbers for black and Hispanic males also remain depressingly low: 46 percent and 52 percent, respectively. This year's report also breaks down the nation's workforce into five income "zones" and analyzes each zone's average level of education. This produces sobering findings such as: "while 15.7 percent of the labor force in the District of Columbia occupies Job Zone 5--in which more than nine in 10 workers have at least some college and more than three quarters have a bachelor's degree--most of those jobs are inaccessible to Washington's public school students, more than four in 10 of whom fail to earn a high school diploma within four years." The report also features some solid commentary and unique data on how states are preparing their students for college and work. EPE concludes that only 11 states have adequately defined "college readiness" and 21 "work readiness," based on measures such as course requirements and curricular standards. (Compare this with a recent report from Achieve, which found that 13 states "require students to complete a college- and work-ready curriculum.") On the bright side, more than a dozen states claim to be developing such definitions. Finally, like last year's report, Diplomas Count 2007 features an interactive map that displays graduation rates in a colorful, easily digestible format, right down to the level of individual schools. Find all this and more here.
Institute of Education Sciences, Research and Development Report
June 2007
No Child Left Behind empowered states to set their own "proficiency" standards in reading and math. To keep them honest, NCLB also requires states to participate in the National Assessment of Educational Progress. Ideally, a student who scores at the proficient level on his state exam would perform equally well on NAEP. But the state exams are far from ideal. That's the working conclusion of this paper, which "mapped" student performance on fourth- and eighth-grade state exams in reading and math onto performance on the 2005 NAEP, using a methodology developed by H. I. Braun and J. Qian. (Nota bene: Not all states could be measured. The number of states mapped ranged from 32-36.) If you want the technical ins and outs, you'll need to read the report, but here's the bottom line. For fourth-grade reading, not a single state's proficiency standards rose to the level of NAEP's. And 22 states didn't even come up to NAEP's "basic" level. Results were slightly better for eighth- grade reading. The number of states below the basic level was nine. Still, none reached the proficient level. In math, the story is somewhat better. At the fourth-grade level, two states' proficiency standards matched the NAEP standards (kudos to Massachusetts and Wyoming), while all but six states ranked at or above the NAEP basic level. For eighth grade, three states make the proficiency mark (Missouri, South Carolina, and Massachusetts), but eight fail to reach the basic level. It's a great report, so long as one bears in mind its two inherent shortcomings. First, the content of NAEP is not well aligned to every state's academic standards, so one cannot know to what degree this factor is driving the discrepancies. And second, it measures only a single point in time (2005). We wonder if states, over time, have been progressively easing their tests to elevate the percentage of students making the grade. Stay tuned for our answer this fall.
Research for Action
June 2007
Research for Action completes the trifecta with the publication of its third study on teacher quality in Philadelphia (see Gadfly's takes on the 2003 and 2005 installments here and here). Offering updates on both NCLB- and Paul Vallas-induced reforms, the latest report discusses how the district has upgraded teacher credentials, improved recruitment and retention, and more equitably distributed teachers across schools. The district has strengthened its classroom workforce, for example, by rewarding teachers who head to 24 "incentive schools" and relying on alternative certification programs. Four years of this campaign have yielded promising results. Yet teacher quality discrepancies between low- and high-poverty schools remain. And teachers continue to leave: only 30 percent of current teachers in the City of Brotherly Love have over six years' experience. Research for Action's fairly obvious recommendations include more incentives for teachers in high-poverty schools, more professional development, and recruitment of more minority and specialized teachers. Have a look here.