Smart Testing: Let's Get It Right: How assessment-savvy have states become since NCLB?
American Federation of TeachersJuly 2006
American Federation of TeachersJuly 2006
American Federation of Teachers
July 2006
This report aims to examine the clarity and specificity of the states' academic content standards as well as their alignment with state tests, and also judges the "transparency" of the entire standards-based reform enterprise--the very heart of NCLB. Unfortunately, the findings suffer from grade inflation. In its review of state content standards, for example, a whopping 18 states receive perfect scores for their reading, math, and science standards. How can that be, when Fordham's standards reviewers tend to give honor grades in each subject to but a handful of states? The AFT is apparently satisfied when standards for the elementary and middle levels are detailed, explicit, "rooted in the content of the subject area," and articulated grade-by-grade. The organization doesn't, however, evaluate the soundness or worth of the content itself. Thus Michigan earns perfect scores by AFT standards, but receives two Ds and a C (for English, science, and mathematics, respectively) from Fordham. The Wolverine State's English standards lack, among other things, a list of key authors, works, literary periods, and literary traditions. The state's science and math documents have similar content holes--hardly worthy of perfect scores. And more problems exist in the report's review of alignment between standards and state tests, though this time it's not the AFT's fault. Few states make their tests and related technical information public (how's that for transparency?), so researchers are forced to base their judgments on test specifications instead. Even this generous approach can't mask the poor jobs states do when aligning standards to tests. This report finds that half of the states' standardized tests don't appear to align with content standards. The AFT is right to complain that "when there's a mismatch between the content that's expected, the content that's taught, and the content that's assessed...it's no wonder that folks in schools throw up their hands in frustration." If you want to read more compelling evidence that rigorous national standards and tests are sorely needed, give the report a look (here); otherwise wait for a compilation of Fordham's standards reviews, due out next month.
Lawrence Mishel and Joydeep Roy
Economic Policy Institute
2006
The graduation-rate debate between Jay Greene and the Manhattan Institute, and Lawrence Mishel and the Economic Policy Institute (which began here and here; subscription required), continues. The latest salvo comes with release of this book, which begins by giving background on the graduation rate argument and moves on to elucidate Mishel's position. But where does the truth lie? The answer, of course, depends on which data sets and methodologies are most reliable. Here the nod goes to Greene. His method essentially compares the number of ninth graders in a particular year to the number of diplomas awarded four years later. Mishel, on the other hand, relies largely on bulky statistical samples--from both the Census Bureau (which simply asks individuals about their level of educational attainment) and the Department of Education's National Education Longitudinal Study (which tracks sets of about 10,000 to 12,000 students per year, interviews them about their educational attainment, and checks their school transcripts to validate responses). This book attempts to explain why Michel's sampling is preferable to Greene's method for determining graduation rates. Greene, Mishel writes, ignores the "9th grade bulge" (students who repeat ninth grade form a bulge, and although they may receive a diploma, it won't be within four years); the unreliability of diploma counts; and how to account for unique populations such as immigrants, prisoners, military personnel, etc. But these idiosyncratic circumstances fail to dethrone Greene's crucial fact that some 30 percent of students--millions each year--who enter high school don't receive diplomas four years later. Call it what you want, but it's a definite problem. To dive into these figures yourself, check out Mishel's book online here.
Anastasia de Waal
Civitas
July 2006
Anastasia de Waal, who is affiliated with the London think tank Civitas, a non-partisan group that says its chief interest is "civic cohesion," has written this stinging, 148-page exposé of how the British government's school inspectorate is neutering many of England's "independent" schools. Though much has been written in praise of the school-inspector system (New York's Joel Klein has actually imported a version of it to his sprawling domain), de Waal insists that the Blair government is using it more like a police force to turn independent schools into facsimiles of government-run schools--right down to their curricula, teaching methods, and record-keeping. And although these independent schools must comply with onerous regulations, they've still seen what little government financial aid they and their students previously received dwindle. This will resonate with U.S. private (and charter) schools fearful of heavy-handed government regulation, with or without government aid. The author also points out that the Blair government's "third way" approach leads it to welcome private funding and initiative throughout the schooling system, but that it has little or no tolerance for true educational diversity. You may want to have a look. You can get more information here.
Principal Linda Marlar of Mountain Sky Junior High School in Phoenix thinks her teachers' language skills could use some beefing up. But at the training session she's planned for them this coming August, the staff will learn less about subject-verb agreement than about when it is appropriate--and when it is decidedly not appropriate--to use words such as "dawg," and phrases such as "don't clown me, playa, or I'll bust you in the grill." Slang lessons, Marlar thinks, will give her teachers another way to communicate with, and comprehend, their students. The Arizona Republic reports, for example, that "Marlar cautions teachers not to overreact if kids use the word ‘pimp' as a verb, as in ‘pimp my backpack,' because they're not referring to prostitution but accessorizing." Gadfly is unimpressed. This has been done before. Like most things hip, the practice of integrating slang into the classroom started in L.A., and Marlar is merely months late in copying ("jocking the style") of chic, West Coast school leaders. Even the bookworms who enroll in SAT prep courses have already heard of this! Sorry, Linda, but the fad has passed. And you're not thirteen anymore.
"Students' ‘tight' terms ‘aiight' with principal," by Karina Bland, Arizona Republic, July 23, 2006
Clive Crook, having just returned from the Aspen Ideas Festival (can the event be as glorious as its name implies?), writes in National Journal that the nation's best and brightest thinkers managed to agree on two things: "(a) better education is the answer to all our problems, and (b) improving education is extremely difficult to do (see how hard we tried?)." Crook disagrees. He notes that perhaps the most successful way yet discovered to improve the quality of schools--instituting competition among them--has been sedulously avoided by many people. Even in the United States, where competition is king in most every aspect of society, experimentation with school competition has, in Crook's view, been much too timid. Of course he's right, to a point. But just as the Aspen attendees held aloft high-quality education as a panacea for the world's social ills (it is not), Crook genuflects at competition's altar, wrongly assuming that market forces, positive and worthy as they've been in many realms, can be applied with similar success in every other sector. Competition is part of a complicated solution to developing high-quality education, (Sorry, Clive, but there's nothing "simple" about reforming education.) and subsequently addressing larger, societal issues. Oh, and we're waiting for our invitation to next year's Aspen gathering.
"The Lure of Education," by Clive Crook, National Journal, July 18, 2006
If you thought injecting political agendas into English and history classes was bad, we've got a doozy for you. In the latest edition of City Journal, Sol Stern writes about a nasty trend whereby progressivist professors (some of them former bomb-makers) aim to hijack not only the "softer" subjects, but hard ones such as math and science, too. You may not be surprised to learn that the ringleaders of this movement to teach "social justice" in even subjects most divorced from the social sphere reside in our nation's ed schools (see here). How bad is it? The authors of one "progressive" science text, for example, write that "the marriages between capitalism and education and capitalism and science have created a foundation for science education that emphasizes corporate values at the expense of social justice and human dignity." Those fat cat chemists in their designer lab coats with their gilded beakers! Research assistants of the world, unite! Ugh.
"The Ed Schools' Latest--and Worst--Humbug," by Sol Stern, City Journal, Summer 2006
"When activism masquerades as education," by Sol Stern, New York Daily News, July 21, 2006
Charles Murray has a beef with the No Child Left Behind Act. He's angry that his children's public schools in bucolic Frederick County, Maryland, have "turned themselves inside out to try to produce the right test results, with dismaying effects on the content of classroom instruction and devastating effects on teacher morale." So he did what any distinguished conservative scholar would do--he took to the op-ed pages of the Wall Street Journal (read his piece, "Acid Tests," here).
Marshalling bell curves and a stream of statistics, he sets out to prove NCLB's accountability metrics are "meaningless" and "deceptive."
Murray certainly knows numbers. He shows how across-the-board gains by all students can make it look like the achievement gap is closing while group differences remain the same. Here it is in a nutshell: Let's say that in your state, students must get 70 percent of the questions on the reading test correct to be considered "proficient." Last year, all of your school's white students scored in the 70s and 80s and therefore passed; all of your school's black students scored in the 50s and 60s and therefore failed. Hence, the "achievement gap" in terms of the pass rate was 100 percent. This year there was improvement across the board. White students now scored in the 80s and 90s, and black students in the 60s and 70s. If half of those black students scored in the 70s, let's say, then 50 percent of them passed, and your achievement gap was cut in half. Yet the average differences between white and black students remain.
So is this progress or not? If your goal is to arrive at a utopian future wherein all 18-year-olds possess the exact same knowledge and skills, group differences disappear, and the bell curve is flattened, the answer is no. But if you believe that "proficiency" has some meaning, that there is a certain level of literacy and numeracy required for meaningful participation in our economy and democracy, and that we are making gains in getting more students of all racial groups to this proficient level, then the answer is undeniably yes.
How does Murray handle the question? He claims that "conceptually, ‘proficiency' has no objective meaning that lends itself to a cutoff"; hence, the gains are illusory. If he's trying to say that most states have set the bar for "proficiency" so low as to be meaningless, well, no argument there. (And he's right that, by lowering the bar further, states can make the achievement gap evaporate. After all, in many states, most white students are already proficient. Any downgrading of standards will serve to redefine mostly Black and Hispanic students as "proficient," artificially shrinking the gap.)
But that doesn't mean that "proficient" is an inherently meaningless designation. True, someone has to make a judgment call whenever standards and "cut-scores" are involved. If the Department of Motor Vehicles can decide that you must score at least a 75 percent on your driver's test in order to get a license, surely thoughtful experts can determine the level of knowledge and skills required to tackle the demands of adulthood.
What does Murray think about the National Assessment Governing Board's (NAGB's) lofty definition of "proficiency"? For example, for high school seniors: "When reading text appropriate to twelfth grade, they should be able to extend the ideas of the text by making inferences, drawing conclusions, and making connections to their own personal experiences and other readings. Connections between inferences and the text should be clear, even when implicit. These students should be able to analyze the author's use of literary devices." If standards-setters do their job well, as NAGB has, the line between proficient and not proficient has a lot of meaning--in terms of life opportunities for individual students.
Murray's skepticism on this point leads him to make the astounding assertion that all children "cannot...even be proficient, if ‘proficient' is defined legitimately." (So now he seems to say that it can be defined, after all.) But why not? Sure, there will always be a bell curve, but couldn't better instruction, higher expectations, and well-prepared teachers move the entire curve to the right, getting most or all students past the "proficient" line?
That's exactly what NCLB aims to do, rhetoric about closing the achievement gap aside. About this Murray is confused. He writes that "the federal government is doling out rewards and penalties to school systems across the country based on changes in pass percentages." Wrong. NCLB's "adequate yearly progress" measure is famously (or infamously--ask the AFT) not about progress. It's about getting a certain percentage of students from all subgroups over the "proficient" bar. Nor do schools have to close the gap in average performance of subgroups.
And for good reason. No one would support a policy that gave schools an incentive to hold down the performance of white students in order to show gains in closing the achievement gap. (It's bad enough that already-proficient students face benign neglect under the current system.) That version of No Child Left Behind would deserve the treatment provided in Murray's column. But when crafting op-eds you really ought to go to war against the law you have, not the imaginary law you want to bash.
Mo' money, mo' problems. Hundreds of families who benefit from Washington, D.C.'s voucher program but have enjoyed modest increases in household income are in danger of exceeding its income guidelines and having their scholarships revoked. As many as 150 students a year may be pushed out of the program--in some cases because their mothers and fathers reunited, bumping up the family's bread basket. Republican Senator Sam Brownback of Kansas wants to help those youngsters by adjusting the income eligibility for students already in the program. For his efforts, Brownback has been pilloried by opponents who say that raising eligibility will replace truly underprivileged children with middle-class kids. But Brownback's proposal still leaves the program's participants with an average income of $22,000 per year for a family of four--hardly enough for a cushy, middle-class lifestyle in the District's overheated real estate market. The truth, of course, is that many who oppose Brownback have fought against D.C. vouchers from their inception. Our suggestion: Senator Kennedy and People for the American Way should co-sign the letter to these families informing them that their vouchers have been confiscated. Have a nice day.
"To Retain Students, Higher Income Rule Sought," V. Dion Haynes, Washington Post, July 8, 2006
"‘Opportunity' is Knocked," Wall Street Journal, July 25, 2006 (subscription required)
Robert Louis Stevenson knew that those who preach the virtues of play as work are talking about an illusion. Not so says Clara Hemphill, who blasts Achievement First East New York Charter School for trading dress-up corners and play kitchens for phonics lessons and grammar practice. Citing educators who argue that "play is work," Hemphill blames NCLB for depriving the children of First East, and kindergarteners across the U.S., of their childhoods and, apparently, of a chance at learning. Hardly. Though some children in Achievement First schools, best known for New Haven's Amistad Academy, no doubt yawn through lessons, they do learn. And given a choice of longer school days or their high-poverty, high-crime neighborhoods, most AF students will take school. Says one Amistad student, my mother "grew up on the streets, and I don't want to go there." Or as Stevenson would put it, "it is but a child of air/that lingers in the garden there."
"In Kindergarten Playtime, A New Meaning for ‘Play,'" by Clara Hemphill, New York Times, July 26, 2006
Whenever ed reformers put a new idea on parade, it's expected that the unions will quickly conjure up storm clouds. So it came as no surprise that, a few days after the release of Fordham's weighted student funding (WSF) proposal, the union thunder rumbled.
Never mind that many respected R's and D's embrace the plan as a viable model for tackling one of the biggest problems facing our public school system: creating equal opportunities for all children to attain a high-quality education in schools that work for them. Never mind that even prominent conservatives like Bill Bennett and Rod Paige came out in favor of spending more on poor kids. Both the NEA and AFT dismissed the plan outright.
Providing more education dollars for poor children used to be an issue dear to the unions' hearts. Yet their criticism of WSF unwillingly revealed today's primary goal: More money, period. As NEA President Reg Weaver wrote in a letter to the New York Times, "The answer is not to carve up the pie so some schools get a bigger piece. We need to make the whole pie bigger."
The unions yearn to shift the debate from "equity" to "adequacy" and from "fairer funding" to "more funding." Indeed, some of their comments make it sound like the equity problem has already been solved. EdWize--the blog of the United Federation of Teachers (AFT's New York City affiliate)--argues that Title I's need-determined supplements and weighted state and district formulas are sufficient to address problems of inequitable funding. But the problem is the money hits a dam at the district level. From there, it flows into a complex maze of tributaries--staffing ratios, funds for buildings, or whatever formulas the districts use--and never trickles down to the students who really need it. Even Title I includes a loophole that drives funds away from poor schools by ignoring the vast differences in teacher seniority (and thus salaries) from campus to campus. In a typical city, poor students still attend schools that receive many thousands of dollars less than those across town.
WSF is also designed to empower school leaders--to give them the ability to make decisions without relying on central-office bureaucrats. In other words, it cuts through red tape and allows people closer to kids to make important decisions. Teachers should love this idea--and many of them do--but their unions seem to think otherwise.
EdWize worries that WSF will "give principals an incentive to ‘buy' cheap teachers in order to divert more funds to [their] pet programs or people." On the contrary, principals would be motivated to spend money more wisely than districts do now. Why? Because principals are held accountable under state and federal law for their schools' progress. More likely than EdWize's alarmist prediction is that principals would pay more for highly qualified teachers. And that action--while not good for the few teachers who coast through the year without really teaching--would increase overall teacher pay in hard-to-staff schools.
WSF also makes education funding more transparent. The government has managed to track the secret financial transactions of suspected terrorists all over the world, but it can't tell you how much the public school down the street spends on its students. A simplified funding system would remedy this. Perhaps union leaders aren't eager for the education system's bloat to be more readily apparent--and for the world to clearly see how much they take off the top of their members' five-figure incomes to pay their own six-figure salaries.
But the union's biggest fear is that Weighted Student Funding would pave the way for a voucher system. EdWize asks, "So is this vouchers redux--via a Trojan horse?" By that logic, they should oppose homework because it could pave the way for home schooling.
We know where the unions stand. But we don't think students, parents, or most teachers stand with them. That's why innovative ideas such as WSF command bipartisan support. And that's why if unions don't change their defeatist tune, they're going to fade into obsolescence. Storm clouds, no matter how foreboding, always dissipate in the end.
American Federation of Teachers
July 2006
This report aims to examine the clarity and specificity of the states' academic content standards as well as their alignment with state tests, and also judges the "transparency" of the entire standards-based reform enterprise--the very heart of NCLB. Unfortunately, the findings suffer from grade inflation. In its review of state content standards, for example, a whopping 18 states receive perfect scores for their reading, math, and science standards. How can that be, when Fordham's standards reviewers tend to give honor grades in each subject to but a handful of states? The AFT is apparently satisfied when standards for the elementary and middle levels are detailed, explicit, "rooted in the content of the subject area," and articulated grade-by-grade. The organization doesn't, however, evaluate the soundness or worth of the content itself. Thus Michigan earns perfect scores by AFT standards, but receives two Ds and a C (for English, science, and mathematics, respectively) from Fordham. The Wolverine State's English standards lack, among other things, a list of key authors, works, literary periods, and literary traditions. The state's science and math documents have similar content holes--hardly worthy of perfect scores. And more problems exist in the report's review of alignment between standards and state tests, though this time it's not the AFT's fault. Few states make their tests and related technical information public (how's that for transparency?), so researchers are forced to base their judgments on test specifications instead. Even this generous approach can't mask the poor jobs states do when aligning standards to tests. This report finds that half of the states' standardized tests don't appear to align with content standards. The AFT is right to complain that "when there's a mismatch between the content that's expected, the content that's taught, and the content that's assessed...it's no wonder that folks in schools throw up their hands in frustration." If you want to read more compelling evidence that rigorous national standards and tests are sorely needed, give the report a look (here); otherwise wait for a compilation of Fordham's standards reviews, due out next month.
Anastasia de Waal
Civitas
July 2006
Anastasia de Waal, who is affiliated with the London think tank Civitas, a non-partisan group that says its chief interest is "civic cohesion," has written this stinging, 148-page exposé of how the British government's school inspectorate is neutering many of England's "independent" schools. Though much has been written in praise of the school-inspector system (New York's Joel Klein has actually imported a version of it to his sprawling domain), de Waal insists that the Blair government is using it more like a police force to turn independent schools into facsimiles of government-run schools--right down to their curricula, teaching methods, and record-keeping. And although these independent schools must comply with onerous regulations, they've still seen what little government financial aid they and their students previously received dwindle. This will resonate with U.S. private (and charter) schools fearful of heavy-handed government regulation, with or without government aid. The author also points out that the Blair government's "third way" approach leads it to welcome private funding and initiative throughout the schooling system, but that it has little or no tolerance for true educational diversity. You may want to have a look. You can get more information here.
Lawrence Mishel and Joydeep Roy
Economic Policy Institute
2006
The graduation-rate debate between Jay Greene and the Manhattan Institute, and Lawrence Mishel and the Economic Policy Institute (which began here and here; subscription required), continues. The latest salvo comes with release of this book, which begins by giving background on the graduation rate argument and moves on to elucidate Mishel's position. But where does the truth lie? The answer, of course, depends on which data sets and methodologies are most reliable. Here the nod goes to Greene. His method essentially compares the number of ninth graders in a particular year to the number of diplomas awarded four years later. Mishel, on the other hand, relies largely on bulky statistical samples--from both the Census Bureau (which simply asks individuals about their level of educational attainment) and the Department of Education's National Education Longitudinal Study (which tracks sets of about 10,000 to 12,000 students per year, interviews them about their educational attainment, and checks their school transcripts to validate responses). This book attempts to explain why Michel's sampling is preferable to Greene's method for determining graduation rates. Greene, Mishel writes, ignores the "9th grade bulge" (students who repeat ninth grade form a bulge, and although they may receive a diploma, it won't be within four years); the unreliability of diploma counts; and how to account for unique populations such as immigrants, prisoners, military personnel, etc. But these idiosyncratic circumstances fail to dethrone Greene's crucial fact that some 30 percent of students--millions each year--who enter high school don't receive diplomas four years later. Call it what you want, but it's a definite problem. To dive into these figures yourself, check out Mishel's book online here.