Reassessing U.S. International Mathematics Performance: New Findings from the 2003 TIMSS and PISA
American Institutes for ResearchNovember 2005
American Institutes for ResearchNovember 2005
American Institutes for Research
November 2005
As reports go, this one is as close to a thriller read as it gets. In this brief but fascinating analysis American Institutes for Research (AIR) brings fresh, and surprising, insights to the 2003 Trends in International Mathematics and Science Study (TIMSS) results for students in grades 4 and 8, and the 2003 Program for International Student Assessment (PISA) results for students aged 15. Many folks have used these results to demonstrate our schools' mediocrity relative to those in other nations, especially at the secondary school level. For example, our own Mayhem in the Middle report noted that "in math, our fourth graders were at the international average, but by eighth grade, students in 27 other countries scored higher than U.S. students, with significant differences in 20 of those." (These numbers are based on 1995 TIMSS scores.) While AIR doesn't dispute such points (in fact, it finds "consistently mediocre U.S. results on all three assessments"), it does say the reality is not so simple. To begin, the three assessments (TIMSS-4, TIMSS-8, and PISA) tested different countries. When one examines our relative standing against a common subset of those countries, we fare the same across all three assessments - U.S. scores do not plummet as the children age (we're consistently ranked 8th or 9th out of 12 nations). But this is just the beginning. Noting that TIMSS focuses on content knowledge, while PISA emphasizes real-world applications, the AIR study debunks the myth that math class should increasingly emphasize the "real-world" to correct our PISA failings. Since U.S. students don't do too well on either TIMSS or PISA, the nation's math shortcomings are hardly limited to just one area. The study also explores gender differences, showing that only in the U.S. and Italy do boys outperform girls in math in grades 4 and 8, while by age 15 boys score better across all countries. And, it finds no evidence that the female drop-off is caused, as some suggest, by girls' weakening confidence in their own abilities. Moreover, the study shows that "statistics is the area of clear U.S. strength," while we do "relatively poorly on measurement and geometry." AIR argues for a national curriculum, noting that without one the U.S. has absurdly broad textbooks - as publishers strive to cover the topics prescribed by 50 different state standards. Consequently, we sacrifice instructional time on the essential topics. (Total teaching time is not holding the U.S. back - we're about average. We just allocate our time differently.) And finally, not surprisingly, they point out that we have far fewer math teachers with degrees in math than do other countries. It's an impressive amount of useful data and conclusions packed into a mere 32 pages. For your own copy, click here.
The Education Trust
November 2005
We know a lot about closing achievement gaps among the very young, but what about among adolescents? This report examines ways to improve American high schools for the 9th grade students who are already behind. Because the focus is on poorly achieving students, the authors didn't look to ritzy schools with high-flying test scores for insights. Instead, they only examined ones with high-poverty, high-minority populations and compared so-called "average-impact" high schools (those whose students achieve, on average, below state test standards) with "high-impact" high schools (those whose students post, on average, achievement data in line with their states' averages) in order to determine the characteristics that distinguished one type from the other. Researchers talked to students, teachers, and administrators and collected class schedules, student transcripts, and assignments. They conducted multiday site visits and observed classroom instruction. The results don't surprise. "High-impact high schools are clearly focused on preparing students for life beyond high school - specifically, college and careers," while average-impact high schools concentrate merely on graduation. Moreover, while schools in both categories offer support for new teachers, average-impact high schools focus on personal and social interaction among staff, while high-impact high schools spend far more time on imparting new arrivals with curriculum and instruction advice. Creating a culture of achievement, setting high expectations that correlate to set standards, and encouraging students to challenge themselves academically is a recipe for success. It's not a panacea for high schools' woes, but it's surely a good place to begin. Check it out, here.
Donald R. McAdams
Teachers College Press
January 2006
Veteran Houston school reformer (and 12-year school board member) Don McAdams, who also leads the Center for Reform of School Systems, authored this 175-page handbook for urban board members who want to reform their school systems. As former Education Secretary (and former Houston superintendent and school board colleague) Rod Paige says in the foreword, the book offers "a comprehensive theory of governance described by a simple conceptual framework that encompasses all the work of a reform board. Everything is here: from big ideas about core beliefs and theories of action for change, to the fundamental relationships and processes by which boards and superintendents work together, to the leadership responsibility boards have to build community support for sustained change." This volume may not get as large a readership as would benefit from it - so have a look, and spread the word. You can obtain ordering information here. (Be warned: it's pricey for a paperback.)
What if the No Child Left Behind Act works as intended? That's the question addressed by Susan Goodkin in her perceptive Washington Post op-ed, which argues that the law's focus on boosting low-performing students to the "proficient" level (as defined by the states) is harming gifted and talented students who are already far above that meager designation of adequacy. NCLB punishes schools for not improving the test scores of low-performing students, but it assiduously ignores academically gifted youngsters who often languish in classrooms where they are neither challenged nor engaged. Concludes Goodkin, "NCLB may end up producing an entire generation of merely proficient students - a generation that will end up working for the science leaders produced by other countries." This charge deserves to be taken seriously - and examined with empirical evidence - rather than dismissed out of hand by NCLB supporters. Our education system should be designed to help all students achieve their full potential, and that's the only way NCLB will achieve its full potential.
"Leave No Gifted Child Behind," by Susan Goodkin, Washington Post, December 27, 2005
It's better late than never; the thousands of children who were displaced by Hurricane Katrina are about to get federal assistance with their educations. While Congress took its time finalizing the package, the result is fair and reasonable. Importantly, it does not discriminate against parents who chose a non-public school for their displaced children. Aid is coming in two forms: 1) $645 million is earmarked for covering the costs associated with educating students who fled the disaster. States that took in hurricane victims will receive up to $6,000 per student ($7,500 for special ed students) they have accommodated. The money will flow to local school districts, who will cover the costs of displaced students attending both public and non-public schools. 2) $750 million will be distributed to both public and non-public schools in Alabama, Louisiana, Mississippi, and Texas to offset the cost of re-opening their doors. Predictably, and shamelessly, NEA's Reg Weaver (a.k.a., "The Grinch") slammed the program, saying the bill is "the worst assault on public education in American history. For the first time ever, taxpayers will be forced to pay for a nationwide voucher program." While the program might be an important precedent, demonstrating Congress's willingness to support the education of private school students, it is mostly a compassionate response to a national tragedy. Still, as Eric Hanushek of the Koret Task Force on Education says, "We would hope that it doesn't take a natural disaster to get everybody to focus on the learning of individual kids that haven't been particularly well served in the past."
"Bush To Sign 'Monumental' School Voucher Law," by Meghan Clyne, New York Sun, December 30, 2005
As if the NEA's pretending to care about helping schools and students wasn't bad enough, we now know it doesn't really care that much about helping teachers, either. A recent Wall Street Journal editorial highlighted how the nation's largest teachers union - that is, an organization ostensibly dedicated to bettering the prospects of, yup, teachers - spent about a third of last year's member dues on political goals, many of which have little to do with improving educators' lives. (How does Jesse Jackson's Rainbow PUSH Coalition help teachers?) Further, the NEA "has a $58 million payroll for just over 600 employees, more than half of whom draw six-figure salaries." But that doesn't mean the union isn't looking out for you, teachers of America. The NEA website opines: "As the national voice of more than 2.7 million public education employees, NEA knows that too many educators have been denied professional pay for too long.... Toward this end, NEA supports a minimum salary of at least $40,000 for all teachers in our nation's public schools and at least a living wage for every education support professional." Hear, hear! Living wages for all teachers! And caviar, Cuban cigars, and bottles of Cristal and 1986 Ch??teau Mouton Rothschild for the union reps who so dearly love them!
"Teachers' Pets," Wall Street Journal, January 3, 2005
Just before the holiday season, some 5,500 Birmingham middle school students received an early gift: the option to leave their failing schools. The city's school system was planning to offer students in its 17 low-performing middle schools voluntary tutoring, rather than the option of transferring to one of the district's better-performing schools. A community group complained to the U.S. Department of Education, which promptly ordered the Birmingham school system to offer the youngsters their NCLB-guaranteed option to transfer. The district said it was simply trying to avoid a mass exodus and overcrowding with the tutoring offer, but the excuses sound disingenuous. This past August, BSS sent 3,000 letters to children locked in failing elementary schools offering them a transfer, and only 23 took them up on the offer. Some "exodus." Gadfly has faith that the Magic City can find a way to accommodate those middle school students who might just desire a better education than their chronically failing schools provide. Kudos to the feds for not backing down from this one.
"U.S. orders city to allow middle school transfers," by Gigi Douban, Birmingham News, December 21, 2005
It's that time once again. Education Week has released its major (and as always, a bit unwieldy) annual Quality Counts report. This year's theme is "A Decade of Standards-Based Education." There are some bright spots, including an interesting regression model that shows increased student achievement linked to more-stringent state-level policies on standards, testing, and accountability. In the category of old news, NAEP scores from 1992-2005 display clear gains in math and flat-lined results in reading. (See here.) Another nonshocker: there is no correlation between equitable school financing and student achievement. (See here.) There's lots to read, including traditional state-by-state grades and analyses at the back of the book, and some thoughtful commentaries, especially one by Diane Ravitch calling for national standards and tests. Give it a read, but try not to lift it one-handed.
This past November, school violence again made headlines. The latest federal data was released in Indicators of School Crime and Safety: 2005, which tallied and analyzed the incidence of theft, violent crime, and teacher victimization in 2003. Brimming with demographic stats, the Indicators highlighted a nontrivial decline in school violence over the previous decade. In particular, it noted that the incidence of victimization among 12-18 year-old students fell by almost 50 percent between 1992 and 2003.
Surely, not a bad thing. But hold the applause. In fact, the violence rate in U.S. schools held steady from 1999 to 2003 at an average of 70 incidents per 1,000 students among 12-18 year olds. Violent incidents include theft, bullying, teacher victimization, rape, sexual assault, aggravated assault, and murder. This means that about 1 in 13 secondary students experiences some sort of victimization at school during the year.
How alarmed should we be? Set aside for now the serious matter of whether such data are accurate. (Experts in the field say these federal reports understate the true incidence of school violence because the Education and Justice departments base their information on surveys and "academic studies," not actual episodes reported to police and other law enforcement agencies.)
Try also to set aside one's horrified outrage at the rare but recurrent high-profile shootings on school campuses, such as Columbine in 1999 and the past year's grim episodes in Minnesota and Tennessee. We are properly shocked and appalled by these events, but the truth is that few students or teachers are actually killed in school, which is, on the whole, a safer place than the streets outside.
The larger questions are whether high-quality learning is apt to occur in settings where kids (or teachers) have to worry about their safety - and whether the United States is becoming inured to an unacceptably high level of school violence.
The late Senator Daniel Patrick Moynihan penned a celebrated 1993 article in The American Scholar entitled "Defining Deviancy Down," which picked up Emile Durkheim's assertion that "crime is normal," asked how much crime and other "deviance" a free society should expect to tolerate, and argued that the U.S. had begun to take far too high a level of misbehavior for granted. "It appears to me," Moynihan wrote, that "over the past generation ... the amount of deviant behavior in American society has increased beyond the levels the community can 'afford to recognize' and that, accordingly, we have been re-defining deviancy so as to exempt much conduct previously stigmatized, and also quietly raising the 'normal' level in categories where behavior is now abnormal by any earlier standard."
Have we been doing that with school violence? What's an acceptable level? For reference and comparison, it's useful to gaze across the Pacific and ask how Japan, a famously law-abiding land and one with student achievement levels that routinely surpass ours, views such things.
This past September, Japan's Education Ministry (MEXT) published its annual report on school violence, containing 2004 data. The Japanese newspapers reported widespread shock and outrage ("particularly alarming," said one) at the revelation that violence was on the rise in elementary schools, where such incidents had escalated from 1,253 incidents to 1,890 incidents across the country over the previous two years, even as middle and high school violence rates held steady.
How does Japan compare with the U.S.? Our national statistics do not include elementary school violence. But a rough comparison can be made at the secondary level, using MEXT statistics for "middle and high schools" alongside U.S. data for 12-18 year-old students.
The Japanese education ministry defines a violent act as an act against students, teachers, or property. Over the past three years, the Land of the Rising Sun has averaged 23,591 such incidents in middle school and 5,080 in the high schools. That translates to 6.4 and 1.3 violent incidents per thousand pupils per year, respectively. Recall that the U.S. figure is 70 per 1,000 students. In other words, the incidence of violence among teenagers in American schools is substantially more than 10 times the rate in Japanese secondary schools.
Japan's statistics apparently aren't perfect, either. An official in the Osaka prefecture noted that Tokyo's elementary schools were reported to suffer only 43 incidents compared to Osaka's 320. Considering that Tokyo is a much larger city, he surmised that its educators were under-reporting the problem - and the MEXT statistics incorporated the faulty data. It's hard to imagine, though, that their statistics are less reliable than ours, and harder still to suppose that data problems and definitional differences could account for more than a tiny portion of the huge difference between violent incidents in Japanese and American schools.
The facts are that (despite their own occasional horrific incidents) Japan's schools are vastly less violent than ours, and Japanese society is accustomed to them being peaceful, studious places. By contrast, we're accustomed to lots of misbehavior in school. We have defined down this form of deviancy from the norms we ought to expect. And it's a near-certainty that in the process we have also come to accept a school environment in which it's much harder to focus on academic achievement. In anybody's hierarchy of human needs, physical safety takes precedence over intellectual activity. In short, our kids' performance is not likely to rival Japan's until we demand - and produce - a school environment that is more conducive to teaching and learning.
How might this be achieved? In an earlier seminal article, published in The Atlantic in 1982, James Q. Wilson (the distinguished political scientist and, it happens, a close friend of Moynihan's) and George Kelling promulgated the theory that, if communities attend to their "broken windows," they will have far less serious crime. In effect, they defined deviancy up, establishing new norms for behavior and law enforcement. In the few cities that operationalized this approach, most notably New York, it has had a powerful positive impact.
The "broken windows" reasoning sometimes gets misapplied by schools in the form of mindless "zero tolerance" policies that fail to distinguish between youthful pranks and serious violence. But that reasoning, properly applied, contains the best hope for boosting the behavioral norms in K-12 education. Our most successful district, charter, and private schools expect a lot from their students, behaviorally as well as academically, and they surround youngsters with a culture that is as admiring of performance on both those fronts as it is intolerant of sloth, disruption, and violence. That's why really good schools are never "out of control" and rarely have much serious misbehavior. Their norms don't permit it. They repair cracks in the windows before they even break. If we had a lot more such schools, we'd have a lot less school violence.
Michael O'Keefe was Fordham's fall research intern.
American Institutes for Research
November 2005
As reports go, this one is as close to a thriller read as it gets. In this brief but fascinating analysis American Institutes for Research (AIR) brings fresh, and surprising, insights to the 2003 Trends in International Mathematics and Science Study (TIMSS) results for students in grades 4 and 8, and the 2003 Program for International Student Assessment (PISA) results for students aged 15. Many folks have used these results to demonstrate our schools' mediocrity relative to those in other nations, especially at the secondary school level. For example, our own Mayhem in the Middle report noted that "in math, our fourth graders were at the international average, but by eighth grade, students in 27 other countries scored higher than U.S. students, with significant differences in 20 of those." (These numbers are based on 1995 TIMSS scores.) While AIR doesn't dispute such points (in fact, it finds "consistently mediocre U.S. results on all three assessments"), it does say the reality is not so simple. To begin, the three assessments (TIMSS-4, TIMSS-8, and PISA) tested different countries. When one examines our relative standing against a common subset of those countries, we fare the same across all three assessments - U.S. scores do not plummet as the children age (we're consistently ranked 8th or 9th out of 12 nations). But this is just the beginning. Noting that TIMSS focuses on content knowledge, while PISA emphasizes real-world applications, the AIR study debunks the myth that math class should increasingly emphasize the "real-world" to correct our PISA failings. Since U.S. students don't do too well on either TIMSS or PISA, the nation's math shortcomings are hardly limited to just one area. The study also explores gender differences, showing that only in the U.S. and Italy do boys outperform girls in math in grades 4 and 8, while by age 15 boys score better across all countries. And, it finds no evidence that the female drop-off is caused, as some suggest, by girls' weakening confidence in their own abilities. Moreover, the study shows that "statistics is the area of clear U.S. strength," while we do "relatively poorly on measurement and geometry." AIR argues for a national curriculum, noting that without one the U.S. has absurdly broad textbooks - as publishers strive to cover the topics prescribed by 50 different state standards. Consequently, we sacrifice instructional time on the essential topics. (Total teaching time is not holding the U.S. back - we're about average. We just allocate our time differently.) And finally, not surprisingly, they point out that we have far fewer math teachers with degrees in math than do other countries. It's an impressive amount of useful data and conclusions packed into a mere 32 pages. For your own copy, click here.
The Education Trust
November 2005
We know a lot about closing achievement gaps among the very young, but what about among adolescents? This report examines ways to improve American high schools for the 9th grade students who are already behind. Because the focus is on poorly achieving students, the authors didn't look to ritzy schools with high-flying test scores for insights. Instead, they only examined ones with high-poverty, high-minority populations and compared so-called "average-impact" high schools (those whose students achieve, on average, below state test standards) with "high-impact" high schools (those whose students post, on average, achievement data in line with their states' averages) in order to determine the characteristics that distinguished one type from the other. Researchers talked to students, teachers, and administrators and collected class schedules, student transcripts, and assignments. They conducted multiday site visits and observed classroom instruction. The results don't surprise. "High-impact high schools are clearly focused on preparing students for life beyond high school - specifically, college and careers," while average-impact high schools concentrate merely on graduation. Moreover, while schools in both categories offer support for new teachers, average-impact high schools focus on personal and social interaction among staff, while high-impact high schools spend far more time on imparting new arrivals with curriculum and instruction advice. Creating a culture of achievement, setting high expectations that correlate to set standards, and encouraging students to challenge themselves academically is a recipe for success. It's not a panacea for high schools' woes, but it's surely a good place to begin. Check it out, here.
Donald R. McAdams
Teachers College Press
January 2006
Veteran Houston school reformer (and 12-year school board member) Don McAdams, who also leads the Center for Reform of School Systems, authored this 175-page handbook for urban board members who want to reform their school systems. As former Education Secretary (and former Houston superintendent and school board colleague) Rod Paige says in the foreword, the book offers "a comprehensive theory of governance described by a simple conceptual framework that encompasses all the work of a reform board. Everything is here: from big ideas about core beliefs and theories of action for change, to the fundamental relationships and processes by which boards and superintendents work together, to the leadership responsibility boards have to build community support for sustained change." This volume may not get as large a readership as would benefit from it - so have a look, and spread the word. You can obtain ordering information here. (Be warned: it's pricey for a paperback.)