We are endlessly tempted—and strongly encouraged by OECD’s Andreas Schleicher—to infer policy guidance from PISA results. If a country’s score goes up, maybe other countries should emulate its education practices and priorities, as they surely must be what’s causing the improvement.
The temptation is obvious, yet is best resisted, for the same reason that one ought not suppose that someone’s elevated temperature is the result of their fitness regimen (or lack thereof). A fever is generally the result of a bug—bacterial or viral—and is not caused by a surfeit or dearth of weightlifting and treadmilling. Similarly, a rising PISA math score among a nation’s fifteen-year-old population might be the consequence of new curriculum—or better teacher preparation—or longer school years—or heightened accountability pressure—or school choice—or stiffer college entrance requirements—or greater prosperity that enables parents to keep more of their kids in school longer—or changing societal mores regarding the education of girls. Or whatever. Schleicher and his team, as well as innumerable wonks and policy makers, around the world, just love to detect their favorite policy at work in a country with rising scores and then seek to propagate it elsewhere on the planet. They’re even happier to finger places with sagging scores for doing things they don’t approve of.
Yet much like a thermometer can yield an important clue that one’s health is not good—and a rising or plunging barometer usually signals changing weather—PISA scores are a valuable source of data on matters that any sane education policymaker wants to know: Is my country getting better or worse on the three core subjects that PISA tests? How does it compare with other countries that matter to us? And—digging in a little deeper—what do we see about the relative performance of boys and girls, of advantaged and disadvantaged youngsters, and of other important demographic groups?
PISA scores fluctuate, of course, and small bumps up and down don’t reveal anything significant. But when a country’s scores dramatically or steadily rise, or persistently decline, it’s at least a signal to those responsible for that country’s education that something good or bad is happening. And—let’s admit it—big ups and downs in Country A’s scores are also sure to catch the attention of policy wonks in Countries B, C, and D, whether or not their scrutiny lead to sensible inferences about causation.
Recent rounds of PISA have illustrated all these things at work. Why are China’s scores so high? Is it great schooling or totalitarian manipulation of who gets tested—or maybe some of both? What is it about Singapore that causes that wee nation’s lofty scores to keep rising? What’s the secret sauce in Estonia these days?
Back in 2001, German officials were appalled by how poorly their young people scored on the first round of PISA, and the resulting “PISA shock” catalyzed multiple changes in the FRG’s education system.
Without doubt, however, the most famous case of what we might term PISA envy is Finland, whose unexpectedly lofty scores back then—on three consecutive rounds of testing through 2006—triggered widespread interest in how that small Nordic land was handling K–12 education. The flames of fascination were expertly fanned by Pasi Sahlberg, who wrote and spoke convincingly (and in English) of the Finnish education miracle and whose Harvard Ed School affiliation lent added credibility to his insights.
Ed-policy tourists trooped to Helsinki and most of them loved what they saw there, because it wasn’t about school choice or test-based accountability, the kinds of policy reforms making major inroads in the U.S. and elsewhere on the planet, but about tolerant, well-educated, well-compensated, high-status teachers with plenty of planning time, hard at work in a uniform system of government-run schooling. It’s no exaggeration to say that, for a decade or so, just about every progressive educator on earth yearned to work in (or lead) a system like Finland’s.
What happened thereafter, however, cooled some of that ardor.
Beginning with the 2009 assessment round, Finland’s scores sagged in all three PISA subjects, a decline that continued in 2012, 2015, and in the 2018 results released last month. Not only did average scores sink but rich-poor gaps have widened and gender gaps are sizable.
Why this fall from Finnish education glory? That’s an excellent question, nicely explored by The Economist in a recent article titled “The parable of Finland.” The most interesting insight to be found there—at least in my eyes—is that Finland practiced a more traditional form of education (textbook based and teacher driven) in the 1990’s and the kids sitting for the early rounds of PISA had attended such schools. “By the time the results came out [however], many Finnish schools had started to move in a very different direction,” far more student-centered, discovery-oriented, and with teachers serving as guides-on-the-side. The uber-progressivism of recent years at least correlates with the sagging scores that Finland has lately notched on PISA.
Fascinating, yes, but also speculative. It could also be climate change, alterations in the reindeer migration, shifts in the Russia-Finland relationship, or perhaps the amazing new library in central Helsinki distracting kids from their lessons. Further analysis is definitely called for, yet we are unlikely ever to know for sure. What is certain is that, like the body temperature on that thermometer, PISA scores can helpfully signal that something is changing, but in and of themselves they cannot explain what’s causing it.
