The Education Gadfly Weekly

Guanglei Hong and Stephen W. Raudenbush
Educational Evaluation and Policy Analysis
Fall 2005

If a kindergarten student is struggling to succeed academically, does it make sense to hold the child back? And how does that decision affect the performance of his/her classmates? These are difficult questions to address, but the authors give it the old college try. On the latter question, they find no effect, simply because fewer than 5 percent of students are typically retained. The former question yielded more interesting results. The authors found that for the retained students, their academic performance diminished significantly. Had the students instead been promoted to first grade, their math and reading achievement gaps (relative to the normally-promoted students) would be halved. Could this be correct? As with much education research, a key question is whether the analysis has sufficiently approximated a random experiment. Ideally, one would study a group of children who all deserved to be retained, while randomly promoting some and retaining others. Of course, no such experiment exists, so the authors did the next best thing: they compared similar students, some of whom were promoted and some of whom were retained. To ensure similarity, they relied on a robust data set from the National Center for Education Statistics, the Early Childhood Longitudinal Study Kindergarten, with detailed student-level data including demographics, and even indicators of parental involvement and home life. Hong and Raudenbush constructed their model from 207 such indicators. It's impressive work, but doubts remain. These students clearly were retained with cause - their teachers and/or parents judged them not ready for the first grade for some reason. Was that reason some subtle or immeasurable, but important, difference between retained students and those promoted? Furthermore, was it this undefined or undetected difference, and not just the act of retention, that contributed to their slow growth? (The authors briefly explain their statistical test and dismiss the possibility of "unmeasured confounders," but there is little information by which a lay person can judge it.) Though written for statisticians, it's an interesting paper that raises some significant questions. We're not persuaded that their answer is as conclusive as they suggest, however, and of course we'd be curious to know whether one would find similar conclusions in higher grades. You can find it online here.

Michael Gurian and Kathy Stevens
Jossey-Bass
2005

If you're a male, read this review in an open field while gazing up a haystack; it might help. Or so think this book's authors, who believe boys are falling behind girls in the classroom because boys have been cut off from the farm since the Industrial Revolution. I kid you not. "If you think back to how your ancestors were educated, you'll notice that until about a hundred years ago, in all parts of the world, our sons' primary teachers were not lone individuals in schoolrooms but families, tribes and natural environments.... Not until about two hundred years ago did printing and the written word become a major part of a boy's educational life. It was at that point that the Industrial Revolution was upon us." It is true that until recently most boys did not learn in classrooms, but don't chalk it up to their preference for being outdoors. Rather, for most of the world's history, education was limited to the elite. Boys have learned quite well in the classroom, thank you, for most of recorded history. Whether in ancient China, where boys and young men ran a gamut of civil service exams that make current high-stakes U.S. tests look laughable; or late-medieval Europe, where boys studied in monasteries or scriptoria to become manuscript copyists; or in Eastern Europe, where young Jewish males sat at the feet of the rabbi and studied in intimate detail the words of the Hebrew scriptures and commentaries thereon; boys have studied, and excelled, in the classroom. The authors seem oblivious to such historical facts, however, as they're too eager to impress us with what new technologies (PET scans, MRIs, and SPECT scans) teach us about how boys learn. In very few pages (roughly less than 10 percent of the book), they describe some breakthroughs in neuroscience over the past decade, and then spend the balance of the book suggesting ways schools should change to accommodate boys better. Among their more dubious ideas: introduce "male-friendly" language arts programs that draw on movies and videos. Still, the book has some merit. It correctly notes that boys are disproportionately diagnosed with ADHD and that they receive the vast majority of Ds and Fs in school. Moreover, if you're into armchair psychology, you may enjoy analyzing Gurian, who goes on ad nauseam about how misunderstood he was as a child. Girls can do so by the fireplace, at their mother's apron. Guys, you'd best take it outside.

Brian P. Gill, Laura S. Hamilton, et al.
RAND Corporation
2005

It takes times to make a fine wine, and time to raise academic achievement. That's RAND's less-than-scintillating conclusion in its report on Edison Schools, the nation's largest, private education management company. According to the study, achievement gains for schools in their first two years under Edison control generally don't vary much from those of comparison institutions. And in many cases, student performance actually declines immediately following an Edison takeover. Four years after Edison takes charge of a school, however, many of them begin to show significant improvements. And after five years, Edison school students in general produce test scores that are significantly higher than those in comparable, non-Edison classrooms. (RAND doubters say it matters greatly which year the analysts selected as their starting point.) In any case, little here is predictable on the basis of age alone. The study notes that, while some Edison schools function at high academic levels, others never emerge from their achievement slumps. What's the difference? According to RAND, Edison operations that post the greatest gains are those not micromanaged by local education authorities. In short - the fewer the constraints, the better the school. RAND's findings are said to be the "most comprehensive independent assessment of Edison schools ever conducted." But education reformers will find little new here. In fact, Edison founder Chris Whittle serves up most of RAND's conclusions in his own new book (see here). Those who thirst for statistics may enjoy the study's bevy of charts and graphs. The rest can probably satisfy themselves with a glass of Sonoma's finest and this squib.

Andrew Rudalevige, Dickinson College
Kennedy School of Government, Harvard University
October 2005

One canard frequently hurled at the No Child Left Behind Act is that it was a right-wing plot to discredit, and then dismantle, public education. As for hard evidence, however, conspiracy theorists, who live in a fantasy world where NCLB-backer Ted Kennedy is either a Republican pawn or a closet conservative, have little to substantiate their beliefs other than a dubious pessimism that schools are incapable of improving themselves. Back on earth, political scientist Andrew Rudalevige reminds us that, far from dismantling public education, NCLB may well lead to even greater funding and resources. He lays out his evidence in a provocative paper written for last week's Harvard conference, "Adequacy Lawsuits: Their Growing Impact on American Education." His counterintuitive, though ultimately commonsense, argument is that the truckloads of performance data produced by NCLB-style accountability systems provide an edge to litigants filing "funding adequacy" lawsuits in states across the land. As the author says, "By requiring every student to reach proficiency on challenging content standards by 2014, by requiring a 'highly qualified' teacher in every classroom and a variety of interventions when students fail to make progress, NCLB effectively declares that every child can learn, if only given the resources to do so." This declaration buttresses arguments that litigants and legislators were already making at the state level. For example, Maryland's "Thornton Commission," set up to uncover a solution to the state's school finance inequity problems, found that "schools are being adequately funded when the amount of funding provided is sufficient to allow students, schools, and school systems to meet prescribed state performance standards." Through this circular reasoning, states have boxed themselves into a responsibility to help all students achieve high standards - a much tougher task than simply making education available to all. No Child Left Behind only raises the stakes - and, in time, probably, the costs. Rudalevige concludes by wondering whether the odd coalition of accountability hawks and adequacy proponents will stick together through the 2007 reauthorization and beyond. There's every reason to believe that the adequacy crowd will continue to prize NCLB as a boost for their cause; whether supporters of accountability (including conservatives) will feel comfortable aiding the big-spenders of the adequacy movement is another question altogether. Decide for yourself; the draft paper can be found here. You can view all of the conference papers here.

[Editor's Note: The following editorial draws on the 2004 long-term trend NAEP results. These should not be confused with the 2005 "main NAEP" reading results discussed in our October 19 press release, Gains on State Reading Test Evaporate on 2005 NAEP.]

In times not too-far gone, if you wanted to get to know someone you asked him what he was reading. Today, the question is a joke, especially among teens.

"Reading - for fun?" (Big smile, followed by loud laugh.) "But seriously, how many tunes are on your i-Pod?"

The decline in leisure reading is no laughing matter, however. In fact, the Department of Education's report, NAEP 2004 Trends in Academic Progress: Three Decades of Student Performance in Reading and Mathematics, which was released this summer, offers some tantalizing information that suggests a connection between students' generally poor performance on academic reading tests and their declining leisure-reading habits.

Let's consider academic reading scores first. Among nine-year-olds, the news is very good. Their performance on the NAEP 2004 showed significant gains over previous assessment years. In fact, 2004 recorded this age group's highest scores ever. But thirteen-year-olds improved not at all between 1999 and 2004, and as a group they've only improved slightly since 1971.  For seventeen-year-olds, there is no measurable difference between their scores in 2004 and 1971. Further, the percentage of seventeen-year-old students deemed capable of understanding "complicated information" fell 3 percentage points, from 41 percent in 1994 to 38 percent a decade later.

Now consider a chart buried deep in the report that records how often students read "for fun" - not for homework or for the workplace, but for their own pleasure. Among seventeen-year-olds, the number who "never or hardly ever" read rose 10 percentage points, while those who read for fun "almost every day" fell 9 percentage points from 1984 to 2004. Thirteen-year-olds followed a similar pattern. Those who reported reading "almost every day" dropped 5 percentage points, while those who read "never or hardly ever" rose 5 percentage points.

And what of the nine-year-olds, whose reading scores are up? Their leisure reading is also up, slightly. Those who report reading "almost every day" rose 1 percentage point since 1984, while those who reported reading "never or hardly ever" dropped 1 percentage point.  That more than half of nine-year-olds (54 percent) reported reading for fun practically all the time is significant in itself, but that their numbers have remained firm while the older students' have plummeted deserves attention.  At the very least, with achievement in each age group correlated with outside reading, we should widen our vision to consider the role of books in the leisure hours of teenagers.

There is an impressive and growing body of survey research on leisure reading that complements the NAEP 2004 study.  A year earlier, the National Endowment for the Arts (NEA) surveyed more than 17,000 adults on their reading habits.  (I was the study's project director.)  The ensuing report, Reading at Risk: A Survey of Literary Reading in America, showed that from 1992 to 2002 the number of 18- to 24-year-olds who read at least one novel, story, poem, or play in the previous 12 months fell from 53 percent to 43 percent.  At the same time, the portion of adults reading any book at all fell 8 points, from 59 percent to 51 percent.

A few months later, the Bureau of Labor Statistics (BLS) released its American Time Use Survey.  That survey asked more than 21,000 respondents to document their activities during the day, including work, sleep, school, and leisure activities.  Among the leisure activity options listed on the survey was reading of any kind. Among fifteen- to 24-year-old respondents, the average number of minutes spent per day reading was a meager 8 minutes - Eight!

Soon thereafter, UCLA issued its 2004 American Freshman Survey, which showed that the number of entering college students who never read for pleasure rose five percentage points from 1994 to 2004.

The NEA, BLS, and UCLA surveys all measure voluntary reading, not reading for school.  The NAEP report did ask about assigned reading in school and for homework Unfortunately, though, there are no surprises.  Nine-year-olds are reading more in school, while seventeen-year-olds report no change.

Clearly, teens are spending their leisure hours on activities unrelated to homework or pleasure reading. It doesn't take another study to prove what most adults already know about how teens spend their leisure time, but we have one nonetheless. Last March, the Kaiser Foundation's study, Generation M: Media in the Lives of 8-18 Year-olds, found that kids in this age group consume fully eight-and-a-half hours of media per day in just six-and-a-half hours. How do they do this? Multi-tasking! Teenagers will watch television, for example, while downloading music. Cell phones are an added diversion. NOP World Technology's mKids Study (2005) found that 75 percent of fifteen- to seventeen-year-olds, and 40 percent of twelve- to fourteen-year-olds, own cell phones.

This avalanche of diversions is consuming teenagers' out-of-school hours.  As a result, when analyzing reading scores we need to add the voluntary reading habits of teenagers to our ongoing concerns over curriculum, pedagogy, and school policy.   Consider the proportions.  English teachers have a student, on average, for five hours a week in class. They may also use homework to demand students' attention for a few additional hours.  Outside of this, however, young people are chatting, surfing, blogging, recording, downloading, and playing computer games.  Many of these are, to be sure, language activities, but they don't help develop verbal aptitude.  (If they did, we'd see a spike in reading scores for seventeen-year-olds.) 

All this means that during the semester, teens spend about eight hours a week reading, and up to 50 hours on various other forms of media. The imbalance is worse during vacation periods. The monumental reform efforts in the public schools may continue, but if reading scores among teens are to improve, the leisure habits of high school students had better change.

- Mark Bauerlein is Professor of English at Emory University, and recently served as Director of Research and Analysis at the National Endowment for the Arts.

"Seniors of Kellenberg Memorial High School - You've just had your prom cancelled. What are you going to do now?" Why, go to Disney World, of course. It seems the $20,000 rental house in the Hamptons and liquor-loaded chauffeured limousines became simply too much for the principal of Long Island's tony Catholic high school to accept. So he called the whole shebang off. In a letter to the school's parents, he said, "It is not primarily the sex/booze/drugs that surround this event, as problematic as they might be; it is rather the flaunting of affluence, assuming exaggerated expenses, a pursuit of vanity for vanity's sake - in a word, financial decadence." Despite some modest backlash from parents, who are considering hosting an un-official prom, the cancellation has gone down fairly well. One student noted that "you can't really argue with the facts they present.... It's just what it's evolved into." But no worries - he's still got the four-day senior class trip to Disney World to look forward to this April. Climbing into his jet-black Infiniti, he told the AP reporter, "We fly down together and stay in the same hotel and so it's not like we're totally losing everything." Mr. Principal, it looks like you still have some work to do. Call your chaperones.

"L.I. Principal Nixes School's Senior Prom," by Frank Eltman, Press of Atlantic City, October 16, 2005

District schools in Columbus, Ohio, are finally exploring ways to bring students back into the fold. Why now? Because the exodus of students to charter schools is hitting the district in its pocketbook. Last spring, the district set its budget for 2005-2006 based on an estimate of 6,200 students taking the charter option. But so far this year, it's looking more like 7,100 students. This means the district will have to tap into reserves to cover a looming budget shortfall. (In Gadfly's hometown of Dayton, at least 30 percent of the kids are enrolled in charter schools.) Columbus school board member Jeff Cabot thinks the school system can rally. "We're going to fight for kids and offer what parents want. We'll get them back." Among their methods for fighting back is changing the way Columbus delivers education. The district is investigating the popular schools-within-a-school model for possible adoption at some institutions. And this year, Columbus is offering three single-sex schools, "a direct response to thriving single-sex charter schools," according to the Columbus Dispatch. With so many districts "fighting back" against charters by pulling dirty tricks (see here), it's heartening to see Columbus fight the old-fashioned and honorable way—by competing. (If dirty tricks are also afoot in the capital of the Buckeye State, we trust that concerned readers will inform us.)

"Columbus schools trying to analyze, predict charter exodus," Associated Press, Dayton Daily News October 13, 2005

"Battle for Columbus Students: Charter Exodus Pains District," by Jennifer Smith Richards, Columbus Dispatch, October 1, 2005

Even as Education Secretary Margaret Spellings and President Bush were struggling to shine the best possible light on (the mostly disappointing) 2005 NAEP scores (The Nation's Report Card), charter school supporters have reason to celebrate. Fourth-grade charter school students nationwide improved on their 2003 scores, especially in reading, "at a faster rate than students in traditional public schools," according to a National Alliance for Public Charter Schools (NAPCS) analysis. The trends are especially encouraging for poor and minority fourth-grade students. "African-American, Latino, and low-income charter students ... registered larger reading gains than their fourth-grade peers" in non-charter public schools." Moreover, "gains among Hispanic charter fourth graders were so strong that [these students] have opened a 10-point gap with non-charter students." The news wasn't as rosy for eighth-grade charter students, who trail their non-charter public school counterparts both in math and in reading. Bryan Hassel of Public Impact sounds a word of caution. "There's no way, using NAEP tools, to determine if the differences in scores are statistically significant," he told the Gadfly in an interview. Still, the charter progress is positive, though all of the nation's schools have a long way to go before anyone can declare success.

"Charter Schools Closing Achievement Gap in Fourth Grade Reading and Math," National Alliance for Public Charter Schools, October 2005

"Education Law Gets First Test in U.S. Schools," by Sam Dillon, New York Times, October 20, 2005

"Closing the Achievement Gap: 2005 NAEP Reading and Math Results Show Some Gains, But Slowing Progress," The Education Trust, October 19, 2005

To Idaho, now, where the state Board of Education wants to implement high school entrance requirements. If enacted, all eighth-grade students would need to earn a cumulative C average in four subjects and pass pre-algebra before moving on. Those who do not will - presumably - be retained for another year. That's the catch, though. The board hasn't specified how it intends to provide for those middle-schoolers who miss the mark. Would they simply stay where they were for another year, repeating that which they failed at the first time around, or would they move to a special, remedial middle school? The board has no specific plan for financing either option. And how fair is it to base requirements on GPA when a C in one school's science class could easily translate into a D or F in another's? The impetus for all this is Idaho's standardized test scores which, like scores around the nation, drop off precipitously between the elementary and high school years. Instead of implementing vague exit requirements, Idaho might do better to base promotion on the passage of a rigorous exam. Better yet, it might focus on reforming and restructuring its middle school curriculum. (See here.) If the Gem State's middle schools are like those in the rest of the country, they suffer from low academic expectations and an emphasis on social development rather than learning. Giving middle school teachers an incentive to inflate their students' scores could make that illness even more acute.

"Ticket to high school may be a C average," by Bill Roberts, Idaho Statesman, October 12, 2005

The number of home schoolers is on the rise, thanks to the combined impacts of hurricanes Katrina and Rita. Displaced residents, many grown tired of placing their children in new surroundings, have decided to take on the education burden themselves. In Plaquemines Parish, Louisiana's southern-most, school officials estimate that some 800 families are taking the do-it-yourself route. For some parents, the added duty is not only physically taxing, but mentally as well. "Math'll be hard," said one resident. "It's not just addition and subtraction - it's everything." Other parents are more optimistic. "This is a beautiful short-term solution, especially given where we are now," said a former resident of New Orleans now living in Baton Rouge and teaching her twin 9-year-old daughters. Policy people should be mindful of what's happening. This is a unique opportunity to measure and observe how well home schooling parents - both the well-educated and the not-so-well-educated - can do with this challenging task.

"Across Louisiana, parents become makeshift teachers," CNN.com, October 6, 2005

Andrew Rudalevige, Dickinson College
Kennedy School of Government, Harvard University
October 2005

One canard frequently hurled at the No Child Left Behind Act is that it was a right-wing plot to discredit, and then dismantle, public education. As for hard evidence, however, conspiracy theorists, who live in a fantasy world where NCLB-backer Ted Kennedy is either a Republican pawn or a closet conservative, have little to substantiate their beliefs other than a dubious pessimism that schools are incapable of improving themselves. Back on earth, political scientist Andrew Rudalevige reminds us that, far from dismantling public education, NCLB may well lead to even greater funding and resources. He lays out his evidence in a provocative paper written for last week's Harvard conference, "Adequacy Lawsuits: Their Growing Impact on American Education." His counterintuitive, though ultimately commonsense, argument is that the truckloads of performance data produced by NCLB-style accountability systems provide an edge to litigants filing "funding adequacy" lawsuits in states across the land. As the author says, "By requiring every student to reach proficiency on challenging content standards by 2014, by requiring a 'highly qualified' teacher in every classroom and a variety of interventions when students fail to make progress, NCLB effectively declares that every child can learn, if only given the resources to do so." This declaration buttresses arguments that litigants and legislators were already making at the state level. For example, Maryland's "Thornton Commission," set up to uncover a solution to the state's school finance inequity problems, found that "schools are being adequately funded when the amount of funding provided is sufficient to allow students, schools, and school systems to meet prescribed state performance standards." Through this circular reasoning, states have boxed themselves into a responsibility to help all students achieve high standards - a much tougher task than simply making education available to all. No Child Left Behind only raises the stakes - and, in time, probably, the costs. Rudalevige concludes by wondering whether the odd coalition of accountability hawks and adequacy proponents will stick together through the 2007 reauthorization and beyond. There's every reason to believe that the adequacy crowd will continue to prize NCLB as a boost for their cause; whether supporters of accountability (including conservatives) will feel comfortable aiding the big-spenders of the adequacy movement is another question altogether. Decide for yourself; the draft paper can be found here. You can view all of the conference papers here.

Brian P. Gill, Laura S. Hamilton, et al.
RAND Corporation
2005

It takes times to make a fine wine, and time to raise academic achievement. That's RAND's less-than-scintillating conclusion in its report on Edison Schools, the nation's largest, private education management company. According to the study, achievement gains for schools in their first two years under Edison control generally don't vary much from those of comparison institutions. And in many cases, student performance actually declines immediately following an Edison takeover. Four years after Edison takes charge of a school, however, many of them begin to show significant improvements. And after five years, Edison school students in general produce test scores that are significantly higher than those in comparable, non-Edison classrooms. (RAND doubters say it matters greatly which year the analysts selected as their starting point.) In any case, little here is predictable on the basis of age alone. The study notes that, while some Edison schools function at high academic levels, others never emerge from their achievement slumps. What's the difference? According to RAND, Edison operations that post the greatest gains are those not micromanaged by local education authorities. In short - the fewer the constraints, the better the school. RAND's findings are said to be the "most comprehensive independent assessment of Edison schools ever conducted." But education reformers will find little new here. In fact, Edison founder Chris Whittle serves up most of RAND's conclusions in his own new book (see here). Those who thirst for statistics may enjoy the study's bevy of charts and graphs. The rest can probably satisfy themselves with a glass of Sonoma's finest and this squib.

Guanglei Hong and Stephen W. Raudenbush
Educational Evaluation and Policy Analysis
Fall 2005

If a kindergarten student is struggling to succeed academically, does it make sense to hold the child back? And how does that decision affect the performance of his/her classmates? These are difficult questions to address, but the authors give it the old college try. On the latter question, they find no effect, simply because fewer than 5 percent of students are typically retained. The former question yielded more interesting results. The authors found that for the retained students, their academic performance diminished significantly. Had the students instead been promoted to first grade, their math and reading achievement gaps (relative to the normally-promoted students) would be halved. Could this be correct? As with much education research, a key question is whether the analysis has sufficiently approximated a random experiment. Ideally, one would study a group of children who all deserved to be retained, while randomly promoting some and retaining others. Of course, no such experiment exists, so the authors did the next best thing: they compared similar students, some of whom were promoted and some of whom were retained. To ensure similarity, they relied on a robust data set from the National Center for Education Statistics, the Early Childhood Longitudinal Study Kindergarten, with detailed student-level data including demographics, and even indicators of parental involvement and home life. Hong and Raudenbush constructed their model from 207 such indicators. It's impressive work, but doubts remain. These students clearly were retained with cause - their teachers and/or parents judged them not ready for the first grade for some reason. Was that reason some subtle or immeasurable, but important, difference between retained students and those promoted? Furthermore, was it this undefined or undetected difference, and not just the act of retention, that contributed to their slow growth? (The authors briefly explain their statistical test and dismiss the possibility of "unmeasured confounders," but there is little information by which a lay person can judge it.) Though written for statisticians, it's an interesting paper that raises some significant questions. We're not persuaded that their answer is as conclusive as they suggest, however, and of course we'd be curious to know whether one would find similar conclusions in higher grades. You can find it online here.

Michael Gurian and Kathy Stevens
Jossey-Bass
2005

If you're a male, read this review in an open field while gazing up a haystack; it might help. Or so think this book's authors, who believe boys are falling behind girls in the classroom because boys have been cut off from the farm since the Industrial Revolution. I kid you not. "If you think back to how your ancestors were educated, you'll notice that until about a hundred years ago, in all parts of the world, our sons' primary teachers were not lone individuals in schoolrooms but families, tribes and natural environments.... Not until about two hundred years ago did printing and the written word become a major part of a boy's educational life. It was at that point that the Industrial Revolution was upon us." It is true that until recently most boys did not learn in classrooms, but don't chalk it up to their preference for being outdoors. Rather, for most of the world's history, education was limited to the elite. Boys have learned quite well in the classroom, thank you, for most of recorded history. Whether in ancient China, where boys and young men ran a gamut of civil service exams that make current high-stakes U.S. tests look laughable; or late-medieval Europe, where boys studied in monasteries or scriptoria to become manuscript copyists; or in Eastern Europe, where young Jewish males sat at the feet of the rabbi and studied in intimate detail the words of the Hebrew scriptures and commentaries thereon; boys have studied, and excelled, in the classroom. The authors seem oblivious to such historical facts, however, as they're too eager to impress us with what new technologies (PET scans, MRIs, and SPECT scans) teach us about how boys learn. In very few pages (roughly less than 10 percent of the book), they describe some breakthroughs in neuroscience over the past decade, and then spend the balance of the book suggesting ways schools should change to accommodate boys better. Among their more dubious ideas: introduce "male-friendly" language arts programs that draw on movies and videos. Still, the book has some merit. It correctly notes that boys are disproportionately diagnosed with ADHD and that they receive the vast majority of Ds and Fs in school. Moreover, if you're into armchair psychology, you may enjoy analyzing Gurian, who goes on ad nauseam about how misunderstood he was as a child. Girls can do so by the fireplace, at their mother's apron. Guys, you'd best take it outside.