The Education Gadfly Weekly

Many complain, legitimately, that the ed-reform world has been overly focused on math and reading scores, to the detriment of other important—but not as easily assessed—student outcomes. This working paper out of the University of Arkansas aims to address this issue by exploring a potential new measure of non-cognitive ability: survey-item response rates. The authors use data from the 1997 National Longitudinal Survey of Youth that tracks a nationally representative sample of young adults; respondents are born between 1980 and 1984 (making them between the ages of twenty-nine and thirty-three now) and are surveyed annually on issues like employment, assets, and wages. When the analysts compared information collected in 1997 to the respondents’ highest educational outcomes as reported in 2010 or earlier, they find that the number of items either left blank or answered “I don’t know” is a significant predictor of educational attainment, even after controlling for many factors, including cognitive ability. The fewer the number of questions left unanswered, the greater the likelihood overall that the respondent had enrolled in college. (For example, a one-standard-deviation increase in response rates increased the amount of education received by .31 years, or 11 percent of a standard deviation.) The authors posit that failure to respond to these questions could mean a loss of interest or lack of effort, which they contend is a valid measure of conscientiousness. While one might dispute this assertion—and while the inclusion of “don’t-know” responses in their count could be an issue, given that the respondent could legitimately not know the answer to that particular question—this is an intriguing, creative, and promising exploratory study.

SOURCE: Colin Hitt and Julie R. Trivitt, “Don’t Know? Or Don’t Care? Predicting Educational Attainment Using Survey Item Response Rates and Coding Speed Tests as Measures of Conscientiousness,” EDRE Working Paper No. 2013-05 (August 2013).

Mike welcomes Rick Hess back to the show by threatening to shoot him through a tube from San Francisco to Los Angeles. They chat proficiency rates, whether the Common Core is Jeb Bush’s RomneyCare, and Philly’s school-budget woes. Amber approaches non-cognitive ability in a creative new way.

Amber's Research Minute

Don’t Know? Or Don’t Care? Predicting Educational Attainment Using Survey Item Response Rates and Coding Speed Tests as Measures of Conscientiousness by Colin Hitt and Julie R. Trivitt, EDRE Working Paper No. 2013-05 (August 2013)

Growing numbers of parents, educators, and school administrators are calling for a local "opt-out" from state tests and accountability systems.

Is this opt-out a cop-out? Or would students benefit from a system that their own teachers and principals devised? Should all schools be offered an opt-out alternative, one in which they propose to be held accountable to a different set of measures? What about opt-outs for high-achieving schools or schools with good reason to be different? Would such a system move us toward or away from the goals of the Common Core? As for charter schools, must they continue to be tethered to uniform statewide accountability systems? Or should we rekindle the concept of customizing each school's charter and performance expectations?

Let’s invent a game; it’s called “Rate This School!”

Start with some facts. Our school—let’s call it Jefferson—serves a high-poverty population of middle and high school students. Eighty-nine percent of them are eligible for a free or reduced-price lunch; 100 percent are African American or Hispanic. And on the most recent state assessment, less than a third of its students were proficient in reading or math. In some grades, fewer than 10 percent were proficient as gauged by current state standards.

That school deserves a big ole F, right?

Now let me give you a little more information. According to a rigorous Harvard evaluation, every year Jefferson students gain two and a half times as much in math and five times as much in English as the average school in New York City’s relatively high-performing charter sector. Its gains over time are on par or better than those of uber-high performing charters like KIPP Lynn and Geoffrey Canada’s Promise Academy.

Jefferson is so successful, the Harvard researchers conclude, because it has “more instructional time, a relentless focus on academic achievement, and more parent outreach” than other schools.

Now how would you rate this school? How about an A?

***

My little thought experiment makes an obvious point, one that isn’t particularly novel: Proficiency rates are terrible measures of school effectiveness. As any graduate student will tell you, those rates mostly reflect a school’s demographics. What is more telling, in terms of the impact of a school on its students’ achievement and life chances, is how much growth the school helps its charges make over the course of a school year—what accountability-guru Rich Wenning aptly calls students’ “velocity.” This is doubly so in the Common Core era, as states (like New York) move to raise the bar and ask students to show their stuff against a college- and career-readiness standard.

To be sure, proficiency rates should be reported publicly, and parents should be told whether their children are on track for college or a well-paying career. (That’s one of the great benefits of a high standard like the Common Core.) But using these rates to evaluate schools will end up mislabeling many as failures that might in fact be doing incredible work at helping their students make progress over time.

Let’s go back to Jefferson. As a middle school, it welcomes children who enter several grade levels behind. Even if these students make incredible gains in their sixth-, seventh-, and eighth-grade years, they still won’t be at grade level, much less “proficient,” when they sit for the state test. Furthermore, unless the state gives an assessment that is sensitive enough to detect progress—ideally a computerized adaptive instrument that allows for “out of grade level” testing—it might not give Jefferson the credit for all the progress its students are making.

Here’s the rub: There are thousands of Jeffersons out there: Schools with low proficiency rates but strong growth scores. (See figure, borrowed from this Shanker Blog post, and notice in particular the many schools whose “growth percentile” is above 50 but whose percent proficient is below 50.)

This is particularly the case with middle schools and high schools, serving as they do students who might be four or five grade levels behind when they enter. Is it any surprise that middle schools and high schools are significantly more likely to be subject to interventions via the federal School Improvement Grants program? They are being punished for serving students who are coming to them way, way below grade level.

Again, none of this is particularly new or noteworthy. Others (especially reform critics) have made the same arguments countless times before. Yet an emphasis on proficiency rates over student growth is still entrenched in state and federal policy. Yes, Margaret Spellings allowed for a “growth model pilot” when she was secretary of education—but schools still had to get all students to proficiency within three years, an unrealistic standard in states with a meaningful (and rigorous) definition of proficiency. Arne Duncan has also espoused the wisdom of looking at progress over time, yet his ESEA waiver rules require state accountability systems to take proficiency rates into account—those are expected to be the drivers in identifying “focus” and “priority” schools. Nor are Democrats in Congress any better; their ESEA reauthorization bills would maintain No Child Left Behind’s reliance on “annual measurable objectives” driven by proficiency rates.

The charter sector is wedded to proficiency rates, too.  In New Orleans, for instance, the Recovery School District has shut down schools with low proficiency rates but strong individual student gains over time in the cause of boosting quality.* What it has really done, however, is closed schools worthy of replication, not extinction.

***

Have you figured out by now the true identity of our “Jefferson School”? It’s the highly-acclaimed Democracy Prep. According to the new Common Core–aligned New York test, it’s a low-proficiency-rate, high-growth school. Seth Andrew, Democracy Prep’s founder, explained to me,

Like the rest of New York, our Democracy Prep Public Schools saw dramatic drops in "proficiency rates." In fact, we saw declines that were even greater than most. Why?

1) Entry Grade Level: Charters that enroll at the K-1 level did dramatically better than those (like Democracy Prep) who enroll in the middle school grades. This is potentially GREAT news for urban education because it means that if students don’t fall dramatically behind, they can get on grade level by grade 3, and stay on or above grade level over time. However, it is not even remotely reasonable to compare schools that randomly enroll in kindergarten to those that enroll in the sixth grade. One school has had seven years with a student while we’ve had nine months!

2) Growth Matters Most: The metric that no one has seen yet and that will be the most important to our teachers, administrators, students, and families at Democracy Prep is not “proficiency” but “value-added growth.” The reason we have operated only “A” rated schools every year since 2006 is primarily because 60 percent of that grade has been based on individual student growth, a metric on which our scholars and teachers post some of the most dramatic improvements year-over-year. In fact, even this year, our percent of “1’s” goes dramatically down in grade seven while our “2’s” go up, and by eighth grade we’ve dramatically reduced “1’s” and substantially increased “3’s and 4’s.”

My key point here is that NO ONE in this work, especially at Democracy Prep, makes so-called “miracle school claims” as reported by our critics. We believe, in fact we KNOW, that educating low-income students is incredibly hard work, compounded by the challenges of poverty, mobility, ELL status, and disability. These are not excuses; they are facts. To move our scholars from whatever grade or performance level they enter to be ready for success in the college of their choice and a life of active citizenship takes us at least five years. Given that time, our scholars consistently out-perform wealthy Westchester County on their Regents exams in nearly every subject and our first class of graduates outperformed white students on their SAT’s. Nearly 70 percent of our graduates met the NYC “aspirational performance measure” for college readiness compared to 22 percent across NYC and we require that our graduates earn an Advanced Regents Diploma because, as these new CCSS results prove, the old bar was far too low.

Is Democracy Prep an A school or an F school? The answer seems obvious to me—and should apply to any school with similar results, name brand or not, charter school or not. It’s time that policy caught up to common sense and put proficiency-rates-as-school-measures out of their misery once and for all.

* CORRECTION:  The school I had in mind is Pride College Prep; someone associated with the school had informed me that it had made strong student-level gains. But I've learned from Michael Stone, the Chief External Relations Officer at New Schools for New Orleans, that in fact data from CREDO showed the school's gains to be among the weakest in the RSD. Furthermore, Michael wrote to me, "None of the charters BESE or the RSD closed would have been candidates for replication." My apologies for the error.

Over the past twenty years, opponents have charged charter schools with further Balkanizing America’s education system. Give parents a choice, the thinking goes, and many will choose homogenous environments for their children. And there’s certainly evidence that charters in some cities tend to be more racially isolated than traditional public schools.

But could charter schools actually be a solution to segregation—particularly as gentrification brings more white and middle-class families to our urban cores? A growing crop of social entrepreneurs thinks so. In cities across the country, educators and parents are starting charters expressly designed for diversity.

Charter schools have certain advantages. As start-up schools, they can be strategic about locations, picking spots that are well positioned to draw students from different racial and socioeconomic groups. They can design academic programs that take diversity as a given and make the most of it. And they can be thoughtful about putting elements in place to appeal to whites and blacks, Asians and Hispanics, rich and poor.

Capital City Public Charter School in Washington, D.C., was founded in 2000. It’s one of the oldest charter schools with significant racial and socioeconomic diversity. It serves elementary and middle school students from almost every zip code in the city, which helps it achieve a nearly even racial and socioeconomic balance. Of its student population, 36 percent are black, 32 percent are white, 28 percent are Hispanic, and 4 percent are Asian. Forty percent of students are eligible for free or reduced price lunch. Perhaps its demographics help to explain why Cap City was the very first school Barack and Michelle Obama visited after the president’s first inauguration.

Capital City is also a proudly progressive school; it uses an Outward Bound expeditionary-learning approach that engages students in fieldwork, community service, and interactions with experts. This is a big attraction for many parents, especially the more affluent ones. (Research has shown that upper-middle-class parents, especially whites, are more likely to want a “progressive” education for their children, though this type of teaching is generally less popular among blacks and low-income parents.)

Another reason the school has successfully attracted affluent white families and maintained a racial balance is that it was founded by white parents. At the time, the founders’ children attended Hearst Elementary, a public school in a ritzy Northwest D.C. neighborhood. After becoming upset when a new superintendent imposed an unfriendly principal on their school and transferred some of their best teachers elsewhere, these parents recruited Karen Dresden, one of their teachers, to run the charter school. Dresden is still running Cap City today, and she credits the parents for having the vision to create a diverse school. “What I was really impressed with was they were committed from the very beginning to creating a school not just for their own kids to attend, but for all children,” Dresden told me.

The school’s original location was critically important, too. Like many charters, Cap City had to scramble to find available space; it settled on a commercial facility in the heart of the city. Ten years ago, Dresden said, “This was really a rocky area. It was all vacant lots; it wasn’t developed like it is now. When we decided to locate the school here there were a few members of the founding group who decided, for various reasons, that they did not want their kid to come here. I think they were feeling a little too unsafe. But that was actually a really good thing. The people who were really committed to it said, ‘This is where we’re going to have the most diversity.’”

Now Cap City has such a strong reputation that it doesn’t have to worry about recruiting enough white and middle-class students. There is a mile-long waiting list of children from affluent neighborhoods like Capitol Hill and Chevy Chase. “There are families choosing Capital City today that would have never chosen it when we started,” Dresden shares. The school’s bigger problem is making sure that enough low-income and minority students enter its lottery. (Local policies don’t allow charter schools to hold separate lotteries by race or class.) To that end, the school does an enormous amount of outreach in Latino and African American communities. So far, it has maintained its diversity, but as the District of Columbia continues to gentrify, this will be a continuing struggle.

Nevertheless, schools like Cap City demonstrate that charter schools, with their regulatory freedom and entrepreneurial zeal, can be spunky solutions to seemingly intractable problems— including, it turns out, America’s segregated schools.

This piece originally appeared in the American Sociological Association's Contexts magazine and is adapted from The Diverse Schools Dilemma.

A Chicago public school and public library will begin to share space on Thursday, breaking ground for a new “library-within-a-school” model that may be “copied and mimicked all across the city,” according to an enthusiastic Mayor Rahm Emanuel. The Windy City’s schools and libraries have both seen financial troubles in the last couple of years. Library Commissioner Brian Bannon has clarified that proliferation of this model would be about “reducing storefront and leased space” and possibly result in moving libraries, not closing libraries. Gadfly likes efficiency and books—so hat tip!

The school-funding crisis in Philadelphia has reached the boiling point: After Superintendent William Hite issued an ultimatum stating that schools may not open in time if the district does not receive at least $50 million more in funding by Friday, August 16th, Mayor Michael Nutter announced that it would borrow the cash, apparently obviating that eventuality. Now that the district will be able to re-hire some laid-off staff members, the School Reform Commission—Philadelphia’s appointed school board—will vote on whether to suspend portions of state law to grant Hite the flexibility to re-hire for reasons other than seniority. The unions, naturally, are furious, but this appears to be the best possible outcome for students.

This week, Atlanta Public Schools Superintendent Erroll Davis asked the district’s Board of Education to stop approving new charter schools. The reason: Georgia’s Supreme Court has yet to decide whether Davis can withhold millions of dollars in tax revenues from charters to help pay off an old pension debt in the district (one that charters didn't create), and Davis said he couldn't push for new schools that “will not pay their share.” The district's Board of Education turned down his request and approved a new charter anyway, but that still leaves the court fight—and a lot of hostility in Atlanta towards charters. Good thing voters in the Peach State last fall created an independent charter authorizer.

M. Night Shyamalan—a filmmaker of creepy-movie and twist-ending fame—has gone from scaring the daylights out of us with aliens popping out of cornfields to publishing a book about education reform. In his first forays into education philanthropy, Shyamalan granted scholarships to inner-city Philly children—but was disheartened by his contribution’s lack of a long-term impact. So, approaching the issue “in terms of plot structure,” he delved into the literature on racial divisions and education reform—and emerged with his own book. Is it worth a read? Stay tuned for a Gadfly review!

This week, Los Angeles kicked off the first year of its $30 million program to get Apple iPads in the hands of 31,000 students and around 1,500 teachers (to be expanded to $500 million and all 600,000-plus students in about a year, if everything goes according to plan) with a mass teacher-training effort. Pearson is providing the curriculum, and the devices cost $678 a pop (each pre-loaded with educational software, a case, and a three-year warranty). The training seminars touted the iPad’s potential to revolutionize the classrooms—from allowing teachers the flexibility to either use Pearson’s built-in Common Core–aligned materials or introduce their own strategies, to making it easier for students with limited verbal skills to communicate. While Gadfly reserves the right to be skeptical that the results will live up to the hype, we’re willing to reserve our judgment too—for now.

Mike welcomes Rick Hess back to the show by threatening to shoot him through a tube from San Francisco to Los Angeles. They chat proficiency rates, whether the Common Core is Jeb Bush’s RomneyCare, and Philly’s school-budget woes. Amber approaches non-cognitive ability in a creative new way.

Amber's Research Minute

Don’t Know? Or Don’t Care? Predicting Educational Attainment Using Survey Item Response Rates and Coding Speed Tests as Measures of Conscientiousness by Colin Hitt and Julie R. Trivitt, EDRE Working Paper No. 2013-05 (August 2013)

Growing numbers of parents, educators, and school administrators are calling for a local "opt-out" from state tests and accountability systems.

Is this opt-out a cop-out? Or would students benefit from a system that their own teachers and principals devised? Should all schools be offered an opt-out alternative, one in which they propose to be held accountable to a different set of measures? What about opt-outs for high-achieving schools or schools with good reason to be different? Would such a system move us toward or away from the goals of the Common Core? As for charter schools, must they continue to be tethered to uniform statewide accountability systems? Or should we rekindle the concept of customizing each school's charter and performance expectations?

Many complain, legitimately, that the ed-reform world has been overly focused on math and reading scores, to the detriment of other important—but not as easily assessed—student outcomes. This working paper out of the University of Arkansas aims to address this issue by exploring a potential new measure of non-cognitive ability: survey-item response rates. The authors use data from the 1997 National Longitudinal Survey of Youth that tracks a nationally representative sample of young adults; respondents are born between 1980 and 1984 (making them between the ages of twenty-nine and thirty-three now) and are surveyed annually on issues like employment, assets, and wages. When the analysts compared information collected in 1997 to the respondents’ highest educational outcomes as reported in 2010 or earlier, they find that the number of items either left blank or answered “I don’t know” is a significant predictor of educational attainment, even after controlling for many factors, including cognitive ability. The fewer the number of questions left unanswered, the greater the likelihood overall that the respondent had enrolled in college. (For example, a one-standard-deviation increase in response rates increased the amount of education received by .31 years, or 11 percent of a standard deviation.) The authors posit that failure to respond to these questions could mean a loss of interest or lack of effort, which they contend is a valid measure of conscientiousness. While one might dispute this assertion—and while the inclusion of “don’t-know” responses in their count could be an issue, given that the respondent could legitimately not know the answer to that particular question—this is an intriguing, creative, and promising exploratory study.

SOURCE: Colin Hitt and Julie R. Trivitt, “Don’t Know? Or Don’t Care? Predicting Educational Attainment Using Survey Item Response Rates and Coding Speed Tests as Measures of Conscientiousness,” EDRE Working Paper No. 2013-05 (August 2013).