The Education Gadfly Weekly

After three years, $45 million, and a staggering amount of video content, the Gates Foundation has released the third and final set of reports on its ambitious Measurements in Effective Teaching (MET) project (the first two iterations are reviewed here and here). The project attempted to ascertain whether it’s possible to measure educator effectiveness reliably—and, if so, how to do it. According to the project’s top-notch army of researchers (led by Tom Kane), it ain’t easy but it can be done and done well.

First, the research team used predictive data from the 2009–10 school year to randomly assign about 800 teachers in grades four through eight to classrooms (within their original schools) for 2010–11. The data showed a strong correlation between the predicted achievement of teachers’ students and their actual scores, as well as the magnitude of success. That the study randomly assigned teachers offers credence to the researchers’ contention that teachers’ success can be determined (and isn’t merely a byproduct of the quality of students who enter their classrooms in September). Second, they conducted a series of weightings to determine the ideal mix of past student-achievement data (value-added metrics, or VAM), classroom observations, and student surveys to identify the most effective teachers. Ultimately, the authors determined that a model that relies on VAM for between 33 and 50 percent of total teacher evaluation is best, with student surveys comprising 25 percent and classroom observations the rest.

Though the MET analysts concede that the best model of effectiveness heavily weights (as in 65 percent or more) teachers’ prior student-achievement gains on those same tests, they assert that the combo approach—which attaches lower percentages to test gains and higher to observations and student surveys—“demonstrated the best mix of low volatility year to year and ability to predict student gains on multiple assessments” (the latter referring to supplemental assessments described as “cognitively challenging”). This is where disagreement about the study commences: Jay Greene (no fan of the MET study) argues that because metrics like classroom observations don’t make the measure significantly more predictive (simply more reliable), but carry hefty price tags, we should be very wary of their inclusion. (We’ve heard this from him before.) Value-added experts agree that multiple years of test data are our best bet for reducing volatility in VAM measures, but that we should be careful in relying solely on them for high-stakes personnel decisions given their imperfections. Though Kane and colleagues are clear in describing VAM alone as a superior predictive measure, they could have done a much better job describing the tradeoffs involved in relying on VAM, student surveys, observations or any other measure for that matter—and what other purposes such measures might serve (such as guarding against gains due to simple test prep).

In any case, those looking for the policy takeaways: read the short summary report on the findings. Those looking to expand their statistical minds: read the three companion research papers. And those looking to seriously nerd out: watch for the full data sets, which Gates will be making available to other researchers in coming months.

SOURCE: Thomas Kane, et al., Measurements in Effective Teaching: Final Reports (Seattle, WA: Bill and Melinda Gates Foundation, January 2013).

Rick Hess has added an important piece to his shelf-busting bibliography with this new book, which seeks to dismantle conventional thinking about education leadership at all levels. Hess begins by presenting the problem: Ideological traps prevent leaders from adopting simple, widely used leadership tactics that are almost universally effective outside the world of education. Among these “cage-dwelling” ideological traps are the notions that improvement is only possible with more dollars, that hands are so bound by outside forces as to make improvement impossible, and that a district is good simply because it “sucks less” than the average. The rest of his pages give leaders the knowledge and strength they need to chisel through the bars of antiquated, platitude-heavy thinking. For example, he urges them to optimize funding, assign faculty according to their strengths, and know their districts’ collective-bargaining agreements—and work with lawyers to find creative contractual workarounds. This last point may be Hess’s most interesting contribution (and is similar to what he argued in Leadership Limbo): Union contracts, laws, and regulations don’t tie edu-leaders’ hands nearly as much as these leaders (and reformers) assume. They have the discretion and authority to change policy and practice, and they need to find the courage, creativity, and wherewithal to do so. (Reformers, too, would be wise to target energies toward this end, rather than continuing their single-pronged attack on unions.) Still, Hess is mindful that political fear indeed drives ed leaders’ commitment to the status quo. So he shows that reform can be (and has been) done: He interlaces examples of visionary leaders who have already succeeded in shattering the complacency cage, such as Mike Feinberg and Terry Grier. Hess’s book is informative, broad in its scope, precise in its solution, and fun to read (really). Throughout, he lightens this heady topic with pithy pop-culture references and ancient textual wisdom. In the end, Hess has created an informative guide that should give leaders the confidence to bar-bust their way to effective education leadership.

SOURCE: Frederick M. Hess, Cage-Busting Leadership (Cambridge, MA: Harvard Education Press, 2012).

This inaugural (and comprehensive) StudentsFirst education-policy report card grades all fifty states, plus D.C., on their reform orientation. The report focuses on three “policy pillars” of contemporary school reform: elevating the teaching profession (use of evaluations for personnel decisions, alternative-certification pathways, etc.); empowering parents with data and choice (comparable resources for charters, opportunity-scholarship programs, etc.); and spending wisely and governing well (teacher pensions, governance flexibility, etc.). The results are bleak. Louisiana and Florida topped the list with grades of B-minus. But two-thirds of states received Ds or Fs. For the “data and choice” pillar, only five states earned a C-minus or better. Still, there are some rosier points: Louisiana and Florida are lauded for policies mandating that teacher effectiveness be used as the primary driver of personnel decisions, and Indiana—catapulted by the charter and voucher reforms of superstar Tony Bennett (now gone to Florida)—received accolades for empowering parents with data and school choice. The report has understandably stirred much controversy in eduville (some of which surfaced during the release event yesterday). Many question how states that underachieve on NAEP, like Louisiana, could so outdo high-flying jurisdictions like Massachusetts (which earned a D-plus). (Others have less-astute responses to the report.) Some individual policy rankings have also catalyzed much discussion: The fiscal and governance pillar, for example, is built of all non-teacher and non-choice policies, making it difficult to interpret. (Policy objectives in this pillar range from removing class-size restrictions to reforming pensions.) Nevertheless, it is the attempt to quantify the seemingly un-quantifiable (which, trust us, is hard work) and move beyond the worn-thin school-reform banter that makes this study valuable. And if Florida’s slow and steady growth is any indication, most of the reforms upon which these rankings are based should gradually yield real improvement in the top-ranked states.

SOURCE: StudentsFirst, State Policy Report Card 2013 (Washington, D.C.: StudentsFirst, January 2013).

“Nobody is satisfied with the educational performance of Ohio’s poor, urban, and minority youngsters—or the schools that serve them.” This was how we opened our 2010 report Needles in a Haystack: Lessons from Ohio’s High-Performing, High-Need Urban Schools, which examined high-flying elementary schools. That sentiment is just as true for the high schools in 2012 as it was two years ago for the grade schools we examined. Yet there are high schools in the Buckeye State that buck the bleak trends facing too many of our urban students. This report examines six of them -- urban high schools that are making good on promises of academic excellence; specifically, schools that work for low-income and minority students. These high schools make serious efforts not to leave anyone behind.

StudentsFirst's much-awaited (and plenty contentious) 2013 State Policy Report Card awarded its highest rankings (B-minuses) to Louisiana and Florida; a dozen states earned an F. After California was flunked, chief deputy superintendent Richard Zeiger took his ire to the New York Times: “‘This group has focused on an extremely narrow, unproven method that they think will improve teaching—and we just flat-out disagree with them.’”

This video's panel discussion digs into the new report card, the future of education reform, and how to bridge the divide between policy and practice.

Predictably, the anti-reform crowd is having a field day with Sunday’s Washington Post article (and related video), which reported the relatively high rate of student expulsions in D.C.’s charter school sector. There’s some legitimacy to this exercise in schadenfreude, considering how many of us reformer types have used the success of high-flying “no excuses” charter schools to bludgeon middling (or worse) district schools with the accusation that “if the charters can do it, so can you.” The retort—well-founded, in my view—is that most, if not all, of these high-flying charters aren’t serving the same population of kids as their traditional public school peers. They inevitably do a bit of creaming (even if unintentionally) on the front end and a number of them push out disruptive students on the back end. Apples-to-apples comparisons are made difficult by this “selection bias.”

But that’s not necessarily a bad thing. In my view, we should be proud of the charter schools that are identifying and serving high-potential low-income students—kids who are committed to using education to escape poverty and are often supported in that effort by education-minded parents.

The reason to celebrate these schools and the role they play is because the traditional system has been downright hostile to the needs of such striving children and families—as have been many charter critics. Magnet “exam schools,” such as those recently profiled by Checker Finn and Jessica Hockett, are viewed with suspicion; tracking or ability grouping is seen as elitist; any effort to provide special classes, environments, or challenges for motivated or high-achieving kids is cast as perpetuating inequality—even when all the kids are poor, and even though there’s a ton of evidence that high achievers do best around other high achievers.

And now these “social justice” types want to berate schools for asking disruptive students to leave. For sure, there should be checks on pushing kids out willy-nilly. Thankfully, charter officials in D.C. are already on the case, publicizing discipline data and prodding the handful of schools with sky-high expulsion and suspension rates to find better approaches.

But let’s not forget about the needs (even rights) of the other kids to learn. Isn’t it possible that U.S. public schools have gone too far in the direction of accommodating the disruptors at the expense of everyone else? Or been guilty of “defining deviancy down,” in Daniel Patrick Moynihan’s words? As Eduwonk Andy wrote this week, it’s probably because charter schools are willing (and able) to enforce discipline that they are so popular with parents. That wouldn’t be true if they had to retain chronic disrupters.

To be sure, this raises tough questions for the system as a whole. As I said in the Washington Post video, there are reasons to be concerned that district schools will become the last resort for the toughest-to-serve kids.

But in life there are trade-offs, and I would be willing to accept a somewhat less ideal outcome for the most-challenged students if it meant tremendously better life outcomes for their peers.

Misguided notions of “equity” have turned many public school systems into leveling leviathans. We shouldn’t let the same happen to charters, the last salvation of the strivers.

Eric Grannis: "The charter school concept ... [is] about providing families with a wealth of choices."

Deborah Meier recently lamented that the charter school movement has been co-opted by people she terms “Reformer/Deformers”—folks who favor high-stakes testing, merit pay, and overly strict “no-excuses” school models and undervalue socioeconomic integration. While Ms. Meier and I come from somewhat different philosophical perspectives (she is a Socialist and I am, well, not), I must confess sympathy for an aspect of her critique: Our focus on test scores is alienating many educators and parents with more progressive educational philosophies. Her message is one that school-choice advocates should heed, lest we wind up shooting ourselves in the foot.

The charter school concept is about the organization of our school system. It’s about providing families with a wealth of choices and creating schools that can be more innovative and responsive to families because they are independent. It’s not—or ought not to be—about a particular method or philosophy of education. There shouldn’t be an official charter school pedagogy any more than there should be an official state religion.

There shouldn’t be an official charter school pedagogy any more than there should be an official state religion.

However, many parents feel that charter schools only provide a no-excuses option—an ironic twist, as charters are intended to enhance choice. Irrespective of whether no-excuses schools are good or bad, charter schools should not be so heavily associated with one particular pedagogical approach.

Our game plan seems to be that we’ll authorize schools with models proven to produce high test scores and thereby prove to skeptics that charters work. There’s just one problem with this theory: The skeptics don’t care. They don’t like high-stakes testing to begin with. We might even have more success in convincing them if we proved that charter schools did badly on high-stakes testing.

The charter movement’s emphasis on high-stakes testing and no-excuses schools is also unappealing to many education-minded middle-class families. That’s unfortunate because if more middle-class families started sending their children to charter schools, it would both make charter schools more diverse and strengthen our movement by broadening its base of support. Instead of focusing on testing, we need to encourage a greater variety of charter school models and do a better job of empowering educators and parents to create charter schools.

If our zeal for metrics means we end up supporting only a narrow range of school models, then we deprive parents of the ability to make choices.

The Community Roots Charter School in Brooklyn is a good example of how chartering can empower educators and satisfy parents who want a more progressive education. This school, founded by two Bank Street graduates, was built to be progressive and diverse. It is a point of pride that every class at Community Roots has a co-teaching model with a special-education teacher to support students with special needs. The school is fantastically popular among families of all races and classes. Moreover, even with less of a focus on testing than many other charters, its students stack up well on the same standardized tests: More than 70 percent of them are proficient in both math and English, despite the school’s large special-needs population.

Another example is Brooklyn Prospect Charter School. This school was founded by Daniel Rubenstein, an educator who had experience both in charter schools and as a department head at the prestigious private Collegiate School. He founded Brooklyn Prospect to combine “the best practices of public and private school” and to serve Brooklyn’s diverse population. While the makeup of the student body reads like a diverse public school, the school otherwise has more of the feel of a progressive private school. As a result, it attracts many talented educators from independent schools and middle-class families who wouldn’t be interested in a no-excuses school.

People in the financial community fund a lot of charter schools, especially in New York. They like hard metrics, so they focus on test scores. That’s understandable. After all, they want to be sure that the schools they create are actually good. However, if our zeal for metrics means we end up supporting only a narrow range of school models, then we deprive parents of the ability to make choices—a problematic concept for a movement premised on choice.

We need to put more resources into creating diverse-school models.

To be sure, many families like no-excuses schools. But there is also demand for more progressive schools, and that demand is going largely unmet by the charter movement, at least in cities like New York. This hurts us not only with parents but also with educators. And yes, some teachers regard charter schools as privatization and will never like them. However, there are many less-ideological educators who would be attracted to charter schools if they truly felt that charters empowered them.

Charter schools have the potential to satisfy progressive-minded parents and educators precisely because they are independent. But we need to put more resources into creating diverse-school models. Doing so will build a broader base of support for school choice in the education and parent communities. Moreover, it’s the right thing to do. Real school choice means offering real choices.

Eric Grannis is the executive director of the Tapestry Project, an organization that opens and supports diverse charter schools in New York City. He is a former public school teacher and a leading expert on charter school law.

Achieve released the second draft of the Next Generation Science Standards this week. Project-leader Stephen told Education Week that the new draft is quite different from the old one. Here’s hoping that’s true. Stay tuned for a review from our own science experts.

In a study released at last weekend’s American Economics Association conference, researchers argue that mandatory vaccination programs—in addition to reducing morbidity rates from the relevant childhood diseases—effectively increase students’ likelihood of graduating from high school, possibly because of the weeks of school that ailing kids would miss. Interestingly, this effect was twice as strong among minority students.

The tiff over teacher evaluations in New York City turned into an all-out brawl after Mayor Bloomberg likened the United Federation of Teachers to the NRA in his weekly radio show last Friday. The timing of this particular analogy may have been unfortunate. Still and all, the overarching point he sought to make was valid: Teacher-union leaders, like those of some other interest groups, might be out of sync with their membership. Bloomberg seems to have fallen victim to an old political landmine: Telling the truth.

Last week, the Atlantic ran a moving story by the mother of an autistic child whose public school teachers, despite the best of intentions, were largely unequipped to help him. In the end, the best option for her child was homeschooling—what she describes as a tough, protracted, but ultimately rewarding decision.

The New Jersey Education Association is suing to stop the state’s charter schools from availing themselves of blended learning in the classrooms. While the legal argument rests on the fact that the legislature has not yet evaluated such approaches, the union’s true motives are clear: It has foreseen that effective digital learning could cause schools to hire fewer teachers. For more, check out this week’s podcast.

StudentsFirst's much-awaited (and plenty contentious) 2013 State Policy Report Card awarded its highest rankings (B-minuses) to Louisiana and Florida; a dozen states earned an F. After California was flunked, chief deputy superintendent Richard Zeiger took his ire to the New York Times: “‘This group has focused on an extremely narrow, unproven method that they think will improve teaching—and we just flat-out disagree with them.’”

This video's panel discussion digs into the new report card, the future of education reform, and how to bridge the divide between policy and practice.

StudentsFirst's much-awaited (and plenty contentious) 2013 State Policy Report Card awarded its highest rankings (B-minuses) to Louisiana and Florida; a dozen states earned an F. After California was flunked, chief deputy superintendent Richard Zeiger took his ire to the New York Times: “‘This group has focused on an extremely narrow, unproven method that they think will improve teaching—and we just flat-out disagree with them.’”

This video's panel discussion digs into the new report card, the future of education reform, and how to bridge the divide between policy and practice.

After three years, $45 million, and a staggering amount of video content, the Gates Foundation has released the third and final set of reports on its ambitious Measurements in Effective Teaching (MET) project (the first two iterations are reviewed here and here). The project attempted to ascertain whether it’s possible to measure educator effectiveness reliably—and, if so, how to do it. According to the project’s top-notch army of researchers (led by Tom Kane), it ain’t easy but it can be done and done well.

First, the research team used predictive data from the 2009–10 school year to randomly assign about 800 teachers in grades four through eight to classrooms (within their original schools) for 2010–11. The data showed a strong correlation between the predicted achievement of teachers’ students and their actual scores, as well as the magnitude of success. That the study randomly assigned teachers offers credence to the researchers’ contention that teachers’ success can be determined (and isn’t merely a byproduct of the quality of students who enter their classrooms in September). Second, they conducted a series of weightings to determine the ideal mix of past student-achievement data (value-added metrics, or VAM), classroom observations, and student surveys to identify the most effective teachers. Ultimately, the authors determined that a model that relies on VAM for between 33 and 50 percent of total teacher evaluation is best, with student surveys comprising 25 percent and classroom observations the rest.

Though the MET analysts concede that the best model of effectiveness heavily weights (as in 65 percent or more) teachers’ prior student-achievement gains on those same tests, they assert that the combo approach—which attaches lower percentages to test gains and higher to observations and student surveys—“demonstrated the best mix of low volatility year to year and ability to predict student gains on multiple assessments” (the latter referring to supplemental assessments described as “cognitively challenging”). This is where disagreement about the study commences: Jay Greene (no fan of the MET study) argues that because metrics like classroom observations don’t make the measure significantly more predictive (simply more reliable), but carry hefty price tags, we should be very wary of their inclusion. (We’ve heard this from him before.) Value-added experts agree that multiple years of test data are our best bet for reducing volatility in VAM measures, but that we should be careful in relying solely on them for high-stakes personnel decisions given their imperfections. Though Kane and colleagues are clear in describing VAM alone as a superior predictive measure, they could have done a much better job describing the tradeoffs involved in relying on VAM, student surveys, observations or any other measure for that matter—and what other purposes such measures might serve (such as guarding against gains due to simple test prep).

In any case, those looking for the policy takeaways: read the short summary report on the findings. Those looking to expand their statistical minds: read the three companion research papers. And those looking to seriously nerd out: watch for the full data sets, which Gates will be making available to other researchers in coming months.

SOURCE: Thomas Kane, et al., Measurements in Effective Teaching: Final Reports (Seattle, WA: Bill and Melinda Gates Foundation, January 2013).

Rick Hess has added an important piece to his shelf-busting bibliography with this new book, which seeks to dismantle conventional thinking about education leadership at all levels. Hess begins by presenting the problem: Ideological traps prevent leaders from adopting simple, widely used leadership tactics that are almost universally effective outside the world of education. Among these “cage-dwelling” ideological traps are the notions that improvement is only possible with more dollars, that hands are so bound by outside forces as to make improvement impossible, and that a district is good simply because it “sucks less” than the average. The rest of his pages give leaders the knowledge and strength they need to chisel through the bars of antiquated, platitude-heavy thinking. For example, he urges them to optimize funding, assign faculty according to their strengths, and know their districts’ collective-bargaining agreements—and work with lawyers to find creative contractual workarounds. This last point may be Hess’s most interesting contribution (and is similar to what he argued in Leadership Limbo): Union contracts, laws, and regulations don’t tie edu-leaders’ hands nearly as much as these leaders (and reformers) assume. They have the discretion and authority to change policy and practice, and they need to find the courage, creativity, and wherewithal to do so. (Reformers, too, would be wise to target energies toward this end, rather than continuing their single-pronged attack on unions.) Still, Hess is mindful that political fear indeed drives ed leaders’ commitment to the status quo. So he shows that reform can be (and has been) done: He interlaces examples of visionary leaders who have already succeeded in shattering the complacency cage, such as Mike Feinberg and Terry Grier. Hess’s book is informative, broad in its scope, precise in its solution, and fun to read (really). Throughout, he lightens this heady topic with pithy pop-culture references and ancient textual wisdom. In the end, Hess has created an informative guide that should give leaders the confidence to bar-bust their way to effective education leadership.

SOURCE: Frederick M. Hess, Cage-Busting Leadership (Cambridge, MA: Harvard Education Press, 2012).

This inaugural (and comprehensive) StudentsFirst education-policy report card grades all fifty states, plus D.C., on their reform orientation. The report focuses on three “policy pillars” of contemporary school reform: elevating the teaching profession (use of evaluations for personnel decisions, alternative-certification pathways, etc.); empowering parents with data and choice (comparable resources for charters, opportunity-scholarship programs, etc.); and spending wisely and governing well (teacher pensions, governance flexibility, etc.). The results are bleak. Louisiana and Florida topped the list with grades of B-minus. But two-thirds of states received Ds or Fs. For the “data and choice” pillar, only five states earned a C-minus or better. Still, there are some rosier points: Louisiana and Florida are lauded for policies mandating that teacher effectiveness be used as the primary driver of personnel decisions, and Indiana—catapulted by the charter and voucher reforms of superstar Tony Bennett (now gone to Florida)—received accolades for empowering parents with data and school choice. The report has understandably stirred much controversy in eduville (some of which surfaced during the release event yesterday). Many question how states that underachieve on NAEP, like Louisiana, could so outdo high-flying jurisdictions like Massachusetts (which earned a D-plus). (Others have less-astute responses to the report.) Some individual policy rankings have also catalyzed much discussion: The fiscal and governance pillar, for example, is built of all non-teacher and non-choice policies, making it difficult to interpret. (Policy objectives in this pillar range from removing class-size restrictions to reforming pensions.) Nevertheless, it is the attempt to quantify the seemingly un-quantifiable (which, trust us, is hard work) and move beyond the worn-thin school-reform banter that makes this study valuable. And if Florida’s slow and steady growth is any indication, most of the reforms upon which these rankings are based should gradually yield real improvement in the top-ranked states.

SOURCE: StudentsFirst, State Policy Report Card 2013 (Washington, D.C.: StudentsFirst, January 2013).