The Education Gadfly Weekly

In my last post, I described a demand-side approach to bringing evidence-based practices into schools by developing programs and processes that help educators ask the right questions and find new solutions that work for them. Now I’d like to tackle the supply side: the creation and marketing of tools, especially curriculum, that can help drive evidence-based change in the classroom.

While the Golden Age of Education Practice might still be a figment of my imagination, the Golden Age of High Quality Curriculum is here and now. That’s because a number of promising curricula have come onto the market in recent years, driven by the opportunity window opened by the Common Core. (And no, despite what you may have heard, that window has not yet closed. Most state standards are still very close to the Core.)

Here’s how Jared Myracle, Chief Academic Officer of the Jackson-Madison school district in Tennessee, puts it:

[A] few years ago, there weren’t many curricula aligned to the new standards. Fortunately, that has changed—education leaders describe a “curriculum renaissance.” I’ve seen that firsthand: All four curricula that we adopted in Jackson-Madison County became available in the last few years.

He’s right: According to EdReports, thirteen products have met its high standards for alignment and usability in elementary or middle school English language arts; ten in math. These include programs by nonprofit upstarts, like Great Minds’ Wit and Wisdom and Eureka Math K-5, EL Education Language Arts, and Open Up Resources Math, authored by Illustrative Mathematics, and as well as offerings from for-profits and legacy publishers, including American Reading Company’s ARC Core, Pearson’s ReadyGen and MyPerspectives, and McGraw-Hill’s StudySync. 

To be sure, reasonable people may dispute whether validation from EdReports, or other entities that review instructional materials, makes a curriculum “evidence based.” These new curricula haven’t been around long enough to have gone through experimental-design studies and demonstrated impacts on student achievement. But they are aligned to state standards, which themselves are informed by evidence, and have also been found by educators to be teachable in real-world settings. That’s something.

This raises a key question—one recently posited by my friend Rick Hess—about what we mean when we reference “evidence based practices.” Do we mean discreet teacher behaviors, like how best to instruct students on phonemic awareness? These are the sorts of practices you’ll find in the What Works Clearinghouse practice guides, or the National Reading Panel reports.

But another approach is to bundle a bunch of evidence-based practices and insights together into a coherent and teachable whole. When this is also aligned to academic standards—the what plus the how—we call that a high quality curriculum. Of course, there’s a risk that curriculum designers will stretch the evidence and make claims that can’t be defended; who doesn’t say that their pet curriculum is based on the best evidence? So until we can subject new curricula to experimental studies and check on the effectiveness of the whole package, the best we can probably do is rely on expert and educator reviews, like those from EdReports.

If you build it, they won’t come—not without a marketing budget

So let’s stipulate that schools and districts have an increasing number of high quality curricula to choose among, the kind that do a solid job incorporating evidence-based practices and that track the academic standards states have adopted. Surely these programs are sweeping the country, right?

Sorry. As far as I can tell, only 10–15 percent of districts are using the good stuff today. And while that portion may rise somewhat with time, as textbook adoption cycles come around, or as states follow Louisiana’s lead and make buying the good stuff easier, we’d be nuts to think that the non-profit upstarts in particular will be able to compete without a serious investment in sales and marketing.

Even medicine, which many evidence-based-practice enthusiasts look to with envy, has a challenge with the adoption of new and proven methods. And that’s even though doctors number in the hundreds of thousands, not millions; they operate with a high degree of professional autonomy; and they received lengthy, rigorous pre-service training in scientific fields.

That’s why, as you may have noticed, pharmaceutical companies spend zillions of dollars advertising to doctors and patients—because medical practice doesn’t automatically change because of new findings in respected journals, either.

But just running more ads in Education Week or the ASCD SmartBrief won’t be enough to get great curricula adopted by schools. The new providers need massive sales forces, too. Consider this passage from a great Atul Gawande piece in the New Yorker on the challenge of changing doctors’ and nurses’ practices. (Thanks to Ben Riley of Deans for Impact for pointing it out to me; I’ll return to more of this article in a future post.)

I once asked a pharmaceutical rep how he persuaded doctors—who are notoriously stubborn—to adopt a new medicine. Evidence is not remotely enough, he said, however strong a case you may have. You must also apply “the rule of seven touches.” Personally “touch” the doctors seven times, and they will come to know you; if they know you, they might trust you; and, if they trust you, they will change. That’s why he stocked doctors’ closets with free drug samples in person. Then he could poke his head around the corner and ask, “So how did your daughter Debbie’s soccer game go?” Eventually, this can become “Have you seen this study on our new drug? How about giving it a try?” As the rep had recognized, human interaction is the key force in overcoming resistance and speeding change.

The legacy textbook firms have sales reps all over the country—often former superintendents with personal relationships with the people they are marketing to, with budgets for steak dinners, golf games, and such. Not to mention, less salaciously, big-time budgets for free samples of books, which can get expensive quickly given the large size of many districts’ selection committees.

So if funders and advocates want to see a dramatic uptick in the uptake of high quality materials, especially those produced by the non-profits, they need to face these facts. We can’t just invest in building great curricula—we need to invest in marketing them, too.  To my knowledge, only Great Minds has committed serious money to constructing this sort of sales force, and it’s one reason why Eureka Math is grabbing significant market share. Other new entrants will need to do the same if they want to play in the big leagues.

There is another approach, which is get legacy publishers to create and sell high quality curricula. As noted above, some of that’s already happening, according to EdReports, at least in some cases. Or the non-profits can dance with the devils, as Illustrative Mathematics did with its new deals with McGraw Hill and Kendall-Hunt. Rather than compete with the legacy companies, team up with them. Unleash their armies of sales reps to promote good stuff.

This too is not without risk, though. Stories are legion of high quality titles getting abandoned by corporate publishers when not enough copies sell fast enough.

There are also big question marks about how traditional publishers support the professional development needs of districts. Districts and schools risk “implementation failure” if they adopt new curricula that tee up major shifts in practice for teachers, as the high quality curricula do, without supporting teachers with implementation. Historically, publishers have included professional development “free with the purchase” of a curriculum, and the running joke among districts is that it’s worth about what they pay for it. Shallow “spray and pray” PD won’t help teachers improve their practice, and it remains to be seen whether traditional publishers will build quality services around this need.

So the challenge of getting high quality, standards-aligned, evidence-infused instructional materials into schools is daunting. But let’s end on a positive note: Ten years ago, most of these materials didn’t even exist. We’ve made real progress thanks to the hard work of thousands of developers and educators over the past decade. Compared to many other reforms in education, getting more schools and districts to adopt one of the growing numbers of strong programs is eminently doable.

Florida has been a national leader on private school choice for a long time. The state’s tax credit scholarship program is the largest of its kind the nation, supporting over 100,000 low-income children in 1,800 participating private schools. A new study this week from the Urban Institute suggests those students are more likely to go to college and graduate than their peers in traditional public schools. The state also offers private school options and assistance to children with special needs, victims of bullying, and those in grades three through five who are having difficulty reading.

Now the political stars have aligned to allow Florida Republicans—if they choose—to push the school choice agenda even further. With the election of Governor Ron DeSantis, a victory credited by The Wall Street Journal to African American “school choice moms” voting to protect tax-credit scholarships and charter schools, choice proponents are champing at the bit to push school choice into the middle class, even making Florida the first state in the nation with universal choice.

Republicans have had “trifecta” control of the governor’s office, Senate, and House since 2011. But what has school choice advocates dreaming big is three new appointments DeSantis has made to Florida’s Supreme Court since taking office January, which leaves no Democratic appointees on the bench. The state’s highest court has been a stumbling block to the school choice agenda, and a sore point among choice advocates dating back to the 2006 Bush v. Holmes decision, which ruled the state’s Opportunity Scholarship voucher program unconstitutional. The state’s various private school choice options in the years since have all taken the form of tax credit scholarships or state-funded programs targeted to students with special needs making them politically difficult to challenge.

What was once a five-two liberal majority is now six-one a conservative court—by some accounts the most conservative top state court in the nation, and one that’s presumably open to reversing Holmes. Speaking at a School Choice Week event two weeks ago at the James Madison Institute, a free-market think-tank in Tallahassee, Florida’s new state education commissioner Richard Corcoran remarked, “It’s hard not to be a little giddy right now.” DeSantis’s election and his court appointees make it “just a whole different world and different paradigm,” he said.

Almost without exception, every state official and school choice advocate I recently met in Florida believes the state will be the first to have “universal” education savings accounts, opening private school choice options to all who seek them. But a debate has emerged between Republican lawmakers who are wary about “biting off more than we can chew,” and those who see an opportunity to “go big or go home” while the GOP has control—a delicate state of play that is unlikely to last forever in a state with one of the nation’s most evenly divided electorates.

For school choice advocates, all the pieces are in place: a supportive Governor; an education commissioner who was an aggressive choice supporter as Florida’s House speaker; and committed Republican majorities in the House and Senate, who are apt to cooperate with the new Governor during his honeymoon first year in office. Twenty years of private school choice programs mean there are already thousands of schools serving scholarship students—and serving them well based on the new Urban Institute findings. The shift in the court’s composition clears away the last foreseeable hurdle. 

However Florida’s GOP lawmakers are wary about overplaying their high cards. Interviews with several of them suggest they are more inclined to push the choice agenda incrementally, continuing to build political support for it. “We have to get rid of the low-income waiting list before we proceed to anything else,” said Republican Manny Diaz, Jr. who chairs the Senate Education committee, echoing DeSantis’s call to get 13,000 students currently waiting for placements off the waitlist for tax credit scholarships. “We should make sure the [private school] market can sustain and provide quality options for whoever’s entering it for their educational options,” Diaz said.

The apparent reluctance to press immediately the full choice agenda is causing at least some frustrations among school choice advocates, who argue the opportunity to include the middle class in private choice options is not just good for students, but smart politics. “Florida should be wary of becoming the next Massachusetts,” says one, alluding to the failed 2016 ballot initiative to raise the Bay State’s charter cap. Their argument is that where choice programs are seen as serving only low-income students, the middle class can easily turn against them because their children don’t benefit—particularly when teachers unions and others rally support for the idea that choice and charters damage traditional public schools.

No state in the country has a more delicate political alchemy than Florida. When DeSantis was elected by a razor-thin margin, one wag on Twitter noted that even if Florida had to choose between ice cream and a kick in the face, it would still go to a recount. Republicans see school choice as a vote-getter; school choice advocates look at the same set of facts on the ground and urge boldness. “It’s just such a slim margin, says Erika Donalds, a former Collier County School Board member and chairman of School Choice Movement. “I wouldn’t want to put all my chips on current school choice moms swinging it one way or the other.”

It falls to DeSantis to be either incremental or bold. To see which way the wind is blowing, keep an eye on education commissioner Corcoran. A Republican colleague who has worked closely with him describes him as a “brilliant strategist” with first-rate political instincts, and someone whose “heart is all in” on school choice. “Everything we did in the last ten years was to create the environment to be able to get to universal choice,” says this veteran Republican. “If it’s possible and there’s a window to do something aggressive, he would be jumping in and leading the charge.”

If you haven’t read Dylan Wiliam’s Creating the Schools Our Children Need, there’s now more reason to do so. In addition to a helpful review of the book by Robert Pondiscio, the National Center on Education and the Economy (NCEE) recently posted an enthralling nineteen-minute interview with Wiliam. In his conversation with NCEE’s new CEO Tony McKay, they spent much of their time unpacking the book’s subtitle, Why What We're Doing Now Won't Help Much (And What We Can Do Instead).

A global education leader who has won acclaim on the other side of the pond, Wiliam’s perspective is welcome at a time when the national discourse on education—to the extent there is one—is ever more polarized. His principal concern is that the effect sizes of the reforms we’ve been implementing in the U.S. are nowhere worth their opportunity cost to prepare young people to meet the increasingly complex demands of work and society. Moreover, these reforms are often expensive—with a marginal return on investment—and difficult to implement.

For instance, Wiliam points to recent efforts to pay good teachers more, a timely example given the labor unrest here in Denver around the district’s leading-edge pay model, known as ProComp (Professional Compensation Plan). He notes that the effect size of increasing pay on student achievement is nominal, and that substantially more money would be required to have an impact. To wit, as groundbreaking as ProComp was, pundits generally agree that it merited attention only because it was “better than awful.”

The conversation between McKay and Wiliam explores ways to improve teaching and learning. He persuasively argues for more humility in our efforts to improve teacher effectiveness, and offers a four-step framework to evaluate how we might better go about what we invest in. According to Wiliam, prioritizing a knowledge-rich curriculum and improving the decision-making tools at our teachers’ disposal would be a wiser use of time and effort.

As a former teacher, principal, and district leader, I found myself nodding in agreement with Wiliam’s ideas, but as a policymaker and advocate, it was less clear what implications his message might carry. At one point during the interview, McKay steers clear of the topic of alternative pathways into teaching after Wiliam makes his viewpoint known: “I don’t want the best and the brightest if they’re not going to stay in teaching… Schemes like Teach For America [TFA] aren’t going to have a big impact because when those teachers are just getting into their stride, they often leave.”

It’s a familiar criticism—one I’ve heard often as a TFA alumnus—and not one without any merit. Reasonable people can disagree about whether the money and effort spent to recruit and support TFA teachers, and similar routes that bypass traditional schools of education, might be put to better use investing in those who intend to make a career of teaching. Yet one would be hard-pressed to identify an entity that has had more of a lasting effect on public education, especially from a policy and advocacy standpoint, where alumni have led on, among other issues, trailblazing curriculum reforms.

To be fair, Wiliam’s critique of TFA is a relatively minor point in the interview, which I would encourage watching in its entirety. But it raises the question of what to do when educational best practices don’t intersect with sound education policy. All the more so given the big shifts currently underway in education reform.

We get a hint of Wiliam’s thoughts on the policy-practice divide in response to McKay’s final question on whether states have a role in ensuring that effective classroom practices are able to flourish. (A trigger warning to fellow policymakers: his answer on this point is less than satisfying.) Wiliam says that states are primarily political obstacles that practitioners should navigate or work around. It struck me as a cynical response, and one that misses the central role policymakers can and should play in creating better schools.

Indeed, variation across states—including in instructional practices—is caused by what state policymakers do (or don’t do). Testing and accountability may have fallen from grace, but states have other levers they can pull—under-recognized ones like evidence-based grantmaking or mundane ones like licensing and accreditation—to engender the improvements that Wiliam describes. Rather than viewing them as a hurdle, states are uniquely positioned to help accelerate and amplify best practices.

Consider Florida as an example. When Jeb Bush became governor twenty years ago, he made education his top priority and implemented a comprehensive reform model that included school choice, annual testing, and grading schools on an A–F scale. Many of those reforms endure to this today, and achievement gaps have been significantly narrowed. “First do no harm” is a fine adage for state officials, but as states like Florida show, it’s possible to help too.

Getting all of this right is as important as ever with states preparing to pump gobs of money into well-intentioned efforts of questionable cost-benefit. Policymakers are certainly guilty of not getting up on the balcony enough to properly examine the impact of their policies, but asking them to take an auxiliary role misses the significant part they must continue to play in driving system-wide reform.

Building upon the well-known work of Caroline Hoxby and her colleagues, Kalena Cortes and Jane Lincove recently conducted a study that examines the “match” between underserved students and their choice of college. The study exploits Texas’s automatic admissions policy, which granted automatic admission to all public universities in the state to all students who achieved the top 10 percent in class rank as juniors in their high schools (that’s since been changed). This is the sole admissions criteria, although students are still required to complete a college application and take the SAT. (So again, they are admitted regardless of their scores on the SAT, the caliber of their high school, or the courses they took.)

Analysts estimate the differences in the effects of this admissions offer on both low- and high-income students who graduated from public school in 2008 and 2009. Specifically, whether knowing that they are automatically admitted to any public university in the Lone Star State fosters less “undermatch” or more “overmatch”—in other words, when students enroll in institutions where the average SAT score is, respectively, lower than or higher than their own. Analysts also look into whether nonacademic factors play a role in the choice of college, like location and proximity, and whether students prefer colleges with same-race peers, or students from their same high school.

They run a number of sophisticated models that control for numerous variables, but one thing they obviously have to acknowledge is that low-income students are likely more responsive to college costs than are high-income students. So the study essentially compares the effect of perfect admissions information across income groups; they need not assume that students have similar ability to pay. That said, the range of tuition in Texas for public universities is fairly narrow, in part due to the state’s common application system. Analysts are also able to demonstrate that, on average, students do not change their “matching behavior” after they receive financial aid offers.

There are key findings—some of which are fairly nuanced. First, the automatic admissions policy reduced undermatch among low-income, high-achieving students, inducing them to apply to and enroll in more academically challenging state universities. They did not find a similar effect on high-income, high-achieving students, which suggests that the information narrowed the income-related differences in college quality among highly-qualified applicants.  

However (cue second finding), regardless of the automatic admissions offer, high-income students with low SAT scores are more likely to overmatch than their low-income academic peers. So the admissions offer appears to improve the quality of colleges applied to and attended for low-income students only if their observable qualifications (a.k.a. SAT scores) are also well matched. The admissions offer alone was not enough to change their application behavior; they appear to follow the (deterring) signal of their SAT performance, even when those scores do not count for admission.

Third and finally, the results show that most students prefer campuses with students who are like them both demographically and socioeconomically, and who attended the same high school. The exception: Only highly-qualified, low-income students will choose universities where they have fewer same-race and same-income peers.

The bottom line is that while automatic admissions policies help lessen undermatch among low-income high achievers, their SAT scores still sway their behavior, regardless of whether they are used for entry. The irony is not lost: Although there’s a growing list of colleges that say the SAT doesn’t provide useful information about a potential applicant’s likelihood of success, those very same applicants appear to think otherwise about their scores.

SOURCE: Kalena E. Cortes and Jane Arnold Lincove, “Match or Mismatch? Automatic Admissions and College Preferences of Low- and High-Income Students,” Educational Evaluation and Policy Analysis (December 2018).

Elaborate collective bargaining agreements (CBAs) have for decades enshrined the “factory model” into public education. Negotiated by district boards and teachers’ unions, these lengthy contracts dictate numerous aspects of school life—anything from compensation to employee retention to lunch-room duties. In the early part of this decade, some states enacted reforms to CBA laws that limited what could be in these contracts in the hopes of providing greater flexibility to schools.

A new study led by Katharine Strunk, a leading expert on CBAs in the U.S., examines how legislative reforms in Michigan and Washington affected district CBAs. In 2011, Michigan passed sweeping changes to its CBA law that prohibited dozens of topics from negotiation, including things like teacher placement and transfers, classroom observation protocols, and disciplinary and dismissal procedures. Instead of being hamstrung by contract language, district leaders were given the discretion to make these HR decisions. Also around that same time, the Michigan legislature enacted statewide teacher evaluation policies, including the incorporation of student achievement measures. Washington’s reforms, on the other hand, were less extensive: Although legislators enacted a statewide teacher evaluation system in 2012—removing the issue from local bargaining—they did not adopt other reforms that narrowed its scope.

Common sense predicts that these reforms would reduce the restrictiveness of CBAs, thus providing greater flexibility. However, the researchers note that by eliminating some topics of negotiation, others could become more restrictive as they receive more attention at the bargaining table. To gauge changes in districts’ CBAs in Michigan and Washington, researchers examine eighty-eight topics of negotiation, both pre- and post-reform. They create measures for the overall restrictiveness of CBAs and in four subareas: association rights, teacher evaluation, teacher transfers, and leave policies. The CBA changes in Michigan and Washington are then compared to those in California, a state that did not undertake similar reforms. This provides stronger evidence on whether the reforms themselves led to changes in restrictiveness, though the researchers acknowledge that other non-observed factors could still play a role. Hence, they call their analyses “descriptive comparisons” of the pre- and post-reform CBAs in these two states.

Now to the results. Michigan CBAs became much less restrictive overall and in each of the four subareas. Washington CBAs also became less restrictive overall, though less so than in Michigan—not surprising given Washington’s less aggressive reforms. Interestingly, Washington CBAs became less restrictive in two areas (association rights and evaluation) but more restrictive in the others (leave and transfers). This suggests a tradeoff when eliminating topics of negotiation, as unions seem to have concentrated on winning stronger limitations in the areas in which they could still bargain. Due to the major changes in Michigan CBAs, the authors conclude: “It is in Michigan therefore that we might expect to see more substantial impacts on other district outcomes such as teacher staffing and ultimately student achievement.”

Two final thoughts: First, readers should peruse the laundry list of items that districts and unions haggle over in CBAs in the report’s Appendix Table 2. It’s yet another reminder of how much CBAs can constrict school operations and deployment of resources. Second—and this is for state lawmakers, especially those interested in deregulation—this study illustrates that, yes, you can make a difference.

SOURCE: Katharine O. Strunk, et al., Collective Bargaining and State-Level Reforms: Assessing Changes to the Restrictiveness of Collective Bargaining Agreements across Three States, CALDER Working Paper (December 2018).

On this week’s podcast, Kristen Soltis Anderson, pollster and co-founder of Echelon Insights, joins Mike Petrilli and David Griffith to discuss what millennial parents think about education. On the Research Minute, Amber Northern examines how Florida’s tax credit scholarship program has affected participants’ college enrollment and degree attainment.

Amber’s Research Minute

Matthew M. Chingos et al., “The Effects of the Florida Tax Credit Scholarship Program on College Enrollment and Graduation: An Update,” Urban Institute (February 2019).