The Education Gadfly Weekly

I’ve never led a school, run a school system, or served on a school board. So maybe I am about to ask something that is incredibly naïve and possibly insulting. But here goes: Why are so many of our school districts so complacent? I understand why they don’t always do the hard things, like firing ineffective veteran teachers, or expensive things, like starting one-on-one tutoring programs. But I can’t fathom why so many don’t do the easy, more or less no-cost things, either.

Let me offer two examples:

1. Adopting an aligned, high-quality curriculum. This has never been simpler! (Not faithful and imaginative implementation—that’s hard. But picking a good curriculum? Easy!) Both states and non-profits—especially EdReports—are doing reviews of curricular products, and there is a growing number that are meeting high standards for quality, alignment to state standards, and usability, some of which are free. Yet what limited data we have indicate that most schools continue to choose curricula that are not aligned, not rigorous, not proven—if they are choosing curricula at all. I recently got my hands on a “Market Brief” by Education Week that looked at the market share of various English language arts and math curricula for elementary schools. In both subjects, the market leader is “Other”—as in a large plurality of districts is using something “other” than the major commercial products or open education resources. It’s anyone’s guess what these materials are, but other surveys hint that they are probably lessons downloaded from Pinterest and other sites.

Meanwhile, the products that districts are purchasing are generally not those that score well on various rating systems. HMH’s “Journeys” is the top choice for ELA classrooms, with 19 percent market share, even though it only “partially met” EdReports’ very first screen. McGraw-Hill’s “Reading Wonders” is next, with 5 percent, and then Pearson’s “Reading Street,” with 4; the former received just middling marks from EdReports, and the latter scored terribly. On the other hand, Pearson’s “ReadyGEN Text Collection” and Amplify’s “Core Knowledge Language Arts” both received top ratings from EdReports for fully meeting its criteria for content and usability, yet each has less than 1 percent market share.

It’s not much better in math. HMH’s “Go Math” is in the lead with 16 percent; Pearson’s “enVisionmath 2.0” and McGraw-Hill’s “Everyday Math” come in at 8 and 7 percent respectively. None of these materials fully met EdReports’ criteria for alignment to college and career ready standards; enVision and Everyday Math score particularly poorly. But now for some good news: Eureka Math, born as EngageNY Math, has 9 percent of the market, plus rave reviews from EdReports and discerning states such as Louisiana. And the consumers of the product love it too; 82 percent of district administrators surveyed by Education Week who are frequent users of the program said Eureka Math helps improve student achievement. And 77 percent gave Eureka Math positive marks for improving instruction; the next best was 29 percent.

It’s a truism that there’s not “one best curriculum” that fits all schools’ needs. But it’s also the case that some curricula are simply better than others. According to the Education Week data, the good stuff is being used by maybe 10 to 15 percent of schools. Why on earth is this number so low?

2. Making the tenure approval process more rigorous. Teacher evaluation reform may have crashed and burned, but that’s not because the impulse was wrong. Research continues to show that teacher effectiveness varies dramatically from one classroom to the next; our lowest-performing teachers have a hugely negative impact on their students’ trajectories. Identifying these teachers at the get-go and getting them out of our schools could do a world of good. As would recognizing and rewarding our best instructors.

But it remains super hard to do that once teachers have earned tenure (a.k.a. a “permanent contract”), thanks to states’ tenure and due-process laws, as well as local collective-bargaining agreements. What’s not hard is denying someone tenure in the first place. Given that most teachers have to wait three or four years before they are eligible for tenure, and that three or four years is plenty of time to determine whether someone is an effective instructor, districts have a golden opportunity to ensure quality control in the classroom. They can make the tenure-approval process rigorous instead of a rubber stamp. Even better: They could make it a true honor, complete with a ceremony welcoming newly tenured teachers into the most important profession in the world.

Joel Klein made this reform in New York City, and the Big Apple’s rate for approving tenure-eligible teachers went from close to 100 percent to around 50 percent, practically overnight. It has crept back up under de Blasio, but it’s still much lower than the near-universal rates we see in most districts around the country. Nor did Klein have to actually fire unproven teachers; he told most of them to try again the next year, though many decided to leave the classroom instead.

So districts have a choice: They can make tenure an achievement or automatic. Why would anyone choose the latter?

***

I suppose there are two ways to look at examples like these. The hard-edged view is that our public-school system still isn’t responsive enough to the need for fundamental improvements in student learning, despite test-based accountability, choice-driven competition, and everything else we’ve tried. Maybe it really is time to replace it. The other, more optimistic take is that if and when schools start to pick this low hanging fruit, we could see student learning accelerate big time.

Let me place a wager: If we could get the market share of high quality curriculum above 50 percent, and the percentage of junior teachers approved for tenure on their first try below 50 percent, we would see the most dramatic improvement in student achievement since A Nation at Risk—or maybe ever. Is there any reason, other than complacency, not to try?

Within the reform community, a tension that has been brewing under the surface for years is now playing out in full view. It pits those who believe that parent empowerment (i.e., the power to vote with one’s feet) equals accountability versus those who subscribe to the “grand bargain” of accountability for results. Those in the first group are generally aligned with the current administration’s tack, while the latter have argued that a refortification is in order. The choice isn’t binary, of course, but these two camps are illustrative of the larger disagreements at play.

I attended a workshop a couple of months ago that endeavored to bridge the divide. Hosted by EdChoice, the event brought together a diverse group of stakeholders, including researchers, policymakers, practitioners, and philanthropy. There were folks in attendance who were fans of the Bush-Obama era, as well as those who felt the reforms implemented under their tenure were a detriment to students and the system writ large.

Participants wrestled with a number of key questions, such as: What is the purpose of an accountability system, and who should be held responsible for owning and operationalizing it? Though most folks felt that recent history has been too top-down in this regard, finding the sweet spot between local autonomy and technocratic overreach proved elusive—at least for the limited amount of time we had together as a group.

Complicating matters further is that the A-word has become a dirty one now synonymous with standardized testing. Certainly accountability means so much more, but it’s reasonable to argue that the well has been poisoned at this point. “Transparency” and “responsibility” came up as potential alternatives, though how to measure either would also require extended discussion.

There’s also a debate about what test scores tell us about school performance. To be candid, my thinking on the subject has evolved over the years. I still believe in the relationship between short- and long-term outcomes, but I’m more open-minded now to schools that deliver positive life outcomes without the stellar test scores to match. As the father of a young daughter, I’m also more appreciative of the intangible elements that make school and learning special, but may not lend themselves easily to hard measurement.

That said, I see no reason to ditch test-based accountability anytime soon. As a self-professed accountability hawk, I still firmly believe that the problem with standardized testing is less about the complaints (e.g., they’re biased against some students or too easily gamed) and more about the lack of outrage at the number of schools—especially those serving low-income communities—that are not teaching the basic reading and math skills necessary to pass them.

Still, there seems to be a lot more questions than answers on the issue. It’s all enough to make an honest policy wonk nervous (our dinner venue for the evening was an apt one) that even if we were able to build the ideal accountability system, we’d still have Gus the Truck (i.e., there’s a ceiling to what it can accomplish on its own). But this doesn’t mean abandoning accountability. Instead, we need an evolution that moves us toward a focus on improving long-term outcomes and ensures that educators have a seat at the table because they’re the ones who wrestle with the implications of these systems on a daily basis.

With the Every Student Succeeds Act, some states are beginning down this road by weaving college and career readiness or world languages into their accountability systems. Most of these indicators are focused on high schools, though I’d like to eventually see more in the early grades. For sure, there will be questions to be answered (e.g., What’s age appropriate when it comes to career readiness at the elementary level? How do we deal with exposing kids early on, but recognizing that it will be years before we see whether the effort was successful?), but my hope is that states will see these as new opportunities to be innovative.

Encouraging students to grow up to form strong, intact families is another example of keeping long-term outcomes in mind (and not one without controversy). A few months ago, the study released by Raj Chetty et al. created quite the buzz by shining a light on what many of us instinctually already knew about the punishing reach of racism. Buried in the data—behind the shiny New York Times infographics—was a fascinating finding about the power of a father’s presence: Black boys do especially well in neighborhoods with a large fraction of fathers at home. What implications might this finding have for schools and for accountability systems?

Students spend way more time at home than they do in school, and the likelihood of this imbalance changing anytime soon are slim to none. The perennial question is how to maximize the time schools do have to effectively mitigate against some of these larger intractable issues. Try this one on for size: Should educators teach the success sequence, and should accountability systems be employed to nudge schools to do so? Now there’s an idea that’s sure to make some people nervous.

Imagine a classroom environment designed to build students’ background knowledge and vocabulary (in other words, content-rich), with high standards for every child, and structured to meet the needs of students when, where, and how they need it most. Now imagine that this classroom is also using the power of technology to help differentiate learning and bring content to life.

High-quality curricula united with personalized learning*: This is a vision we have presented before. But, as in any marriage, compromise and flexibility are essential.

Thus, as we begin to test and operationalize the vision, we seek to find out: How can we create a new approach to unlock something even more powerful and still maintain what makes each strategy successful? 

Below we explore this question in two areas: time and pace, where we have some idea of what a content-rich approach and personalization fusion might look like; and data and assessments, where the idea is less clear.

Time and pace

A high quality curriculum is one of the most cost-effective ways of improving achievement and increasing equity, partly because it demands synchronization across a clear scope and sequence. When units and lessons are organized around topics and designed to provide students with foundational background knowledge, students have what they need to comprehend future areas of study.

Synchronization also allows for whole-class discussion related to a particular text. Student discussion has proven benefits; as students hear their peers reinforcing, challenging, and adding different perspectives, especially when using a shared text (rather than general strategies applied to separate texts), their learning deepens.

Personalized learning, on the other hand, prioritizes flexibility of time and pace, with students often engaging with different content and progressing at variable rates. Customization based on needs and interests of the individual student is key to accelerating learning in this approach. In a classroom of twenty students, it’s possible that twenty different learning experiences could be taking place at one time, each designed to deliver the right content and rigor to help a student progress toward his or her individual goal.

That’s not to say that a flexible classroom could not enhance the benefits of discussion, especially if strategic, multi-level student groups or differentiated content are designed with the goal of increasing rigor. The risk, however, is that students may be organized around ability, which research has shown to be less impactful and could inadvertently result in tracking students.

Despite these varying approaches, we have seen some schools try to merge the two strategies by segmenting out personalized learning time as a separate class that compliments the core curricular learning. We have also seen the use of multi-week and self-contained deep-dives into a topic where students work both independently and collaboratively. Other models have students cycle through a variety of content-specific learning stations on a fixed schedule, with one or more stations leveraging technology so students can engage with content based on their interests, knowledge base, and individual goals.

These more integrated approaches are rare, but they are the kind of models we are eager to explore, pilot and learn from to see if a true hybrid would make for even greater student gains than we have seen from either to date.

Data and assessment

In a personalized learning environment, technology makes it possible to use student-level data to deliver students the learning experiences that are just right for them, at that moment and in the future.

Although a rigorous, content-rich literacy curriculum is not in opposition with the use of real-time formative data, we must acknowledge that any data-collection tool, formative or summative, risks defaulting to a skills-focus. Why? Because assessing comprehension is hard and often leads to mistaken diagnoses.

If, for example, a third grader fails to satisfactorily summarize a passage or correctly select its main idea on a multiple-choice question, the inclination—whether in a formative or summative context and graded by a human or machine—is to conclude that the student has not mastered Grade 3 Common Core ELA standard RI.3.2: “Determine the main idea of a text; recount the key details and explain how they support the main idea.”

As Timothy Shanahan has commented, it is quite possible that this student indeed mastered the skill but encountered any number of problems that prevented her from properly answering the question. Perhaps she did not know the definitions of key words. Maybe she recognized the vocabulary but did not have the background knowledge to correctly make meaning out of the sequence of words. It is also possible she did not have the attention required to read the long passage. Or she was so excited for lunch that she rushed through her answer. If a computerized assessment misdiagnosed this problem and had a student practice finding the main idea for a week instead of addressing the actual cause of the problem (a vocabulary deficit, for example), the student would make less, not more, progress than if she engaged with the standard curriculum.

A tool that accurately pinpoints the cause of the student’s error—and in doing so, correctly diagnoses gaps in student learning to inform instruction—would be difficult to create but would have far-reaching benefits that would go beyond just student learning to teacher training, curricular rigor, and more. Until we can fill that gap, we can learn from programs that leverage teacher observations of small groups or use technology to assess across several dimensions to accommodate the nuance and complexity of learning to read and reading to learn.

Realizing the vision

Traditionally two separate camps, content-rich advocates and personalized learning enthusiasts are starting to come together to see if the sum can be greater than the parts. This work is hard, but the time is now, as Common Core and other similar college- and career-ready standards have brought the power of content to the forefront, and technology is ever more present and in-demand by students, teachers, administrators, and families.

Let’s figure out together how to leverage this opportunity and support all students in achieving the high expectations we have for them. Help us by sharing your questions, experiences, learnings, visions, and concerns. 

__________________

* In this piece, when we reference “personalized learning,” we suggest a strategy that uses technology intentionally, in combination with traditional, teacher-led instruction, to tailor learning experiences to students’ individual needs through some flexibility of the time, place, and pace of learning. Ideally, student use of technology enables distinct practices and learning experiences that would not be feasible without it.

On this week's podcast, Ben Castleman, an assistant professor at the University of Virginia, and Ethan Fletcher, a managing director at ideas42, join Mike Petrilli and David Griffith to discuss Ben and Ethan’s collaborative project to improve college access and completion, Nudges, Norms, and New Solutions. On the Research Minute, Amber Northern looks at nudges, too, in this case the role of information and incentives in getting students to fill out their FAFSA forms.

Amber’s Research Minute

Oded Gurantz, “A Little Can Go a Long Way: The Impact of Advertising Services on Program Take-Up,” Educational Evaluation and Policy Analysis (May 2018).

Over two years ago, we at Fordham published one of the first studies that examined the potential long-term benefits of career and technical education in high school. Conducted by Shaun Dougherty, the study found that students who “concentrate” their CTE coursework in one area experienced multiple benefits, such as a higher likelihood of graduating and higher wages compared to non-concentrators.

A new study by Daniel Kreisman and Kevin Stange found virtue in a related path—taking more specialized or upper level CTE courses, versus an assortment of introductory courses, as non-concentrators tend to do. The analysts used detailed longitudinal transcript and labor market information for a cohort of respondents in the National Longitudinal Survey ‘97 (NSLY97) to glean the benefits of CTE courses on longer-term outcomes. The NSLY includes a rich set of measures that can be used as individual controls for student background and ability, as well as for location and cohort effects.

The dataset includes about 9,000 individuals who were between the ages of twelve and eighteen when they were first interviewed in 1997. The survey is representative of all American youth at that time period and respondents had been followed annually to glean information on their educational attainment, labor market experience, and family formation, among other areas. High school transcripts were also collected from respondents' high schools; in total, transcript data were collected for over 6,000 respondents. Courses were coded to identify “low” and “high” level: Low-level courses are those classified as the “first course” on transcripts, while upper-level courses include those beyond the introductory level, such as “second or later courses,” “specialty courses” or “co-op/work experience.” The wage analysis was restricted to a “wage sample” of 3,708 students whom analysts determined had entered the labor market and had a valid wage record.

Kreisman and Stange found that more vocational courses were associated with higher wages, on the order of 1.8–2.0 percent for each year of specialized/upper level vocational coursework. In fact, when separating vocational coursework into higher and lower levels, they found that wage gains were driven entirely by upper-level courses, largely in technical fields and among non-college graduates. There were no discernible wage gains when it came to taking an additional introductory level CTE course.

Digging into various CTE strands, Kreisman and Stange determined that wage gains were driven by Transportation & Industry (including construction trades, mechanics and repair, transportation, and production), Business & Management (including business management, services, and marketing), and Health Care. Next, they found little to no evidence that CTE coursework decreased the likelihood of college graduation, nor that the monetary value of the courses are explained away by other factors. Specifically, they wrote, “while wage gains associated with non-vocational courses (core and electives) are entirely explained by college enrollment, wage gains from upper level vocational courses are unaffected by controlling for college enrollment and completion, suggesting that these courses do in fact have real value in the labor market.” That’s terrific news. Finally, through a series of “alternative hypothesis testing,” they found—to the relief of all who rightly shun the “tracking” history of old-school voc-ed—that students appear to positively sort themselves into CTE courses, as opposed to adults funneling low-ability students into them.

The bottom line is analogous to that of our seminal 2016 study, which is that the benefits of CTE coursework accrue to students who specialize, rather than to those who take a smattering of courses (typically introductory) in multiple areas. That means that CTE programs need to be built in a way that allows students to go into depth on any topic that is offered to them. Because, as the title of this report indicates, there’s greater value in depth than in breadth.

SOURCE: Daniel Kreisman and Kevin Stange, “Vocational and Career Tech Education in American High Schools: The Value of Depth Over Breadth,” Education Finance and Policy (June 2018).