The Education Gadfly Weekly

Shouldn’t we all seek to individualize instructions to meet each child’s needs? Who could oppose “differentiation”? Well, I do.

Generally, differentiation means modifying instruction or content to meet the unique needs of each child. In that broad sense, all teachers differentiate. I change my analogy depending on a student’s interest and provide more one-to-one instruction during lunch or after school.

But in practice, differentiation has more specific meaning than that definition would suggest. It’s an attempted solution to a persistent difficulty: Children in a single classroom often vary by several grades in their reading and math ability; one child gobbles up early chapter books and another still slowly sounds out words. How can we challenge high-achieving students without leaving struggling students behind?

One approach would encourage teachers to separate children by their current skill level into separate groups or classrooms and teach to each level. Of late, this separation, sometimes called “tracking” or “ability grouping,” has fallen out of favor due to equity concerns, so schools instead throw children all into one room and expect a teacher to modify assignments and activities, sometimes making three or four versions of each, to properly align to each child’s level. This version of differentiation requires a high level of modification from the teacher.

For a number of years, as an English as a second language support teacher, my job was to differentiate. The more I did it, however, the more I saw the approach’s flaws. For one, no matter how sly or secretive I tried to be about modified assessments or activities, kids always knew. Among academically gifted students, there was a sense of unfairness as their peers got modified work. Among those needing support, a sense of embarrassment pervaded. And the justification of “different kids need different things” fell on deaf ears.

Perhaps more fundamentally, teacher time is limited. It’s all well and good to recommend that educators ought to modify activities and assessments. But they also must grade, contact families, go on lunch duty, watch recess, provide feedback, modify their instruction, sequence curriculum, prep the next day’s activities, script discussion questions, and more.

Teachers cannot do all things. As such, we must always ask not if any given instructional practice works, but how it compares to something else that a teacher could be doing with their time. In the case of differentiation, the evidence is lacking. There is plenty of academic literature on the topic—professors talking about the theory and why it ought to work—but comparative studies are few and far between.

One such study from researchers at the University of Connecticut, the University of Virginia, and Yale University found lackluster results. The topline conclusion focused on the difficulty of implementation. Scroll to the bottom, and the authors acknowledge in a small section about the impact on students that “there is no consistency in the few teachers who did make earnest attempts” and “the patterns of student achievement, student attitudes, or student self-concepts.” They could only “speculate” that “differentiation yields change.”

Universal Design for Learning (UDL) is a popular, holistic approach to education that emphasizes differentiating both tasks and assessments for students. A comprehensive review of UDL’s effects concludes that “the impact on educational outcomes has not been demonstrated.”

If anything, international evidence on the effects of differentiation points in the other direction. Greg Ashman, teacher and education researcher, has plotted the prevalence of individualized instruction against PISA scores among various countries. The trend suggests that more individualization actually decreases achievement.

But there’s a more fundamental problem with differentiation: It’s not clear what the term means, and that muddles any debate about it.

Consider a teacher that assigns a chapter of reading to a class about which they must write a structured, two-paragraph response. “Differentiation” in this context could mean extra support like explicit vocabulary instruction and guiding questions, modified tasks such as simplified readings and audio recordings, or altered assessments like allowing the student to record an audio response and providing sentences starters.

These are a hodgepodge of approaches. Some simplify the content, some support the student towards complex content, others modify the tasks, and still others change the final assessment. Similarly, sometimes differentiation means modifying assessments within a mixed-ability classroom, and other times it means separating students into homogeneous ability groups.

Perhaps this unclear nature explains the negligible, observable impacts of differentiation. The overbroad definition includes a mixture of effective and ineffective strategies. We lump all sorts of practices under the umbrella of “differentiation,” thereby designating them as “good” regardless of any evidence.

The continued use of the term “differentiation” then could justify the incorporation of ineffective teaching strategies. Perhaps we ought to toss it out in favor of debating more specific practices, honing in on what works—not just what fits under some broad, feel-good concept.

Last week, the Biden administration released new guidance for how schools should handle discipline for students with disabilities. While much of it simply restates federal requirements and distills conventional wisdom on “best practices,” this initiative could have a negative effect on classroom order and school safety if school leaders take the Biden administration’s core message too seriously—or if the administration enforces its flawed assumptions too vigorously.

A Washington Post headline characterized the guidance thusly: “Biden warns schools not to overpunish students with disabilities.” The Biden admin argues that students with disabilities are “overpunished” in two ways—with disproportionate frequency and greater severity for similar offenses. But neither claim withstands scrutiny.

It is certainly true that, as a group, students with disabilities are punished more frequently. As one of the guidance letters notes, students with disabilities represent “13.2 percent of total student enrollment but received 20.5 percent of one or more in-school suspensions and 24.5 percent of one or more out-of-school suspensions.” But the important question isn’t the aggregate disparity, but whether it actually represents systemic discrimination. In 2019, Penn State’s Paul Morgan and colleagues published a meta-analysis to try to assess this question. When they examined studies that controlled for a variety of student characteristics—including student behavior—they found that five of seven findings “failed to indicate that [students with disabilities] were more likely to be suspended.” Ultimately, the researchers concluded that “Empirical evidence regarding whether U.S. schools discriminate when disciplining [students with disabilities] is currently inconclusive.”

If, as that evidence suggests, the aggregate disparities are not necessarily evidence of discrimination, then pressuring school leaders to close them could lead schools to engage in reverse-discrimination, treating students with disabilities more leniently than general education students.

Indeed, a 2019 national survey of teachers conducted by the Fordham Institute suggests that this is already common practice. When teachers were asked what would happen “if a student with special needs and a general education student commit the same infraction,” 66 percent of teachers said that the consequence would be more severe for the general education student. Only 4 percent said the consequence would be more severe for the special education student. Almost 70 percent of general education teachers, and more than 60 percent of special education teachers, agreed that students with disabilities were “treated too leniently, even when their behavior had nothing to do with their disability.”

Schools are now suffering from a rising tide of behavioral problems in the wake of the pandemic, and teachers are expressing that they don’t feel adequately supported by their administrators. According to a recent poll by the American Federation of Teachers, 88 percent of teachers cited “poor student discipline and a lack of support for dealing with disruptive students” as a very or fairly serious problem. To the extent that school leaders take the Biden administration’s message seriously, it could lead them to be even less supportive of teachers.

How seriously school leaders will take this guidance will depend on how vigorously the Biden administration’s Office for Civil Rights enforces it behind the scenes. The guidance articulates a disparate impact standard for civil rights investigations. That is, school districts could face investigation based on aggregate disparities in discipline between general and special education students.

This announcement alone may lead school districts to prioritize their aggregate statistics over properly handling individual students and maintaining orderly classrooms. This is especially problematic because students can be categorized with an “emotional disturbance” disability for consistently behaving badly. Given that federal law requires school officials to determine whether misbehavior is rooted in a students’ disability, and that such distinctions are especially arbitrary and subjective for emotionally disturbed students, schools may feel pressure to make the most poorly behaved students the hardest to discipline in order to avoid a federal civil rights investigation.

And when the investigations come, the big question will be: What policy changes will the Biden administration push? The key example given in one letter is quite modest, suggesting that a school might have to make an accommodation to a no-swearing policy for a student with Tourette Syndrome. But another letter contains a broader exhortation for schools to implement “responsive practices in place of exclusionary discipline.” When Biden’s Assistant Secretary in the Office for Civil Rights, Catherine Lhamon, served in that role under President Obama, her office leveraged disparate impact investigations based on race to push sweeping changes to school discipline practices—pressuring schools to limit “exclusionary discipline” regardless of the effect on classroom order and school safety. 

If school administrators use this guidance simply as a refresher document, and the Biden administration keeps its civil rights investigations modest and targeted, then this initiative could ultimately prove constructive. But if school administrators take the Biden administration’s central argument too seriously, or if the Biden administration overreaches in its investigations, it may do more harm than good. State superintendents should keep a close eye on all such investigations, and school leaders should steeply discount the allegation that they are “overpunishing” students with disabilities.

School shootings are profoundly tragic—scarring not only the families whose children become victims, but casting a shadow over the lives, mental health, and outcomes of the surviving students. But evidence is also clear that it’s not only horrific mass shootings that can lead a child to miss school. Any feeling of not being safe can prompt children and teenagers to stay home.

Sadly, students have been telling us this for a long time—that their feelings of safety at school are strongly linked to their attendance. And we also know that pervasive safety concerns are driving disengagement from school and chronic absenteeism. Indeed, we’ve been asking students for more than a decade about how they view their school environments, and their responses about school safety have been disturbingly consistent.

We at the Thomas B. Fordham Institute recently conducted a study in collaboration with Jing Liu, an education policy professor with the University of Maryland, to isolate and measure a school’s impact on attendance. Analyzing sixteen years of California data, he found a school’s effect on attendance was positively correlated with students’ perceptions of its climate, and most powerfully with their belief that schools are safe, adults are trustworthy, and behavioral expectations are clear.

These findings contribute to a broader consensus in the literature that perceptions of school climate and absences are linked. For instance, one study that surveyed nearly 600 public high school students in the South showed that 25 percent felt unsafe and nearly 15 percent had avoided school in the last month because of it. Yet the same study found that positive perceptions of the school environment (not only physical safety, but also clear rules, a sense of belonging, and positive parent-teacher relations) were associated with lower rates of absenteeism. Other research has similarly identified connections between students’ perceptions of school climate and absence rates.

More specifically, we know that students’ fears about safety in school often drive chronic absenteeism. In one study, the University of South Florida’s Amber Brundage surveyed chronically absent sixth through twelfth graders across eight states. When asked why they had missed so much school, students consistently pointed to fears around safety. Another study using data from the National Youth Risk Behavior Survey found that among the 5 percent of students who had avoided school due to safety concerns, their absences were often strongly related to fears of being threatened with a weapon, having one’s property damaged or stolen, and being bullied or cyberbullied at school. A third study indicated the link between safety concerns and absence was particularly strong among Black, Hispanic, and LGBT students.

Other studies comparing various strategies to combat absenteeism underscore the importance of school safety. For instance, experts who authored a summary of best practices published by EdResearch for Recovery emphasized that a focus on safety and school climate is among the most effective interventions. Additionally, a meta-analysis evaluating multiple evidence-based school attendance strategies found that student behavioral interventions were somewhat more effective. These included a wide array of interventions, ranging from peer and adult mentoring and case management services to school resource officers and physical security measures.

So, yes, let’s have and act on those impassioned conversations about how to end the travesty of mass shootings. But as a new school year begins and we rethink how to better protect our students, let’s ensure all schools are places where children feel safe and behavioral expectations are clear.

Editor’s note: This was first published by the San Antonio Express-News.

The career services office is a necessary stop on any good college campus tour, as it offers prospective students a sneak peek at all the help the staff within can provide—resume writing, mentors in many different employment fields, interview prep, job fairs, and much more. Those services are especially important for community colleges, as they are key players in current efforts to boost postsecondary attainment and grow the workforce in many specific industries. A new report from the RAND Corporation throws open the office doors to examine how current career services and industry partnerships—as well as state policy—facilitate student employment.

Data come from remote video interviews conducted between May 2020 and November 2021 with 134 individuals working at or associated with sixteen different community colleges in California, Texas, and Ohio. These states were chosen because they cover a variety of industry, policy, and employment contexts and offer a wide range of settings and sizes of community colleges. One of the institutions, Texas State Technical College (TSTC), focuses nearly exclusively on workforce development education and, thanks to an innovative state policy, receives all of its public funding based on the earnings of its graduates. That makes it a valuable comparison point for the others, which focus simultaneously on workforce outcomes and four-year college transfer readiness. Interviewees included presidents, provosts, directors and staff of career services and student affairs, deans, faculty, navigators and counselors of workforce development education programs, employers, regional coordinators, and state policymakers.

It is important to note that no outcome data are included or analyzed as part of this report. While TSTC’s efficacy in training students for and placing them in jobs is key to its existence, none of the other institutions studied had such information readily available. Additionally, data on students’ use of services were very limited and thus are also not part of this report. As a result, any claims of “success” or “quality” are based almost entirely on inputs—state policy landscape, services offered, business partnerships created, and the like.

For example, the report states that “student use of career services appears generally to be low.” This is a negative, of course, but we have no numbers to analyze. Worse, the only clues in the report as to why this might be are that most schools do not heavily market their career services to students and that most schools opt for a centralized model of service provision that does not focus on supports for specific career fields. (More on that below.) But even if less-than-exemplary practices were to be improved at any of these schools, there is still no guarantee that students would take advantage of them.

Similarly, the researchers find that all three states have numerous policies and incentives in place to encourage a greater linkage between community colleges and the many industries for which they are increasingly training students, but none of the states have any actual requirements for schools to provide services or confirmation that any services were provided.

This is where the RAND report can be helpful. The interview subjects were quite willing to discuss the systems their institutions have put in place. The most common services provided include career exploration for students trying to home in on a field of interest, preparing students to search for and apply for jobs, and matching students with specific job openings. Service models generally took one of two forms: centralized, which means a single office/similar set of services for the entire school, or decentralized, which embeds some or all career services into separate and specific academic areas. The researchers considered the latter to be a stronger model, allowing services to be targeted to specific majors and career fields, especially when career exploration was made part of the academic track in the form of mandatory, graded courses.

In terms of workforce development offerings, work-based learning was widespread among the colleges, including practical, industry-related class projects; job-shadowing opportunities; and facilities and labs that mimic the workplace. However, apprenticeships—actual work in the field—were much less prevalent due to the stronger commitments required by both employers and career services staff to make these happen. In fact, most colleges in the study relied upon advisory committees for creating partnerships with local industry players. Unfortunately, committee members interviewed explained that the terms of membership and the infrequency of meetings limited most members’ ability to build partnerships in a meaningful way. A few colleges in the sample did stand out strongly in this area, structuring advisory committees to ensure employers’ continued involvement, establishing industry partnership offices in the community, partnering with other colleges, and sharing their institutional resources with employers to help build and maintain connections.

Once again, the researchers note a lack of clear and consistent measures for partnership accountability and performance assessment and little in the way of established infrastructure or protocols for data reporting, monitoring, and sharing. Added to all of these limitations is an expressed competition for resources and attention between career-focused majors and four-year-college transfer preparation, with the latter generally seen by interviewees as more favored.

The majority of recommendations emerging from this analysis involve more: more personnel, more industry partnerships, more internship opportunities, more technology, more outreach to students, etc. And it all boils down to more money. But first and foremost, data systems are needed to track who is coming through the office door, how they are served, and what outcomes are achieved.

SOURCE: Rita T. Karam, Charles A. Goldman, and Monica Rico, “Career Services and College-Employer Partnership Practices in Community Colleges,” RAND Corporation (June 2022).

Are charter schools helping students succeed? Many studies have contributed to answering this multidimensional question, including a recent Fordham Institute report highlighting the association between a rise in charter student enrollment share at the metro level and increases in math achievement. A recent set of city-level reports from Stanford’s Center for Research on Education Outcomes (CREDO) provides us with a new body of empirical evidence on student achievement in charter schools compared to traditional public schools (TPS). These CREDO reports are part of an ongoing series, the City Studies Project, tracking student performance in select cities’ public K–12 schools around the country.

CREDO releases these city assessments on a rolling basis, but a May release of three—Denver, Colorado; Memphis, Tennessee; and Baton Rouge, Louisiana—provides yet more evidence that urban charter schools boost the achievement of their students, and also includes scintillating findings about magnets, innovation schools, and other school models.

In each case, it uses student-level data from 2016 through 2019 to compare the progress of students attending charter schools with TPS, along with other schools of choice, further broken down along the lines of sex, race, poverty status, English language learner (ELL) status, and special education status. The researchers first compare each city’s student performance to state averages, and then compare the different organizational models within each city.

In Denver, the researchers found students posted stronger learning gains compared to state averages in both reading and math during the 2017–18 and 2018–19 school years. In 2018–19, Denver district schools (TPS) delivered 23 extra days of reading and 17 of math, while Denver charter schools delivered 54 extra days of reading and 60 of math. Innovation schools, district schools in an “innovation zone” that get extra flexibility from state standards, delivered 66 extra days of reading and 56 days of math.

Broken down by demographic characteristics, female students outperformed male students in every year and organizational category. Denver’s charter school Black student performance versus district school Black performance stands out, with 58 and 71 days’ growth in reading and math for charter schools, respectively, compared to 12 and 16 days of loss for district schools. Charters also posted stronger growth in learning for Hispanic and low-SES students, though Denver innovation schools stood out for their exceptional ELL student reading gains. Denver charters also delivered stronger results than TPS for special education students in reading, perhaps challenging the public perception that charter schools somehow “leave behind” special ed public school students, or boost their performance via these students’ exclusion.

Finally, CREDO presents some evidence that Denver charter schools managed by “CMOs,” or charter management organizations, see somewhat greater success in math than independently-run charter schools.

Moving on to Memphis, the CREDO researchers found students exhibited similar growth in 2017–18 and small losses in ’18–19 compared to state averages in reading, while holding pace with state averages in math for both years. It may be worth noting that, in Denver, average household income in 2018 was $64,000, close to state averages of $69,000; while in Memphis, average household income in 2018 was $39,000, considerably below state averages of $51,000. We should also mention that Memphis has an “Achievement School District” (ASD), designed to take schools in the bottom 5 percent of statewide test performance and, ambitiously, turn them around to reach the top 25 percent statewide. Memphis also uses “Optional Schools,” which screen students for academic performance, somewhat like exam schools.

In Memphis, in 2017–18 and ’18–19, these Optional Schools delivered the strongest returns, far exceeding statewide averages. District and charter schools clustered around the state average, with charters slightly outperforming TPS—though the difference wasn't statistically significant. Schools in the ASD underperformed state averages. The city’s Black students slightly underperformed state averages, though those in Optional Schools posted strong gains. The ELL students in the Optional Schools, however, saw strong gains in reading but significant losses in math, perhaps exposing an underlying flaw in Memphis’s ELL approach. And female students underperformed male peers to a greater degree than state averages.

Baton Rouge, finally, breaks down schools in TPS, charters, and magnets. On reading and math performance, magnet schools dominate, with around 100 more days of learning compared to state averages. Charters and districts return more moderate gains, with charters consistently outperforming TPS—though, like in Memphis, the difference wasn't statistically significant. CMO-managed charters moderately but significantly outperform independently-administered charters. Across Baton Rouge, female students outperform males, and Black students slightly outperform state averages. Hispanic students, however, do well in magnet and charter schools, but actually post losses in 2017–18 and ’18–19 in both reading and math for district schools.

So, what’s driving the differential performance of these school models? Though the City Studies Project does not draw many conclusions beyond presenting the data, it appears that within each city there is sorting of the student population into the most-demanding schools. In Denver, that seems to be charter schools, while in Memphis it’s the academic-screening Optional Schools, and in Baton Rouge it’s the in-demand magnet schools.

However, we should caution that this does not mean charter schools or other schools of choice take away from or diminish traditional/district public school performance. There’s a growing body of evidence that competitive effects between TPS schools and charter schools induce a rising tide that lifts all ships. And as the strong performance of special education students in Denver suggested, this sorting effect does not mean that charter schools boost their numbers by excluding special needs students.

SOURCE: “City Studies,” Center for Research on Education Outcomes, Stanford University, retrieved July 26, 2022.

Correction, August 15, 2022: This review previously gave the impression that charter schools in Memphis and Baton Rouge produced larger academic gains than traditional public schools. These differences in the studies, however, were small and not statistically significant. It also indicated that Denver charter schools outperformed the city’s traditional public schools on academic gains for English language learners, and for special education students in math; these differences, too, were not statistically significant. The review has been corrected to reflect these issues.

On this week’s Education Gadfly Show Podcast, Lindsay Dworkin and Karyn Lewis, senior vice president of policy and advocacy and director of the center for school and student progress at NWEA, respectively, join Mike Petrilli to discuss Karyn’s co-authored research brief on student achievement following Covid-19. Then, on the Research Minute, Amber Northern reviews a survey of district administrators on teacher shortages, student and staff mental health, and pandemic-connected learning loss. 

Recommended content:

• Meghan Kuhfeld and Karyn Lewis’s Collaborative for Student Growth brief, “Student achievement in 2021-22: Cause for hope and continued urgency,” July 2022.
• Mike Goldstein’s podcast feature on learning losses from Covid-19, “Education Gadfly Show #825: Learning loss may get worse before it gets better.”
• Thomas Kane’s learning recovery tracker, “Equitable Recovery: Addressing Learning Challenges after COVID,” May 2022.
• Melissa Kay Diliberti and Heather L. Schwartz, “Districts Continue to Struggle with Staffing, Political Polarization, and Unfinished Instruction,” RAND Corporation (2022).

Feedback Welcome!

Have ideas on our podcast? Send them to our podcast producers Pedro Enamorado and Lilly Sibel at [email protected] and [email protected]

Cheers

• The end of affirmative action in college admissions could drive improvements in K–12 education. —Washington Post
• Three charter schools share their strategies for Covid-19 prevention. —Centers for Disease Control and Prevention
• Louisiana is raising the bar for high school ratings, angering some superintendents. —The Advocate
• Six scholar-athletes from KIPP Life Academy in Newark, New Jersey, make it to the National Junior Olympics, despite training in the hallways because of their lack of proper facilities. —Tap into Newark
• Abolishing homework won’t eliminate inequality, but it will lower expectations for disadvantaged students. —Jay Caspian Kang

Jeers

• Governor Phil Murphy is blocking the expansion of New Jersey’s charter schools for political reasons. —New Jersey Star-Ledger

• The publisher of Lucy Calkins’s troubled reading curriculum has halted the release of an updated version over concerns about conservative states’ new curriculum laws. —New York Times
• Children’s needs were sacrificed more than any other group’s during Covid-19 policymaking decisions. —Alyssa Rosenberg
• By focusing on the culture wars, Democrats are losing ground on education. —Conor Friedersdorf
• Twenty-nine states have passed laws to bring K–12 education in line with the latest research on reading instruction, but some feel plans for implementation are unclear. —Education Week