The Education Gadfly Weekly

Recent stories have cast doubt on the stratospheric graduation rates reported in myriad states, including accounts in Alabama, California, Georgia, Florida, Maryland, Minnesota, New Jersey, Texas—and of course Washington, D.C., where one-third of recently awarded diplomas are reportedly attributable to educators violating district policies related to pupil absences and credit recovery.

These are just the scandals we know about. “This is sad and infuriating and, as local education reporters across the country know, not at all uncommon,” tweeted Erica L. Green, an education reporter at the New York Times, when the D.C. scandal broke. Green used to cover education for the Baltimore Sun. And what we need today is for more folks on more school beats to investigate whether similar malfeasance is occurring in their districts and states—because such behavior is almost certainly more widespread than has yet seen the light of day.

If it is, we must identify causes and propose solutions. The issue is not measurement and accountability writ large, as our friends Lindsey Burke and Max Eden proposed recently. People cheat on Wall Street too, but that doesn’t mean companies should stop reporting quarterly earnings or that the SEC should stop checking on them.

Instead, the most obvious culprits are utopian graduation rate targets that have pushed some educators to do the unthinkable. That is, the problem isn’t reporting graduation rates and using them as part of an accountability system; it’s setting goals for them that can’t possibly be achieved. We should set more realistic targets that take into account the achievement level of students when they enter ninth grade.

But there’s likely also a more important and foundational cause: Education policy’s utter confusion and division about the appropriate purpose of high school has led to the default, aspirational view is that it’s meant to prepare every child for college. That view is mistaken, however, in part because it’s at odds with students’ interests and the skills necessary for most American jobs. If this is the cause, then the reasons for America’s graduation rate malfeasance is systemic and an overhaul is in order.

In D.C., a recent audit and investigation of the district’s scandal concluded that “Teachers and school leaders are subject to a variety of institutional and administrative pressures which have contributed to a culture in which passing and graduating students is expected.” These pressures included, among other things, aggressive graduation goals and empathy for the needs of the city’s many disadvantaged students, who tend to enter high school at lower levels of achievement than their more advantaged peers.

In other words, the auditors found two major forces that led D.C. teachers and principals to fudge their figures: incentives to hand out more diplomas and concern about the disadvantages faced by low-income and minority students and their effect on academic achievement.

Those forces are have been ubiquitous across America for at least a decade. In 2008, new rules from the U.S. Department of Education sought to fix the country’s diploma problem. On documents that states submitted to the agency for approval, each thereafter had to set a single long-term graduation rate goal for all its high schools, as well as annual goals that ensure progress toward that target. States also had to include on state and local report cards aggregate graduation rates and rates for subgroups.

The Every Student Succeeds Act of 2015 ratified these requirements in statute. States must hold their schools accountable for academic outcomes. For high schools, graduation rates are a mandatory metric for this purpose, along with English language learner proficiency and math, reading, and science achievement. States must also set ambitious long-term goals and publicly report their results.

School officials therefore remain under significant, even heightened, pressure to hand out more diplomas to more students, which can be achieved through several means. One is to boost students’ actual performance so that more of them meet rigorous graduation standards. Another is to lower or alter standards. Yet another is simply to ignore standards altogether and confer diplomas on young people who haven’t earned them.

We haven’t seen a whole lot of the first. As my colleague Robert Pondiscio aptly observed in 2016 when he called graduation rates “the phoniest statistic in education,” the then-record-high rate of 82 percent (today it’s 84 percent!) seemed distressingly divorced from increases in achievement: “There has been no equally dramatic spike in SAT scores. Don’t look for a parallel uptick on seventeen-year-old NAEP, better performance on AP tests, or the ACT, either. You won’t find it.”

Yet policymakers are falling all over each other to change, and often lower, standards. In Fordham’s home state of Ohio, for instance, officials have backed off plans to require that graduates pass rigorous end-of-course exams. They worried that too few students would meet those expectations, so they developed various workarounds for the class of 2018. Students can earn a diploma by, for example, attending school regularly and completing a sufficient amount of community service. Now the state board of education wants to extend those same lower expectations through at least 2020.

Even worse are schools that simply ignore graduation standards or fudge their data. In addition to the 937 students to whom D.C. officials improperly granted diplomas in 2017, a recent audit undertaken in neighboring Prince George's County, Maryland, found that as many as a quarter of that district's 2016 and 2017 high school graduates may not have met requirements. A year earlier, five veteran educators in El Paso were indicted on federal charges in connection to, as an FBI agent said, “criminal conduct and brazen efforts to manipulate testing populations, graduation rates, and attendance figures”; three have since pleaded guilty to related charges and agreed to cooperate in the trials of the other two.

All of this is exacerbated by America’s lackluster achievement among students entering high school. Nationally, almost one-third of eighth graders are below the basic level of achievement in math on the National Assessment of Education Progress, as of 2016. And basic is a low bar; proficient is the one they’re supposed to clear. Seventy-three percent of students fall short of basic in Detroit; 60 percent in Cleveland; 59 percent in Baltimore; 49 percent in Atlanta and Washington, D.C.; 48 percent in Los Angeles; and 47 percent in Philadelphia. Similarly low rates exist for disadvantaged eighth graders nationwide: 42 percent of those eligible for free or reduced priced lunch are below basic in math, and that number is 52 percent and 40 percent for black and Hispanic students, respectively. The trends in reading are comparable.

So it’s scarcely surprising that states, districts, and individual schools—faced with strong pressure to graduate more students, yet awash in entering freshman who can scarcely read and cipher—would finagle their diploma metrics. That isn’t to excuse that behavior, but to understand it. For only when those of us in education policy and practice understand it can we widely recognize our refusal to address the aforementioned real issue: The current implied purpose of American high school—getting more students to and through college—is mistaken and is causing more harm and havoc than harmony and good.

But this also requires that we understand the true magnitude of America’s graduation rate inflation. Our current, hugely problematic approach to this metric is baked into ESSA and the fifty state plans the law obligated, so lawmakers won’t overhaul high schools without overwhelming evidence that today’s approach is inherently flawed. Only then will policymakers and parents be able to decide what kinds of secondary schools today’s adolescents should be able to attend—and what that means for holding such schools accountable and for what their students must accomplish to earn diplomas, and not merely be handed them. Local reporters can help. By all indications, the consequences of these innate defects are bound to be widespread and severe.

Chris Yaluma’s and my recent Fordham report on gifted education in high-poverty schools shows that the U.S. still has a long ways to go before it closes the “gifted gap,” the disparity in participation in gifted programs among student groups. Even in the earliest grades, black and Hispanic students participate in these programs at much lower rates than their white and Asian peers. But what is the rationale for gifted education in the first place? For those of us who are concerned about persistent inequities in American society and in our schools, “gifted education,” which through its name (somewhat offensively) implies that God or nature has “gifted” a special few, requires a strong justification.

While I would welcome a name change, I believe strongly in gifted education for one main reason: Kids in the same grade are not all at the same level for each subject. This may be intuitive; every child is a snowflake! But the differences within each grade are greater than you might think.

One recent study found a range of more than eleven grade levels in reading fluency and comprehension among fourth graders in a small group of diverse elementary schools. This is extraordinary variation, implying that a fourth grade teacher may have students reading at a kindergarten level and at a ninth grade level in the same classroom. Whatever the merits of “differentiating instruction,” a teacher cannot realistically teach sonnet analysis, spelling basics, and the wide spectrum of skills in between to the same set of students.

But it’s possible that this remarkable statistic is misleading. Perhaps one child prodigy could be what causes the range to be so vast.

To get a more practical sense of the proportion of students who are performing above or below their grade level, my colleague Nicholas Munyan-Penney and I dug up some school-level and district-level studies that included scores for the NWEA Measures of Academic Progress (MAP) assessment. (One study was of two schools, and the other of a district.) The MAP is a good source for this exercise because its scores are scaled in a way that is theoretically consistent across grade levels. In other words, a third grader and a ninth grader could both score a 225, indicating that they had similar levels of skill (in this example, near the score of the average American sixth grader). Since these studies reported the means and standard deviations for each grade, it is possible to estimate the proportion of students who score at a level closer to students in other grades in their school than those in their own grade.

The results confirm that there is substantial variation within students in the same school and grade, though few schools are likely to exhibit the kind of extreme variation implied by the study described above.

Still, more than 40 percent of fourth grade students in the two studies I analyzed are nearer to the average score of students in other grades than to the average of students in their own grade. This holds true for both reading and math. Some students (from about 3 percent to 10 percent, depending on the subject and sample) score two whole grade levels above or below, meaning at the sixth or second grades.

Figure: Many students score closer to the average of other grades than their own (estimates)

Sources: School 1: Kettler, Ryan J., and Stephen N. Elliott. "A brief broadband system for screening children at risk for academic difficulties and poor achievement test performance: Validity evidence and applications to practice." Journal of Applied School Psychology 26.4 (2010): 282-307; School 2: Capraro, Robert M., et al. "An examination of mathematics achievement and growth in a Midwestern urban school district: Implications for teachers and administrators." Journal of Urban Mathematics Education 2.2 (2009): 46-65.

The national MAP data suggest that the variation expands greatly in later grades. (For this I did not have school- or district-level data to examine.) By eighth grade, the national variation in grade level is at least 50 percent larger than in fourth grade. High schools have developed many types of differentiation, including honors programs, Advanced Placement courses, remedial education, and other ways of addressing students’ diverse needs. Yet according to our recent report, 32 percent of elementary and middle schools still lack any kind of gifted programming.

Providing differentiated programming for students is so critical because connecting the curriculum to students’ interests and providing the appropriate level of challenge are powerful stimulators of student motivation. As Usher and Kolber put it in their study on student motivation, teachers should identify “tasks that are challenging enough to maintain students’ interests but not so challenging as to undermine students’ feelings of competence.” As anyone who has tutored understands, achieving this balance with just one student is not easy, but teaching in a way that will stimulate the interest of a classroom of students when abilities vary wildly is nearly impossible. And if the students themselves are disengaged and unmotivated, we shouldn’t be surprised if they don’t learn much.

With the large range of abilities in a classroom suggested by the MAP data, teachers are in a bind. Focusing on the high-achievers will leave much of the class feeling lost and frustrated, while teaching to the level of the students who are under-performing will bore much of the class, potentially leading to disruptions by students who are restless from under-stimulation. The Goldilocks middle might be okay if the classroom is more homogenous than the ones in the schools I analyzed, but in the average classroom it may turn out to be the worst of both worlds: Struggling students are passed over, and the high-flyers are busy engineering new spit-wad delivery systems.

Stimulating students to build skills and broaden their minds through challenging schoolwork is the main lever the education system has for propelling students out of poverty. Differentiated programming, including gifted programs, is an important way to keep students at all levels challenged and motivated.

On this week's podcast, Mike Petrilli, Alyssa Schwenk, and Brandon Wright discuss what the recent graduation rate scandals say about the state of the American high school. During the Research Minute, Amber Northern examines how parents rank public and private schools in New Orleans’s post-Katrina system of choice.

Amber’s Research Minute

Jane A. Lincove et al., “What's in Your Portfolio? How Parents Rank Traditional Public, Private, and Charter Schools in Post-Katrina New Orleans’ Citywide System of School Choice,” Education Finance and Policy (forthcoming).

The Chicago Public School district (CPS) has been in education news many times over the years, and not typically for its successes. Yet a recent report produced jointly by the Joyce and Spencer Foundations claims that, far from being a poster child for dysfunction, the district is helping students make gains that are among the fastest in the nation. The report presents data and outcomes from a November 2017 conference hosted by the two foundations, in which civic and educational leaders in Chicago met to explore research on the city’s progress, identify possible drivers of the improvement, and reflect on next steps for the district.

The report’s authors primarily rely on a Stanford University study by Sean Reardon as proof of CPS’s progress. The research examines growth of Chicago scores on state tests from 2009 to 2014. Comparing the yearly gains of students in CPS to the two thousand largest districts in the country, Reardon found that CPS elementary school students grew faster than those in most other districts and states. On average, Chicago students achieved six years of growth during the five years between third and eighth grade; the black-white achievement gap held steady because the two groups grew at the same rate; and the Hispanic-white achievement gap narrowed because Hispanic students actually improved faster than their white peers.

Chicago is therefore producing growth, but it’s not clear how. With numerous projects, partnerships, and reforms in play, the report’s authors were not able to identify exactly which programs, policies, or inputs caused the gains. But the report outlines some possible causes that experts and stakeholders discussed at the November conference.

One is the district’s partnership with the University of Chicago Consortium on School Research. The Consortium studies policy changes in CPS and provides frequent feedback, which the district uses to adapt practice. For example, the district’s “Freshman On-Track” indicator, a system to help ninth-graders meet core metrics of standards in attendance, grades, and behavior, is based on Consortium research. Other potential causes for Chicago’s gains are CPS’s efforts to prepare and support new and experienced principals, improve teaching quality through a new evaluation system and stronger curriculum and standards, and train principals and teachers to be stronger users of data on the school level. Additionally, CPS works with a number of outside organizations, including the Chicago Public Education Fund, Network for College Success, and the Academy for Urban School Leadership, whose support might contribute to its progress.

One potential issue, however, is that the report frames the Chicago story rather rosily. It tends to emphasize gains while glossing over the current levels of Chicago’s achievement, like the fact that only 18 percent of CPS ninth-graders in 2016 are expected to earn a BA within ten years of beginning high school (a number which, encouragingly, is up 7 percentage points from 2006). The district also continues to experience conflict over the role of charters, the closing of community schools, and budget issues.

Still, the report is aptly named “Progress and Promise.” Chicago has not discovered an educational panacea; its test scores are not suddenly top in the nation. There is no one magic program or factor to which CPS’s gains can be attributed. In fact, perhaps the takeaway from the report is just that. Investment in personnel quality, consistent use of data and research, and involvement of numerous stakeholders and partnerships are all part of the narrative, as are the effort of countless individuals and decades of struggling for reform. CPS still has a tall mountain to climb, but it seems to have gotten a foothold.

SOURCE: Maureen Kelleher, “Progress And Promise: Chicago’s Nation-Leading Educational Gains,” Joyce Foundation and Spencer Foundation (January 2018).

As teacher evaluation systems evolve around the nation—decreasing the importance of student growth scores in favor of more reliance on classroom observations—how best to support principals in observing and giving feedback on teacher performance will gain importance. While research may play a part in determining best practices going forward, a recent report from the Institute of Education Sciences is more of a cautionary tale than an exemplar.

The study involved 339 New Mexico principals who were scheduled to observe their teachers for the first of multiple times in the early part of the 2015-2016 school year. According to the state’s evaluation framework, principals are required to score teachers on a 22-item rubric after each observation and to hold a feedback conference within ten days of each observation. This was the first year of full implementation of the state’s new evaluation system, which ultimately assigned ratings to every teacher in the state based on classroom observations, student growth data, surveys, and other factors. This study explored whether providing a detailed checklist to principals could improve the quality of the post-observation conferences.

To carry out the experiment, the researchers randomly assigned half of the principals to a control group, while those in the treatment group were given specific, step-by-step information—including a 24-item checklist and a testimonial video from a principal in another state attesting to the efficacy of the checklist. Teachers in the treatment group received the same documents and video as their principals, along with the knowledge that the checklist might be used in their feedback conferences. Adapted from a guide developed by the Carnegie Foundation for the Advancement of Teachers, the checklist aimed to increase the quality and efficacy of feedback conferences, to positively influence teacher professional development, and to raise student achievement. An important effort if successful.

Unfortunately, the experimental design ran into problems. While 77 percent of principals in the treatment group reported viewing the checklist, only 58 percent said they actually used it with one or more teachers. Additionally, 29 percent of control group principals also reported that they had seen the checklist, and 10 percent of them reported using it with one or more teachers, despite an admonition to treatment group participants not to share the checklist. The low uptake rate, lack of consistent use, and blurring of the line between treatment and control groups limits our ability to draw clear conclusions about the effectiveness of these checklists. (This has been a challenge with other federal studies, too.)

Nevertheless, teachers in the treatment group reported that principals using the checklist were less dominant during the feedback conference. This is to be expected as the step-by-step checklist was geared toward principals giving teachers prompts for their response after each piece of feedback was presented. Principals, however, reported no significant impact of the checklist on any measure of conference quality. Both principals and teachers who used the checklist reported that it was useful but also reported concerns that it could lead to formulaic conferences. While teachers who received the checklist were more likely to report following their principals’ professional development recommendations than were teachers in the control group, there was no clear impact on teachers’ subsequent classroom observation rating scores during the school year. The researchers posited that the short timeline for the study meant that PD suggestions were not acted upon within the year. Teachers may use the feedback to improve their practice, but likely not until the summer, with effects potentially visible in the following school year. Finally, the feedback conference checklist had no clear impact on student achievement outcomes—as measured by state math and English language arts exams—or on school report card grades released after the close of the school year. Once again, the short timeline of the study design is to blame.

Productive conversations between teachers and principals about instructional effectiveness is definitely to be encouraged and will likely become more important in states whose evaluation frameworks prioritize observation-based evaluation. Can a step-by-step checklist help facilitate those conversations and improvements? It seems the jury is still out.

SOURCE: Kata Mihaly, et al, “Impact of a checklist on principal-teacher feedback conferences following classroom observations,” Institute of Education Sciences, U.S. Department of Education (January 2018).