The Education Gadfly Weekly

Regular Flypaper readers know that I’ve been skeptical of the “college for all” movement, but I’m 200 percent behind the “college for more” movement. Among other reasons, that’s because completing college brings a strong economic payoff, particularly for young people growing up in poverty. According to Pew’s Pursuing the American Dream, such individuals are almost five times likelier to escape the lowest income quintile as adults if they obtain a bachelor’s degree.

And that’s not just because of the selection effect—the fact that colleges attract relatively able and motivated young people who do well regardless of the path they follow. There’s strong evidence that college adds real value in terms of students’ skills, knowledge, and career preparation, value that translates into higher earnings. Nor is money the only payoff; we’re all familiar with the “scissors charts,” popularized by Robert Putnam, which show the relationship between college attainment, the formation of two-parent families, and other positive life outcomes, including health and even happiness.

So it’s understandable why government and foundation officials have started giving the higher education system the “reform treatment” that was once reserved for our K–12 system; if it’s blocking opportunities for young people—especially low-income young people, as Susan Dynarski argued last week—then removing those barriers should be a top priority for U.S. social and education policy.

But is it? Is higher education really where the problem lies? Are there lots of students who enter college well-prepared for success, only to falter once they get to campus? That might have been the case once upon a time, but I don’t think the data support that storyline any longer. Take a look at this picture, which charts college matriculation, readiness, and completion rates (my sources are listed at the bottom of the page).

Back in 1992, 40 percent of twelfth graders were “college-prepared” in reading, according to the National Assessment of Educational Progress. Yet eight years later, just 29 percent of Americans aged 25–29 had obtained at least a bachelor’s degree. Some of that gap can be explained by high school dropouts—kids who left school before twelfth grade and would not be expected to get a college degree. But most could be seen as lost potential—young people who were academically prepared for college but either didn’t go or didn’t finish.

But note what happened by the high school class of 2005. Thirty-five percent of twelfth graders were prepared for college in reading (and 36 percent in math); eight years later, 34 percent of their age cohort had completed a college degree. This is good news: We closed the gap between college readiness and college attainment. But it also implies that if we want to increase college attainment, we need to make progress on college readiness. There’s little low-hanging fruit left for colleges to pluck.

To be sure, the notion of “college-prepared” is inexact. Some kids just below the line will succeed in college nonetheless (and may even be able to skip remedial courses); some kids over the line will falter (probably because, not surprisingly, young people need to be “prepared” in much more than reading and math to finish a college degree). Almost surely, as Dynarski’s analysis indicates, affluent, ill-prepared students are overrepresented among college completers while poor, well-prepared students are underrepresented. Helping those poor students get across the finish line is absolutely a worthy item on the higher education reform to-do list.

But until we start making significant progress at the K–12 level—and get many more students to the college-ready level before they land on campus—our dreams for significantly boosting the college completion numbers seem certain to be dashed. “Free” community college, co-requisite course-taking, additional student supports, etc.—those may or may not help at the margins. But the big potential still lies in improving our elementary and secondary education system. Government and foundation officials: Capisce?

Sources

The college enrollment numbers come from Census Bureau Table 276 - College Enrollment of Recent High School Completers, defined as “persons 16 to 24 years old who graduated from high school in the preceding 12 months. Includes persons receiving GEDs.”

In reading, the National Assessment Government Board estimates that “college-prepared” is equivalent to “proficient.” So these numbers are the percentage of all twelfth-grade students who were proficient in reading in those given years. (More background here.)

Bachelor’s degree completion numbers come from the National Center for Education Statistics’s Digest of Education Statistics, Table 104.20: “Percentage of persons 25 to 29 years old with selected levels of educational attainment, by race/ethnicity and sex: Selected years, 1920 through 2014." I added an eight year lag—so I used the 2000 numbers for the high school class of 1992, for instance.

Contractors removing old chalkboards from an Oklahoma City high school last week uncovered a second set of chalkboard drawings still covered with lessons and student work from a school day in 1917. The Thanksgiving-themed drawings, multiplication problems, musical scales, and lessons on cleanliness offer an eerie, time-capsule glimpse into the past. But the discovery was important for another reason: Researchers finally have tangible evidence of what kids were learning in at least one American school.

I’m not entirely joking. Pop quiz: Can you name the English language arts curriculum in the public schools where you live? How about the math program? If you can name them, are they any good? How do you know? Do you have student performance data on the program or textbook? Or is your opinion just based on philosophy and preference?

I’ve long lamented the general lack of curiosity within education reform about curriculum as a means of improving student outcomes, despite good evidence that curriculum effects are larger than teacher effectiveness, chartering, standards, and other beloved reform levers. Likewise, I’ve expressed the hope that Common Core might spur something of a golden age in curriculum development (hell, I’ll settle for bronze). But for that to happen, the first step is good, reliable, systematic data on what materials schools are presently using. By and large, such information simply doesn't exist.

A report issued this week from the Manhattan Institute seeks to peer inside the black box that is the classroom. Charles Sahm, the institute’s director of education policy, seized upon a novel technique to determine what curricula New York City schools are using. He asked them.

Sahm emailed an online survey to every elementary and middle school principal in Gotham to find out what math and ELA curricula they’re employing as they implement the Common Core. He also sought to gauge school leaders’ satisfaction with those choices and the extent to which teachers are faithfully implementing the curricula. The upshot is a rare glimpse into one of the most important, yet least studied questions that can be asked at the school or district level: What materials are you actually putting in front of the kids?

Some interesting tidbits from Sahm’s study:

• The curriculum recommendations that the New York City Department of Education (DOE) issued in 2013 (prior to the election of Mayor Bill de Blasio and his appointment of Schools Chancellor Carmen Farina) are largely being followed.
• Low-performing schools were more likely to adopt the DOE’s recommended curriculum than high-fliers. Principals of struggling schools “felt pressure to switch because they knew they would be held accountable if they kept their old curricula and their students’ performance did not improve,” Sahm notes.
• Some schools switched to Pearson’s Connect Math and its ReadyGen ELA curriculum, both on the DOE’s recommended list, simply because they’re published by Pearson, which also produces New York’s annual math and ELA tests. Principals reasoned that those curricula would be the most aligned to the tests their students would eventually take.
• On math, 52 percent of principals report that their teachers are following the curriculum “very closely,” and 32 percent “somewhat closely.” For ELA, 49 percent report following their curriculum “very closely,” and 35 percent “somewhat closely.” (It’s hard to know if a curriculum is any good unless it’s being implemented with reasonable fidelity.)
• New York has been a hotbed of restiveness over Common Core (and Common Core tests), but Sahm’s study shows that principals are largely satisfied with the standards, their curricular choices, and implementation. A strong majority of principals report being “very satisfied” or “satisfied” with their Common Core-aligned math (72 percent) and ELA curricula (65 percent).

Most of these findings will be of particular interest to New Yorkers. But the big question that might interest everyone is, “Hey, how come I don’t know this about my district?”

Step one is to develop and voice curiosity about curricula and materials. “We study teacher effectiveness, where teachers went to ed school, we know what their SAT scores were, but we pay no attention to what materials they’re using in the classroom,” Sahm observes. “It’s just crazy.” He’s right.

The Manhattan Institute report recommends adding curriculum-related questions to annual school surveys administered by the city and state and reporting the results. That would go a long way toward building a data set that will allow researchers to evaluate curriculum effectiveness, help teachers decide where to teach, and allow parents to become more critical consumers where choice exists. And lest you think this is a New York-only problem, Morgan Polikoff, assistant professor of education at the University of Southern California, recently filed over three thousand Freedom of Information Act requests simply to find out what textbooks are being used by school districts in four states: Illinois, Florida, New York, and Texas

There are no obvious reasons why such data isn’t already widely available. “It’s such a non-threatening piece of information. It’s based on the use of public funds,” Polikoff notes. “States, for one reason or another, have not been interested.” He cites California, Indiana, and Florida as exceptions. “I’m definitely hoping states will start collecting this information routinely. It’s such an easy piece of information to capture,” he notes. “Why are we not doing it?”

Why not, indeed? Are you listening, state lawmakers? 

• We can safely say that the Great Recession caused significant pain for American schools, ravaging state budgets and foisting teacher layoffs on some states and municipalities. But in Reynoldsburg, Ohio, at least, necessity has been the mother of more than just pink slips. The Columbus suburb has responded to shrinking revenues by reimagining its public schools and responsibly deploying technology in the classroom. Thanks to a new model of personalized learning, students can now enroll in one of four concentrated academies (chiseled out of what had been the district high school) and earn credit through internships and online classes. Enrollment is up 25 percent from four years ago, performance on state assessments has risen, and costs have been contained.
• Some positive national trends have kept marching right through financial meltdown and mass unemployment—most notably the four-year graduation number, which has climbed steadily in the thirteen years since the passage of No Child Left Behind. But a fantastic (and imaginatively compiled) investigation from NPR reveals that staunching the flow of dropouts isn’t the same as producing students ready for college and career. Peaking graduation rates, the authors report, have resulted from a combination of new approaches: earlier interventions to keep stragglers from giving up (yay!), specialized programs that allow weak students to make up lost credits (meh), and cynical manipulation of statistics at the local level to clear dropouts from district rolls (ruh-roh). Juking the stats is a disgusting abdication of responsibility, but the more legitimate methods of inflating sheepskin totals—like laughably easy credit recovery programs—can be just as pernicious: In Camden last year, half of the “graduating” seniors originally flunked, submitted an appeal, and were given a diploma anyway. If credit recovery and second chances are going to mean anything, they have to be about more than handing out mulligans.
• The benefits of college completion are trumpeted even more loudly than those of high school graduation; to some education reformers, a four-year degree is the only legitimate ticket to the middle class. So what does it say about our higher education system when in some states, the holder of a two-year associate’s degree earns more than his counterpart with a bachelor’s? The American Institutes of Research’s Mark Schneider writes in the Wall Street Journal that associate’s degrees or training certificates in fields like nursing, building construction, and computer science can provide even greater rewards than the traditional college experience at a fraction of the cost. These “sub-baccalaureate” credentials offer a desperately needed pathway to the middle class for students who lack the grades (or funds) to matriculate to four-year programs right out of high school. 

Raising college completion rates, faking high school graduation numbers, and an archeological study on Sesame Street.

Amber's Research Minute

Those of us who have hoped Common Core would hasten the demise of dry and deleterious skills-driven literacy practices at the elementary level can only be heartened by Education Week’s recent in-depth report on building early literacy skills. The package is deeply practice-based and will cheer those who have championed the cause of content knowledge and vocabulary development as a means of raising proficiency—particularly among low-income kids, for whom early reading success (or lack thereof) establishes a trajectory that is devilishly hard to alter.

Highlights include Catherine Gewertz’s first-rate dispatch on the transformation of early-grade read-alouds: Teachers increasingly ask “text-dependent” questions that can only be answered with “detailed understanding of the material, rather than from students’ own experience.” She focuses on a collaborative effort of more than three hundred teachers called the Read-Aloud Project, which was launched by the Council of Great City Schools and Student Achievement Partners.

One of the most important pieces in the package ever-so-slightly misses its mark. Liana Heiten’s report on vocabulary development correctly notes—heavens be praised—the limits of direct vocabulary instruction. (Do the math: there’s not enough time to grow the fifty-thousand-word vocabulary of a literate adult by memorization or word study alone.) “A better approach, some say, is to have students focus on a topic—anything from the musculatory system to the Great Depression to Greek myths,” she writes. This is kinda, sorta right, but it’s misleading to say that the best way to grow rich in “tier-two” academic vocabulary is “by becoming expert in one particular topic.” Single-subject expertise isn’t the secret sauce; the key is familiarity with a broad range of subjects, enabling young readers to make inferences smoothly and reflexively across topics. A child, for example, may read that “annual flooding in the Nile Delta made Egypt ideal for agriculture.” If she’s doing a unit on ancient Egypt, she has the background knowledge to contextualize the unfamiliar word “annual.” If she knows nothing of Egypt and the Nile, or has no idea what agriculture or a delta is, then “annual” is just one more word in a stew of non-comprehension. The child who knows those things learns a new word; the child who doesn’t falls one more word behind. Repeated exposure to new words in familiar contexts in and out of school—Native Americans observed annual rituals; it’s time for your annual check up; some plants are annuals while others are perennials—solidifies the child’s understanding until the word becomes part of her working vocabulary, even without explicit study. In elementary school, reading comprehension and vocabulary development are key, and breadth of knowledge builds both.

I hope I’ll be forgiven for arguing what may seem like a point of orthodoxy, but these details matter. Still, let it not overwhelm the broader takeaway: Those of us who have long argued for content-rich curricula and a laser-like focus on elementary school will find much to like in Education Week’s package on building early literacy. For low-SES students, it gets late early.

SOURCE: “Building Literacy Skills: The state of reading instruction in grades K–3,” Education Week (May 2015). 

Although charter schools were created to be laboratories of innovation, regulations and policies often prevent them from reaching their full potential. Take, for instance, teacher education and certification requirements that can obstruct schools from training educators in the manner that best meets their unique missions, values, and goals. According to a new case study from the Clayton Christensen Institute for Disruptive Innovation, a few highly successful charter schools have overcome these obstacles by creating their own teacher certification and master’s degree programs. These schools include High Tech High in San Diego; Uncommon Schools, KIPP, and Achievement First in New York; and Match Education in Boston.

Each of these schools began their forays into teacher credentialing because they had trouble finding teachers whose “philosophies and methods” aligned with their missions. In addition, they found that many of the teachers they hired lacked the skills to be immediately successful in the classroom. By creating their own teacher training programs, these schools were able to connect formal teacher education with what happens on the ground in actual classrooms. Each program focuses on its parent school’s innovative instructional approach: For High Tech High, it’s project-based learning; for Relay (the graduate school created by Uncommon, KIPP, and Achievement First), it’s competency-based training; and for Match Education, it’s an emphasis on tutoring.

The study provides an overview of the programs’ focuses and structures, the teachers they train, tuition, accreditation status, and plans for future expansion. Interestingly, each program includes a performance-based graduation requirement. High Tech High has different requirements for their various programs including state performance assessments or final projects; Relay requires that second-year candidates in their Master of Arts in Teaching (MAT) and teaching residency programs demonstrate that their students achieved at least one year of academic growth using quantitative and qualitative measures; and Match requires that second-year candidates in their Master in Effective Teaching (MET) program outperform rookie teachers who aren’t trained by Match on an evaluation system that includes principal observations and student survey data.

Unfortunately, the programs also share common obstacles, the two largest of which are (1) meeting the regulatory requirements necessary to obtain state and accreditation approval and (2) developing revenue and cost structures that allow for a sustainable business model. While the process of overcoming these obstacles took years (in some cases, it’s still ongoing), each of these programs offers a valuable look at the benefits of allowing innovative schools to train teachers in customized ways. Kudos to these charter schools for stepping up to the plate and strengthening teacher preparation.

SOURCE: Thomas Arnett, “Startup Teacher Education: A Fresh Take on Teacher Credentialing,” Clayton Christensen Institute for Disruptive Innovation (June 2015).

In this research brief, Angela Duckworth of the University of Pennsylvania and David Scott Yeager of the University of Texas compare three measures of students’ non-cognitive skills: student surveys (in which students self-report on their non-cognitive skills), teacher surveys (in which the teacher provides his or her assessment of a student’s skills), and so-called “performance tasks” (such as the famous "marshmallow test"). After comparing these measures, the authors discuss their suitability for various purposes, including individual diagnosis, improved practice, program evaluation, and accountability.

According to the authors, each measure has advantages and disadvantages. For example, although student and teacher surveys are cheap and reliable, they suffer from “reference bias,” which occurs when individuals or groups use different frames of reference in making a judgment. Consequently, schools that are best at promoting non-cognitive skills may score lowest on a survey measuring such skills.

Unlike surveys, performance tasks don’t rely on the subjective judgments of students or teachers. Yet they too have drawbacks. To be a valid measure of a non-cognitive skill, a performance task must be administered under carefully controlled conditions, which may be difficult to achieve at some schools. They are also expensive and time-consuming, with a single task taking as long as twenty minutes to administer.

Obviously, using any of these measures for accountability purposes introduces additional dangers. For example, teachers could give their students higher ratings than they deserve, and students could be coached to provide the “correct” responses when prompted. To combat these dangers, the authors argue against incorporating surveys of non-cognitive skills into accountability systems or program evaluation (although they have some hope that the surveys can improve teacher practice). As for performance tasks, Duckworth and Scott suggest that a carefully designed suite of tasks could be used in program evaluation if the tasks were brief and administered by computer—although even this possibility is largely theoretical.

As the authors note, existing measures of non-cognitive skills, which were designed by researchers for use in their research, are ill-suited to other purposes. As accountability tools, therefore, these measures simply aren’t ready for prime time. More fundamentally, despite the undeniable importance of non-cognitive skills for students’ long-term success, the inherent difficulty of accurately assessing the character of America’s youth—and the incalculable risks associated with attaching consequences to such an assessment—should give policymakers pause. After all, we already worry about teaching to the test. Do we really want to worry about teaching to the performance task?

SOURCE: Angela L. Duckworth and David Scott Yeager, "Measurement Matters: Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes," American Educational Research Association (May 2015).