The Education Gadfly Weekly

Education reform advocates nationwide are putting together their wish lists for the state legislative sessions that generally start in January. Here’s one for the top of those lists: tackling how state assessments are administered and how kids’ results on those assessments are reported.

Getting this right is good policy, good politics, and a hedge against another anti-testing backlash.

For inspiration note the changes enacted in Florida last year via HB 7069. Its controversial provisions have drawn most of the attention, especially its requirement that local districts give charter schools access to local property taxes. But it also made some worthy tacks on testing. These include:

• Moving state tests to the last four weeks of the school year to give teachers more time to teach—and reducing dead time at year’s end. (The previous state testing window started in March.)
• Requiring that teachers receive the scores of their incoming students before the next year starts.
• Including in the score reports that are sent home to parents:
  • Information about students’ strengths and areas for improvement
  • Specific suggestions for actions parents can take on their child’s behalf
  • Data on proficiency and growth over time, over multiple years
  • When available, projections of how students with scores like theirs are expected to score on the ACT or SAT
• Returning the results from any formative assessments, like the MAP or iReady, to classroom teachers within one week and to parents within thirty days. 
• Publishing the statewide testing schedule two years in advance to give districts maximum flexibility to plan their calendars.

All of this feels like common sense, but it’s also popular. Patricia Levesque of the Foundation for Florida’s Future and Foundation for Excellence in Education says these ideas came from focus groups with parents about their biggest testing bugaboos. Among them, according to Levesque: “The tests are not aligned to what teachers are teaching, nor used to help my child; too much cramming before the test, and too much dead time after the test; teachers who haven’t seen the information from the tests; and a lack of transparency in what is tested and why.”

Representative Michael Bileca, who sponsored the omnibus education measure in the Florida House, and who is a member of Conservative Leaders for Education, said that “getting better reporting on student growth and proficiency and getting it into the hands of parents: That’s useful.”

To Florida’s list of reforms, more about which here, I’d add one more: States should ensure that test score reports actually reach parents. It’s crazy that we spend so much time and money assessing kids and then ignore the “last mile” issue: getting the scores into the hands of families.

The best outcome would be for states to mail or email test results directly to parents. If that’s not feasible, they might require districts to do so. Or, if districts insist on sending these documents home to parents in kids’ backpacks—as happened with my son—they should develop a system like emailing parents that they’re coming on Tuesday and/or asking for a parent’s signature to make sure that caregivers actually see them.

None of this will mollify hardcore haters of standardized testing. But these commonsense improvements will ensure that parents and educators get the full value out of assessments, and might keep bigger, more damaging changes at bay. Certainly they’re worth a hearing.

For the first time in their lives, my twin daughters are attending separate schools. It was a hard decision made after a lot of research and soul searching. My wife and I think both schools are good ones, but I’d be lying if I said I was 100 percent confident. The national debate over whether and how parents can know best when it comes to school choice has me wondering if we’ve chosen well. I am somewhat comforted by the fact that we had full information and access to many options, but I know that’s not the same for every family. That should be the debate on parental choice. Perhaps the process that my family went through—and the differences between the schools we ultimately chose—can help shed light on the larger discussion.

The school that both girls attended through ninth grade last year is an odd one, to be sure, and not just because of its sixth-through-twelfth-grade orientation. As a standalone STEM school, it is more like a charter than a traditional school, but it has no sponsor or elected board; it is supported by a consortium of higher education, philanthropic, and district leaders. As a public school, however unusual, it had a bevy of data for us to look at when we first considered it five years ago: test scores, proficiency and growth data, graduation rates, college-going numbers, teacher education levels, student and teacher diversity information, etc., all of which were impressive, especially given the dismal outcomes for the district’s middle and high schools that my wife and I had investigated and decided against. A school visit to observe classes and talk about educational philosophies left us energized and excited for the possibilities this unusual option could provide.

But there were items on the con side too. The school was very small and had minimal arts and language options, no sports or organized extracurriculars, no auditorium, no busing for us, and no music programs. Accelerated curriculum and early college tracks took the place of more recognizable AP and honors courses. The alums brought out to talk about how great the school is were first-generation college goers who did not look like my daughters. Would they fit in? Would they be motivated? The school is so academically focused it seemed barely able to muster enthusiasm for two social events per year. Most of the people we talked to about our options had never heard of the place. Heck, its lunch program was an array of vending machines.

Did we make the “right” choice? If you are a data-driven decision maker, you might think so, despite the many down sides. If you think student achievement as measured by annual tests is important, you might also approve of our choice. But if you are more inclined to value school culture, a fully rounded socio-emotional experience, some impressive names among the list of alumni, or perhaps a winning football team complete with a homecoming dance, you might argue we made a mistake. Perhaps you may even think that we harmed our children by our choice due to what our chosen school lacked.

One daughter has thrived in this environment and will likely remain there until the end of high school–although the rapid pace and mastery focus has allowed her to complete nearly all of her high school requirements already. Unfortunately, the rapid acceleration and relentless STEM focus led our other daughter to hit a wall last year and thus we needed to make yet another choice for her.

The school to which we moved her this school year is another odd duck: a secular private school of more recent vintage than most. It shares traits with the more familiar prep school model but seemed to us to be a bit more down-to-earth when we investigated. As an independent private school, it had no historical test score data to share, and no academic ranking to compare. In fact, staffers and parent liaisons were adamant that these things shouldn’t really matter. To them, its reputation as a good school was most germane. The quality of its faculty seemed based on how long they had been teaching or where they received their degrees, but the list of school founders was impressive and familiar. The campus was lovely, secluded, and well-appointed. The theatre production we attended was quite good, and there was no escaping the schoolwide excitement over the basketball team’s current run, the impending study trip to Russia, and the quiz bowl team’s fine year. Alumni names dropped regularly in conversation. Dining room, commons, library. Interestingly, this school also eschewed AP, IB, and honors classes, which were replaced by independent study and research projects in students’ junior and senior years, designed and conducted by the students themselves. What constitutes “A” work on such a project? No one seemed to know or care. That was not the point.

Did we make the “right” choice here? Reputation and school culture were the marketing angles, the exact opposite of our previous choice. In the end, we chose this school largely because our daughter had nearly completed her high school requirements—as determined by end of course exams per Ohio law—at the public STEM school and would be able to take advantage of “extras” in the new private school.

We have experienced ups and downs at both schools and questioned our decisions multiple times. Both schools we chose are featured prominently in a recent article about what the future of education in Columbus should look like, which is something of an affirmation, but it’s not proof. So far, our children are learning, growing, and thriving to our satisfaction whether or not we have data to prove it. Even this far into high school, however, there are other options we can explore should the current schools prove ineffective for either of our daughters. That, to me, is the essence of school choice.

The current national debate on parental choice seems to me to lack the nuances illustrated by my family’s story. First, even families with lots of information and access to options struggle with decision. Reputation or data or both? Perhaps other non-academic factors will prevail. Second, families like mine are advantaged over others who don’t immediately have the same resources and systemic familiarity. For their sake, the national debate cannot yet be “parents know best” versus “policy wonks know best.”

School choice supporters must continue to push for multiple options to be fully and freely available to all families regardless of zip code, skin color, and income. Inter-district open enrollment, charters, vouchers, and even standalone schools must be expanded to all who need it. Most importantly, supporters must make sure that as much data as possible is available on all aspects of school “goodness,” both quantitative and qualitative. Solid data are valuable, as are a well-deserved reputation and spending time in a prospective school to see if it passes the eye test. Data should be easily accessible, understandable, and comparable across as many school types as possible.

Until that is a universal reality, all parents can’t know best, and those who have information and access are advantaged over others. The national debate must change to reflect this reality.

On this week's podcast, special guest Anne Hyslop—education consultant and former Senior Policy Advisor in the Obama Department of Education—joins Alyssa Schwenk and Brandon Wright to discuss staffing and policy issues facing the department under Secretary Betsy DeVos. During the Research Minute, Amber Northern examines how teachers affect students’ attitudes and behaviors.

Amber’s Research Minute

David Blazar, “Validating Teacher Effects on Students' Attitudes and Behaviors: Evidence From Random Assignment of Teachers to Students,” Education Finance and Policy (October 2017).

In this study, the authors examine the impact that being born in the Year of the Dragon—the luckiest and most desirable in the Chinese Zodiac—has on the academic achievement of Chinese youth.

Data for the study come from three sources: the 2010–13 waves of the Chinese General Social Survey (CGSS); the China Education Panel Study (CEPS), which includes over 13,000 middle school students from 112 schools in twenty-eight districts, counties, and/or cities; and the Beijing College Students Panel Survey (BCSS), which includes about 5,000 undergraduates from fifteen universities in Beijing.

Based on their analysis of these data, the authors estimate that, relative to individuals born in other years, so-called “Dragons” score about 0.05 standard deviations higher on middle school Chinese and English exams and 0.1 standard deviations higher on China’s National College Entrance Examination, and are 5–10 percentage points more likely to have a college education.

Because parents of Dragons are no richer, better educated, or more likely to have white-collar jobs than other parents, the authors conclude that “the differential educational success of Dragon children is not related to family background.” Moreover, because the CEPS includes questions related to “dimensions of language, perceptions of figures and spaces, and calculations and logic,” the authors are also able to control for “cognitive ability” in their work with these data.

Perhaps more surprisingly, despite their lucky status, Dragons do not believe they are more able than other students or that they will have a more successful future. However, parents of Dragons do have substantially higher expectations when it comes to their children’s educational attainment and future success, and they invest more time and money in helping their children meet these expectations. For example, parents of Dragons are more likely to enroll them in kindergarten, more likely to initiate conversations with their teachers, and less likely to make them do household chores. Based on these patterns, the authors conclude that the superstitious beliefs associated with Dragon status create a self-fulfilling prophecy.

Other than the possibility that Dragons really are lucky (which the authors inexplicably ignore) the biggest threat to the study’s validity is the possibility that parents of Dragons differ from other parents in unobservable but outcome-relevant ways. For example, they might be better planners or more excited about having kids. (The Year of the Dragon is helpfully preceded by the Year of the Rabbit, with the result that number of live births spikes noticeably in Dragon years.)

Ultimately, the authors can’t completely rule out this possibility. However, they do show that the “Dragon effect” on test scores disappears when they account for parental expectations. This strongly implies that higher expectations are the channel through which Dragons gain an advantage. But what’s not quite so clear (at least to this reviewer) is to what extent those higher expectations are truly a consequence of Dragon status.

For the sake of argument, let’s assume that some or all of the “expectations gap” between Dragons and their less fortunate schoolmates is indeed attributable to superstition, meaning that parental expectations are mutable at some level. What then should policymakers and practitioners do to raise those expectations?

One obvious answer is to lie: “Jonny is one of the smartest kids in the class.” Another, equally obvious answer is to tell the truth: “If he applied himself, there’s no reason Jonny couldn’t be an A student.” But of course, it is ultimately up to parents to take to heart the central message: You too could be a mother—or father—of Dragons.

SOURCE: Nacu H. Mocan and Han Yu, “Can Superstition Create a Self-Fulfilling Prophecy? School Outcomes of Dragon Children of China,” National Bureau of Economic Research (August 2017).

Harvard’s Center for Education Policy Research has released a study that provides the first look at how Newark schools are faring after the enactment of controversial reforms. Kicked off by an infusion of $200 million from the Startup: Education Foundation (now the Chan Zuckerberg Initiative) and other philanthropists in 2010, the reform included curricular overhauls, scaling up charters, dramatic staff changes, and school closures. The initial findings indicate that these disruptive changes are improving student achievement.

The study examines student growth on New Jersey state achievement tests from 2009 to 2016—before and after funding—comparing students of similar academic achievement and demographics to other lower-income districts in the state, as well as to overall state results. Analysts also look to determine which reforms were responsible for the changes in growth. As the scope of reforms is quite large, they break them into two categories: within-school reforms, such as personnel changes, new teacher contracts, and curriculum reform; and between-school reforms, including closing low-performing schools, the expansion of charter schools, and the implementation of a universal choice system that allowed families to submit a single application to attend a district or local charter school.

Analysts find that student growth dropped in the first two years of reform, before rebounding in the 2014–15 and 2015–16 school years. By the final year of the study, overall achievement growth was 0.09 SD above the state average in English, an improvement of 0.07 SD from the implementation of the reforms. In math, growth was statistically the unchanged from the beginning of the study. These trends mirrored the results as compared to other low-income districts in the state.

Analysts also find that between-school reforms, driven by changes in enrollment between schools, consistently improved achievement growth over the course of the reform efforts. Major enrollment changes started in the 2012–13 school year, as a number of Newark’s lowest performing schools were closed, and expanded in 2014–15 with the adoption of the universal choice system. The result was that schools that added more academic value grew faster than schools that added less, so more students were in better schools. The analysts calculate that this movement accounted for 62 percent of Newark’s improvement in English growth.

One major uncertainty of the study is that the dramatic rise in test scores in 2014–15, which occurred across the state, though more dramatically in Newark, corresponded with New Jersey changing the state test from the New Jersey Assessment of Skills and Knowledge (NJASK) to the PARCC assessment. PARCC emphasizes different, more critical thinking skills in math and English than NJASK. Thus the bump in scores could be more related to the test change rather than the reforms taking place at the school level.

These results do not, as some reformers may be tempted to claim, mean that Newark’s dramatic reforms are a blueprint ready to be exported to other districts. Rather they are a product of unique set of circumstances, including a previously strong charter sector and a state controlled district. Still, they are at least as tentative vindication for local officials who persisted in enacting reforms in spite of much controversy and the departure of philanthropists and high-profile supporters.

SOURCE: Mark J. Chin, Thomas J. Kane, Whitney Kozakowski, Beth E. Schueler, and Douglas O. Staiger, “School District Reform in Newark: Within- and Between-School Changes in Achievement Growth,” Center for Education Policy Research (October 2017).