Despite a stampede of interest in students’ social-emotional development (SED), gathering data on—and measuring the success of—such initiatives remains a challenge. Researchers often rely on self-reported surveys in which students respond to questions about their experiences, attitudes, and beliefs related to school and themselves. But can these be used to accurately determine whether some schools are doing better at promoting SED than others? If so, can attending such a school boost a student’s chance of postsecondary success? A recent study from researchers at the University of Chicago and Northwestern University suggests that the answers are “yes” and “yes.”
Using survey, administrative, and test-score data for six cohorts (55,560 students) of first-time ninth-grade students who attended a Chicago public high school between 2011 and 2017, the analysts examine the extent to which some schools do more than others to improve students’ social-emotional development in two major domains—social well-being and work habits—and how such schools impact students’ short and long-term outcomes.
First, to identify which schools were better at promoting social-emotional development, the researchers conducted a value-added analysis that sought to identify causal effects of individual schools on students’ test scores, social well-being, and work habits. Social well-being and work habits were measured using the 5essentials Survey, a survey that the Chicago Public School system has been administering to sixth- through twelfth-grade students since 2010. (Click here to see sample survey questions.) To conduct the value-added analysis, test scores and survey responses of each student in a particular school were compared to similar students in other CPS schools at the end of ninth grade. Students were defined as “similar” based on a broad range of criteria that included gender, ethnicity, the socioeconomic status of their census block, and free-or-reduced-price-lunch status. A school’s value-added was then determined by how much it increased students’ test scores and social-emotional development relative to measured changes in other schools. Schools were determined as having a high value-add in SED if the score in each domain—social well-being, work habits, and test scores—was 1.0 standard deviation higher than the average school in the study, or equivalent to attending a school in the 85th percentile of performance in a particular domain.
The researchers then used these data to measure how schools with high value-add in SED affected students’ test scores, as well as other short and long-term outcomes. These short-term outcomes included the number of absences, number of disciplinary incidents that made the student eligible for suspension, whether a student had ever been arrested on or off campus, and a “freshman on track” measure (i.e., whether a student earned five full-year course credits and no more than one F per semester of work in a core course in their freshman year). Long-term outcomes included high school completion and college enrollment.
Of the paper’s many results, one noteworthy finding is that schools that excelled at improving students’ social well-being and rigorous work habits also raised students’ test scores by 6.0 and 5.7 percent of a standard deviation, respectively. This important finding suggests that schools that have a strong history of improving test scores aren’t necessarily focusing on academic achievement alone; rather, strengthening students’ social-emotional development may help facilitate academic success. “Both/and” rather than “either/or” wins the day.
Also of note is that better social-emotional development appears to improve rates of high school graduation and college enrollment. For example, students who attended a high school that excelled in all three domains measured by the study—social well-being, work habits, and test scores—were 1.9 percentage points more likely to graduate high school, and 3.6 percentage points more likely to enroll in a four-year institution within two years of graduation, compared to students attending the average school. Meanwhile, schools with strong test score value-added alone only improved students’ likelihood of graduating by 1.2 percentage points and four-year college enrollment by 2.3 percentage points.
Why might schools that have a history of building students’ social-emotional development see better long-term outcomes for their students? Authors posit that this is because schools with high value-added scores in all three domains improve students’ “freshman on track” status. These schools are also better at reducing school absences (-1.3 days), decreasing disciplinary instances (1 percentage point), and reducing school-based arrests (21 percent) than the average school in the system. While schools with a reputation for high test-scores only did reduce these negative school experiences, they did so at a much smaller rate.
Whether these findings will affect test-based accountability policies aimed at boosting students’ academic success and postsecondary outcomes remains to be seen. If tests are a weaker predictor of these outcomes—and if schools that prioritize social-emotional development see improvement in both test scores and key long-term outcomes—instruments such as Chicago Public Schools’ survey could play a role in the “next generation” of assessments and school accountability. We could see systems incorporate broader measures of school quality that take into account multiple definitions of student development, taking some of the pressure off of tests alone. This study suggests that that would be OK—even for organizations like ours that see the merit in test-based accountability.
SOURCE: C. Kirabo Jackson, et al., “Linking Social-Emotional Learning to Long-Term Success,” Education Next (Fall 2020).
