Arguments for and against “no zeroes” and other types of “hold harmless” grading policies in K–12 schools are often rooted in emotion rather than evidence. Proponents argue that rigid grading schemes are punitive and crush students’ love of learning. Opponents say that lenient grading disincentivizes student effort and fails to reward high achievers. A recent NBER working paper attempts to put some empirical meat on those rhetorical bones, looking specifically at college courses graded on a pass/fail basis. The study was conducted at Wellesley College, a private liberal arts institution for women founded in 1881 that boasts as alumnae Madeleine Albright, Hillary Clinton, and Cokie Roberts, for starters.
Wellesley students can take an unlimited number of courses as voluntary pass/fail. “Pass” appears on students’ transcripts if they receive a letter grade of A–C, while “fail” appears for D–F. Between fall 2004 and spring 2013, 9 percent of Wellesley courses were designated as pass/fail based on students’ choices.
In fall 2014, the school began mandating pass/fail grading for first-year, first-semester students. The intention was to ease the transition and growing pains from high school into college and to nudge new students to take courses—presumably ones they perceive as harder—that they might not otherwise undertake. However, instructors still followed a “shadow grading” policy, meaning they recorded letter grades but only shared them privately with students. Transcripts for these first-year courses reported “pass” for letter grades A–D and “fail” for the letter grade F, making it even easier to successfully complete a course than under the voluntary pass/fail grading scheme above.
The analysts, a trio of economists from Wellesley itself, use a regression discontinuity-in-time design, in which the academic year is the running variable and fall 2014 is the policy cutoff around which they examine outcomes. So they are analyzing two periods: 2004–05 through 2013–14 (pre-mandate) and 2014–15 through 2019–20 (post-mandate). They test for bias in numerous ways while refining their models, attending to the possibility that their results could be affected if students were non-randomly sorting into or out of the sample (either in anticipation of the pass/fail policy or in response to it). Ultimately, they are able to conclude that sorting is not a major threat.
As to the question of course choice, results show that first-semester students exposed to the policy of mandatory pass/fail were 82 percentage points more likely to take a course as pass/fail compared to the earlier groups (pre-2014) that had only the option of doing so. Students in some later semesters—meaning after the first semester when the mandate no longer applied—were slightly less likely to use the voluntary pass/fail option, possibly providing tacit understanding that “too many” ungraded courses on official transcripts is off-putting to employers or graduate schools. The mandate also specifically influenced course choices in the first semester, increasing the probability of taking a STEM course by 5 percentage points. The analysts looked to see if early engagement with introductory STEM classes might spur later engagement via course and major choices. However, they find no evidence of an increased number of STEM courses taken or of an increased probability of graduating with a STEM degree. Rather, the policy’s main effect on students’ choices was to shift enrollment into introductory STEM courses to earlier in the college career—potentially nudging them to take courses they perceive as harder, per the policy’s intent.
“Shadow grading” also revealed that the pass/fail policy lowered the average grade points of first-semester students by about 23 percent of a standard deviation as compared to that of first-semester students in the pre-policy era (prior to 2014). What’s more, it lowered the grades of first-semester students relative to later-semester students enrolled in the same courses.
Why would that be? Analysts test a number of theories, and find little evidence that differences in the classroom composition of students, lower-quality instruction, tougher grading, or arbitrary modifications in grading standards for first-semester versus later-semester students are to blame. In the end, their most plausible explanation for the policy-induced reduction in grades is that students in mandatory pass/fail classes exerted less effort relative to letter-graded students. (No surprise there!) More specifically, removing transcript consequences for first-semester students likely led to a lowering of effort in those classes and did not translate into longer-term pivots to STEM fields.
Still, let’s keep in mind that this is Wellesley, a prestigious destination for smart and accomplished students. It seems likely that first-semester jitters—and induced GPA reduction—will resolve themselves as these small fish feel more at home in their big new pond. Alas, the same cannot be said of general-education, open-to-all K–12 schools, where administrators should be quite wary of hold-harmless grading policies that could usher in a perpetual lack of effort—with permanent consequences on educational attainment for many students.
SOURCE: Kristin Butcher, Patrick McEwan, and Akila Weerapana, “Making the (Letter) Grade: The Incentive Effects of Mandatory Pass/Fail Courses,” National Bureau of Economic Research (December 2022).
