A new NBER paper examines whether student coaching can be implemented just as effectively through technology as it can be in person. Specifically, it looks at helping coaches foster students’ motivation, effort, good study habits, and time management skills—and whether all of this can be done just as effectively through automated texts and emails as it can be done through real people. How might this affect grades and credit accumulation?
This study utilizes a sample of over four thousand undergraduate students from a large Canadian university who registered for first-year economics classes in the fall of 2015. Analysts randomly assigned them into a control group or into three treatments meant to help promote the aforementioned skills (management, study habits, etc.): (1) a one-time online exercise where they explored their values and goals for the current year and for their future and how they intend to meet them; (2) a text-messaging campaign that provided them with mostly automated advice about academics and that motivated them to do their best (contact was not initiated with individual students nor were emails presented as coming from a real person—just from a program); and (3) a personal coaching service in which students were matched with upper-year undergraduates coaches who are encouraged to initiate contact and build relationships with students over time, both in person and through texts.
Analysts found no effect from either the online exercise or the text-messaging campaign across the general population or student subgroups. Yet there were large and significant effects from coaching, which increased average course grades by 5 percentage points and grade point average by 0.35 standard deviations over one year. These impacts also strengthened over the semesters of the year. And coached students failed fewer credits and earned more credits on average. The in-person coaches had small groups to coach (no more than five students) because that component was randomly administered to a subset of the group (no more than twenty-five students) that did the online exercise, which meant that coaches could spend more time with their assigned students, averaging seven hours total per week in coaching services. But the high-touch approach carried a high cost. Analysts estimate it cost at least $13,000 to service their small group of coached students; the online cost was negligible; and the text-messaging cost $1,200 for all 1,500 students. They think proactively reaching out to students and building trust were key differences that made the in-person coaching more effective and are now trying to bake that into “virtual coaches.”
What might K–12 learn from this study? Here’s one idea: Ninth grade is historically a tough transition time for students leaving middle school. What if upper classmen in high school chose to spend a few hours a week coaching them so as to pave their smooth transition? Happily, some schools are already ahead of the game.
SOURCE: Philip Oreopoulos and Uros Petronijevic, "Student Coaching: How Far Can Technology Go?," NBER (September 2016).
