Artificial intelligence and machine learning are ubiquitous, playing a role in everything from Netflix and Instagram algorithms to transportation and healthcare delivery. But it’s also increasingly being used to improve educational pedagogy and delivery through a process called educational data mining (EDM).
This growing field of AI R & D uses data gathered in educational environments to recognize patterns and predict outcomes, informing the development of educational theories and practice. The hope is that the predictive capabilities of EDM will enable teachers to do things like identify at-risk students before it’s too late and provide personalized instruction to all students. This is part of the recent rise of personalized learning—a hot-button issue in education technology that is being embraced by researchers, practitioners, and tech-savvy philanthropists like Bill and Melinda Gates and Priscilla Chan and Mark Zuckerberg.
A small but growing body of research suggests that personalized learning strategies can improve students’ academic performance, but doing this well and at scale will require technology that has the ability to identify which students are in danger of falling behind before it’s too late to reverse their trajectory.
To test whether today’s tech is up to the task, researchers at Greece’s University of Petras gathered data on 3,700 secondary school students between the ages of twelve and seventeen, comprising performance on two fifteen-minute tests and a one-hour assessment, overall grades for the two consecutive semesters, and students’ year in school. They then examined how good five of the field’s most popular EDM algorithms were at using these data to predict performance on an a same-year end-of-year math examination.
Researchers used a sophisticated formula known as a “semi-supervised machine learning algorithm.” Here’s how it works: The algorithm uses two kinds of data, “labeled” and “unlabeled.” Labels are added by human operators and signal to the algorithm what is important for the problem it’s trying to solve—in this case, for example, engineers might tell the algorithm that earlier scores on math tests are especially important and likely to be predictive. This makes finding the solution easier, but it also takes significantly more time and effort on the part of humans—one of the things artificial intelligence and machine learning are meant to decrease. Moreover, data collected from schools usually lack these labels, so the goal is to develop technology that predicts performance with minimal human intervention.
To see how much data, and how much labeling, is necessary to make an accurate prediction, researchers tested each algorithm in six ways. Using all of the data, they assessed predictive power when 10 percent, 20 percent, and 30 percent of the student information were labeled. They subsequently conducted the same process only using data from the first of two semesters.
The research found that in all instances, the algorithms were able to predict scores on students’ end-of-year math exam with between 70 and 80 percent accuracy. In other words, this held even when 90 percent of data was unlabeled, and even when only data from students’ first semester was used.
Researchers noted that each algorithms’ predictive consistency was unusual, particularly when only 10 percent of the data were labeled. Although offering no explanation for this surprising result, they argue that what type of labeled data are incorporated into an algorithm matters a lot; in other words, it’s more about identifying which student attributes serve as reliable indicators of future academic performance, such as previous performance, socioeconomic status, certain demographic characteristics, or interactions with learning interventions. They also pointed out that achieving a 70 to 80 percent accuracy rate is fairly uncommon. For example, a recent study examining the predictive accuracy of COMPAS, a similar data-mining algorithm that predicts recidivism of criminal offenders, found that the program only accurately predicted 65 percent of repeat offenses, even when as many as 137 attributes—including offenders’ race, age, number of offenses, and more—were incorporated. The takeaway on both these points is that there needs to be more research that explores how and why certain attributes affect student performance more than others.
Clearly, identifying struggling students and intervening before it’s too late is an educational strategy that researchers and practitioners alike agree is important. But doing so is not always realistic, and because teachers are human, they are bound by limitations both in terms of time and ability to identify at-risk students. Several new studies, including one about using artificial intelligence to develop military combat strategy, indicate that properly-deployed AI is able to more accurately and quickly predict outcomes than even the experts most familiar with the subject at hand.
That’s why research like this is promising. Teachers’ responsibilities and limitations hinder their ability to provide personalized interventions that can reverse a trajectory of poor performance. As this study illustrates, AI can help. And because it and similar technologies will continue to draw significant R & D investment, it only stands to improve over time.
SOURCE: Ioannis E. Livieris et al., “Predicting Secondary School Students' Performance Utilizing a Semi-supervised Learning Approach,” Journal of Educational Computing Research (January 2018).
