Schools around the country have been expeditious in responding publicly to the rapid onset of ChatGPT and other interactive platforms that utilize sophisticated artificial intelligence, and those in the know say this technology could change teaching and learning forever. Is that change a good one or not? It’s impossible to tell because strong arguments are being made in both directions—ditto for those school responses. Some are banning AI reflexively, while some are embracing it with a lot of gusto. Time will tell which path, if any, is the right one.
It is serendipitous, then, that a newly-published meta-analysis on technology integration (TI) in K–12 education focuses not only on how TI has gone over the last fifteen years, but also how it’s been analyzed and assessed in research literature.
A trio of Swiss researchers, following up on a previous study, identified thirty-six unique articles published between 2010 and 2021 that investigated the integration of technology—any technology, but largely new devices and software—in any area of K–12 education. They were looking not only at the findings of each, but also at the way in which those findings were reached. Their efforts include identifying what types of survey instruments were used to collect data, at whom those instruments were directed (teachers, students, school leaders, etc.), and a long roster of pedagogical aspects to which technology is or is not applied (such as independent learning, providing electronic texts, facilitating collaboration, focusing on media literacy, allowing for research, and more).
Their analysis found thirty-five unique survey instruments were used to collect data among the full set of articles. While only two instruments were used in more than one article, many of those deemed “unique” were in fact very similar to each other. For example, although the information gathered on TI seemed almost too wide-ranging at first glance, a deeper look revealed that focusing on teachers’ attitudes and preferences—as opposed to those of students, parents, or school leaders—predominates.
Furthermore, the researchers were unimpressed by the rigor of some questions. They cite a specific example in which teachers were asked whether they “vary the use of information technology” to “increase the expected benefits” and to “maximize the effects” of TI with their students, but were not asked about what they did to integrate technology nor about the benefits and effects. A dearth of such vital definitions plagued most of the survey instruments reviewed—and these studies are all quantitative in nature! The researchers note that similar terms—such as “integration,” “enhanced learning,” and “transformation” of pedagogy—are applied to planning, executing, and assessing TI in various surveys, but those terms can mean very different things in various contexts. Another survey item recurring in various forms—“I can apply what I know about technology in the classroom” or “I am able to use it as an instructional tool and integrate it into the curriculum”—is entirely about self-efficacy and is not followed up with any indicator that the teacher is actually integrating technology into their lessons.
Precious little detail emerges from the articles about what types of technology students are using, in what courses or for what purposes they’re using it, how adept they are with it, or whether their learning outcomes are impacted by the introduction or mastery of technological treatments. The most-frequently cited student uses of TI are to “enhance students’ cognitive engagement,” to promote collaboration between students, and to allow students to conduct research online. A cynic might suggest those translate to YouTube videos, Microsoft Sharepoint, and Google Scholar, but it is also possible that innovative and effective new opportunities are rising in our schools. Either way, none of the thirty-six articles discusses any findings as to the efficacy of these “technology integrations.”
The main recommendation in this meta-analysis is for researchers to try and come to some common understanding of what the term “technology integration” means. Is it a process? Is it a quantity of technology? Is it the effective replacement of, say, textbooks with digital resources? Or is it the success of students using new tech? The researchers hope that a better definition will lead to better assessment. (Although perhaps a simpler route would be to jettison the nebulous “integration” questions and focus on individual technologies and assessing their success—or not—at their intended academic improvement goals.) But given the contents of this report, any such change seems a long way off. Meanwhile, technology development marches on, with students at ground zero, integrating itself into their lives regardless of whether schools have a definable grip on it.
SOURCE: Tessa Consoli, Juliette Désiron, and Alberto Cattaneo, “What is ‘technology integration’ and how is it measured in K-12 education? A systematic review of survey instruments from 2010 to 2021,” Computers and Education (April 2023).
