Larissa Campuzano, Mark Dynarski, Roberto Agodini, and Kristina Rall
Mathematica Policy Research for the Institute of Education Sciences
February 2009
This experimental study examines a small slice of the educational technology pie: educational software programs. A second-year follow-up, it examines 10 reading and math programs (by grade level/subject and individually) in twenty-three districts to determine whether students who use them outperform those who don't. For the most part, the answer is no. The outlier was Algebra I; students using Algebra products as a whole scored significantly higher than students who didn't in year 2 (although not in year 1). Individually, only one of the ten--LeapTrack from LeapFrog Schoolhouse--had a positive statistically significant effect on test scores. But before we condemn the other programs as a waste of time and money, there are several study factors that might make us reconsider. First, overall usage of the products was much lower than publishers' recommendations (most programs recommended students use the product 15-30 minutes three times a week as opposed to once-a-week, which was the study average); second, school districts in the study volunteered to implement particular products, so differences in district characteristics may impact a product's effectiveness; and finally, we have no idea how teachers may have modified the use of the software products since classroom observations/teacher surveys were not included as part of the study. Still, it's a worthwhile read and adds to our understanding of one of the many forms of educational technology. You can find it here.
