Last week, I offered thoughts on the Fordham Institute’s research paper on e-schools in Ohio, “Enrollment and Achievement in Ohio’s Virtual Charter Schools.”
It is a strong study. As I wrote, however, it suffers from four significant data limitations—none the fault of the analysts, but instead arising from inadequacies intrinsic to the data that are currently available to measure the outcomes of e-school students—that should give us pause.
It’s worth discussing one of those limitations in more depth, as well as exploring the policy implications of Fordham’s study.
Understanding students’ jobs to be done
The study controls for both the demographic and prior achievement variables of e-school students that are measured at the state level. This is useful for giving us a window into the quality of a school on average, but it misses the full circumstances in which a student enrolls in a full-time virtual school. In our research on innovation, we refer to this underlying causal reason as the student’s “job to be done.” It therefore does not allow researchers—and this report in particular—to actually compare like situations and results.
Demographic data often misleads researchers. In business, for example, most companies simply look at the average demographic likely to use a given service and what features those people say they want. They then make improvements accordingly. But customers often don’t follow what the average in their demographic is likely to do, which is a big reason why, even in the era of big data, the vast majority of new product and service launches fail. Instead, jobs arise in people’s lives—a particular circumstance in which they need to make progress—and they hire different services to help them do those jobs. Understanding the job allows us to understand causality. Traditional demographic data can’t explain why a man might take a date to a movie on one night but order pizza and watch a DVD from Netflix the next night. In each case, a different job has arisen that has caused the behavior.
In the case of virtual schools, a significant percentage of students enroll because of medical issues or bullying. Such a student may be “hiring” the virtual school to help her escape from her present situation—with little regard to what that decision means for her future. Academic considerations are secondary at best. Understanding this dynamic is critical to making a valid comparison and to understanding what success is.
Policy implications
Despite its limitations, the Fordham Institute’s policy recommendations are solid and recognize some of the nuances in the full-time virtual school landscape. Fordham helpfully does not call for a ban on full-time virtual schooling. It recognizes that virtual schools have been critical to the academic and life success of many students. Fordham instead presents constructive recommendations for improving virtual schools—and online learning and schooling more generally.
One of the key recommendations offered in the foreword and conclusion is that policy makers should “explore ways to improve the fit between students and e-schools.” This could help tremendously in addressing the various jobs for which students are hiring virtual schools—and help them see that there may be better options than the particular virtual schools they have chosen. The authors write, “Ohio recently enacted a provision requiring e-schools to offer an orientation course—a perfect occasion to set high expectations for students as they enter and let them know what would help them thrive in an online learning environment (e.g., a quiet place to do schoolwork, a dedicated amount of time to devote to academics).” For students who are not considering the future and are just looking for an escape from the present, helping quickly shift their focus to the future could help them improve their decision making so that they don’t get stuck in a situation they need to escape yet again. I would love to see schools also adopt a page from the new EQUIP higher education program and bring in an outside quality assurance entity. That organization would audit the claims on outcomes that a school makes, driving better student and family decision making.
Additionally, Fordham recommends that state lawmakers explore rules that would allow e-schools to be selective in which students they serve, such that they could optimize their operations for a particular set of jobs students that have and only serve students who have those jobs. I would also recommend offering students and families the services of an advisor as they consider transitioning to a full-time virtual school, which would help them better understand their options and goals.
The call to support statewide online course access also makes sense. This could allow students who are interested in online learning for part of their day to avoid making an all-or-nothing decision between transferring to a “full-time e-school or stay[ing] in their traditional school and potentially be[ing] denied the chance to take any tuition-free credit-bearing virtual courses aligned to Ohio state standards.” This would create options better aligned to students’ preferred jobs.
Finally, Fordham recommends that policy makers
adopt performance-based funding for e-schools. When students complete courses successfully and demonstrate that they have mastered the expected competencies, e-schools would get paid. Implementing performance-based funding policies would create incentives for e-schools to focus on what matters most—academic progress—while tempering their appetite for enrollment growth and the dollars tied to it. It would also encourage them to recruit students likely to succeed in an online environment—a form of “cream-skimming” that is not only defensible but preferable in this case.
Those who study online learning believe that this idea makes sense. But it’s critical that it be implemented correctly. An all-or-nothing funding formula would not work; it ignores the fact that full-time schools, which already have fixed costs to serve students, must make significant up-front investments. A sounder way forward would be to give virtual schools some up-front funds for these administrative and technology costs (such as furnishing students with laptops and home internet connections) and then fund schools with a weighted per-pupil rate as students master competencies and make academic progress. (This approach embeds performance funding in the formula and is similar to how New Hampshire’s Virtual Learning Academy Charter School is funded.) In other words, a student’s academic progress would replace seat time—such as average daily attendance—as the measurement that determines funding levels. As a simple example, when a student mastered 10 percent of a course, the school would receive 10 percent of funding for that course; when the student mastered the next 10 percent, the school would receive the next 10 percent of funds. External state assessments could be used to make sure schools’ determinations of learning were rigorous and valid.
There are those from the online learning community who say they like this performance funding mechanism, but they will agree to it only when all brick-and-mortar schools are subject to it as well. They’re thinking too narrowly. First, seat time funding is totally anathema to virtual education—far more so than for brick-and-mortar schools that have been funded based on seat time for generations. Second, we need to try out a competency-based funding model somewhere in the system first to learn how to make it work correctly. Starting small is important. Third, the traditional schooling system will never let performance-based funding start there; arguing for waiting is akin to saying “no.” And fourth, the new upstart system is always going to look “worse” than the traditional system because it’s not how “things have always been done.” Why not step up and show that the upstart system performs better on a more rigorous standard as a way to get early wins—and then build momentum, flip the public relations narrative, and start conversation about funding for the larger system?
Ultimately, as I’ve written, virtual schools must step up their game. It’s unlikely that they will ever grow to educate more than 5 or 10 percent of K–12 students. But given the important service they provide for many students and families, we need to better understand the value they are delivering. Harnessing their benefits while reining in their downsides is critical.
This post originally appeared in a slightly different form on Forbes.com. Michael B. Horn is a co-founder of the Clayton Christensen Institute.