One of the more variable aspects of charter school operation around the country is the system by which schools are authorized and managed. Some states limit the number and type of authorizers and management structures; others open the door wider, allowing different types of entities to authorize and to run charter schools. Each approach has pros and cons, but evidence on the outcomes associated with various models is thin. A new report examines Indiana, a state with an expansive structure allowing multiple authorizer and operator types, to see how the interconnected parts impact student academic outcomes.
A research team led by Joe Ferrare of the University of Washington, Bothell, chose the Hoosier State because of its long history of charter schools, with multiple authorizer and operator types, and because of its similarity to the National Alliance for Public Charter Schools’ “model law.” On the authorizer side, Indiana allows institutions of higher education, a state charter school board, and the Indianapolis mayor’s office (a type unique to Indiana) to authorize charters. In terms of operators, the state allows charter management organizations (CMOs), which are nonprofit entities that manage two or more charter schools; educational management organizations (EMOs), which are for-profit entities that manage charters by contracting with operators and perform similar functions as CMOs; and independent operators, who perform all management functions themselves. The researchers were looking to see if students fared better or worse in schools with different authorizer and operator combinations.
Student-level data come from the Indiana Department of Education and cover all public school students in grades three through eight in the school years between 2008 and 2018, including their performance on state math and ELA tests given annually. Ferrarre and his team identified an initial group of students who switched from any traditional district school to any charter school (virtual and brick-and-mortar) during the study period, as long as students moved in grades three through seven and had spent at least one year in their district school before switching. Each switcher was then matched with at least one peer—sharing gender, race/ethnicity, socioeconomic level (as determined by free or reduced-price lunch status), and math and English language arts baseline test scores—who remained in his or her district school. School switchers were matched with as many non-switchers as fit the criteria, thus the treatment group ultimately included 11,284 students attending eighty-eight charter schools across Indiana, compared to 46,060 control group students attending 1,343 traditional district schools. This represents 61 percent of the initially-identified switchers, nearly 20 percent of all Indiana charter students in grades three to eight, and 93 percent of all charter schools. Students and their matched peers were followed for a maximum of three years. Importantly, there was no minimum amount of time required of switchers to stay in their charter schools: initial school switchers were treated as permanent switchers even if they moved back to a traditional district school (or made a combination of moves) at any point following the initial switch. More on this below.
Critical to the analytical intent was the distribution of treatment students across authorizers and operator type. Higher education entities—comprising Ball State University, Grace College, and Trine University/Education One—represented forty-six schools and 8,617 students; the Indianapolis Mayor’s Office represented thirty-two schools and 2,460 students; and the Indiana State Charter Board represented ten schools and 207 students. As to operator types: Charter Management Organizations (CMOs) represented forty-one schools and 3,349 students, independent operators represented thirty-four schools and 2,908 students, and Education Management Organizations (EMOs) represented six virtual schools (4,343 students) and seven brick-and-mortar schools (594 students).
So how did students switching into charters fare on their math and ELA tests based on the authorizer type? On average, switchers in schools authorized by Ball State experienced large, negative effects in math that persisted across time and moderate losses in ELA that were not statistically significant after three years. Trine/Education One charter students saw large losses in math for the first two years, but recovered by year three, and saw large losses in ELA for the first year only. Grace College charter students experienced a similar pattern of losses to Trine, but smaller in magnitude. By contrast to the higher education-based authorizers, overall estimates for students switching into charter schools authorized by the state returned null effects across the board, while students switching into charters authorized by the Indianapolis mayor’s office saw moderate but significant increases in both math and ELA scores.
But does this mean that higher education entities are the worst-performing authorizers overall? That depends. When looking at outcomes by operator type, schools run by CMOs had small positive effects in math and ELA over all three post-treatment years. Independently-operated charters showed small but significant negative effects in math across all three years, and null effects on ELA. Brick-and-mortar schools run by EMOs showed positive effects in ELA across all three years, null math effects for years one and two, and positive effects on math scores in year three following a student’s switch. Virtual schools run by EMOs showed almost entirely large and negative effects, except for a null impact on math scores in the third year post switch.
Combining these two sets of results yields a sizeable matrix of outcomes showing that the impacts of authorizer type and operator type interact. All of the various permutations are worth a read, but just for one example: The researchers find that the negative effects of Ball State’s charters were driven by their EMO-operated virtual schools, whose students showed significant declines on test scores in both ELA and math across all years. Negative impacts were also seen across all years and subjects in Ball State’s independently-operated charters, although far lower than in the EMO-virtual schools. Meanwhile, their EMO-operated brick-and-mortar schools actually produced positive and significant effects in both subjects in all but one year under study, completely opposite of the findings if authorizer type is the only variable.
In short: High-quality authorizers and high-quality operators are required to provide the best academic outcomes for students.
Ferrare and his team executed numerous robustness checks to help eliminate concerns about the internal and external validity of their model and seem satisfied. However, some additional nuance is likely required due to the fact that the model treats all initial school switchers as permanent switchers even if they moved back to a traditional district school (or made a combination of moves) at any point following their initial switch. More differentiation between groups is likely required to fully explore the impact of school structure on single vs. multiple switchers.
While the overall finding is that authorizer and operator type matter greatly in producing positive student outcomes, the interactions between the two layers are clearly complex. Generalizability to other states is probably limited as well based on several characteristics unique to Indiana. However, the researchers’ recommendations here seem relevant to other contexts: Making authorizers focus on improving or closing persistently low-performing schools (at the risk of losing their ability to authorize if they don’t), changing fee structures to incentivize academic achievement, and increasing transparency around corrective action efforts. Much of this already exists in Indiana law, but with plenty of wiggle room. Perhaps Hoosiers should take a page from the Buckeye playbook in the realm of charter law improvements.
SOURCE: Joseph J. Ferrare, R. Joseph Waddington, Brian R. Fitzpatrick, and Mark Berends, “Insufficient Accountability? Heterogeneous Effects of Charter Schools Across Authorizing Agencies,” American Educational Research Journal (May 2023).
