Rigorous studies have been conducted on various school voucher programs – most notably those in Milwaukee, the District of Columbus, and Florida – but this study by CATO’s Matthew Carr is the first of its kind to study Ohio’s EdChoice Scholarship program.
Rigorous studies have been conducted on various school voucher programs – most notably those in Milwaukee, the District of Columbus, and Florida – but this study by CATO’s Matthew Carr is the first of its kind to study Ohio’s EdChoice Scholarship program. Specifically, it examines whether traditional public schools are spurred to improve in the face of a threat of losing students to private schools. It does not examine whether vouchers are effective for students who use them but rather investigates a key school choice theory: whether competition “creates incentives for systemic improvements.”
To test this, Carr collected achievement data from the Ohio Department of Education on EdChoice-eligible schools over three distinct time periods beginning in 2006 (note, eligibility changed multiple times), which creates a unique research design in that there are three “treatment” periods enabling analyses of whether each school changed its own behavior in response to the voucher threat. (In contrast, other studies have compared fundamentally different types of schools, eligible v. non-eligible schools.)
The study measured school improvement by looking at fourth- and sixth-grade reading and math scores; and the percent of students scoring at various levels (limited, proficient, advanced) to gauge the extent to which schools under threat focused on “bubble students” (those just above and below the proficiency cut-off and upon whom a school’s rating depends most heavily). It also controlled for factors such as school quality (rating on A-F scale), and percent of students who are white, disadvantaged, and/or disabled. Unique to this study (and impressive) is that Carr manages to tease out the “scarlet letter” effect, i.e., did schools improve not because of the voucher threat but rather because of the stigma associated with receiving a highly publicized poor rating from the state?
Several significant findings emerged. The voucher threat was correlated with a achievement gains in fourth grade reading (the equivalent of 2200 extra students reaching proficiency) and this did not result primarily from stigma. Second, performance among bubble students didn’t shift much; rather, students in the lowest and highest performing categories made gains. Carr theorizes that voucher-threatened schools may have focused on those students most likely to exit, and calls for more research in this area.
The Impact of
Ohio’s EdChoice on Traditional Public School Performance
CATO Institute
Matthew Carr
August 2011
With a continued emphasis from the federal government on accountability for K-12 schools, specifically among the lowest performers, state education agencies (SEAs) have had to take a more direct role in school improvement. However, do SEAs have the capacity and resources necessary to take on this work? A recent report from the Center on Reinventing Public Education attempts to tackle this question by examining functions currently performed by SEAs, how they distribute their funds to various functions, and whether or not they are capable of more school improvement work.
To answer these questions CRPE examined SEAs in eight states: California, Colorado, Louisiana, Minnesota, New York, Tennessee, Texas, and Washington, and collected data from each SEA’s website as well as contacted staffers about financial data. While strict generalizations cannot be drawn from eight states, it is important to note that their experiences are reflective of that in many other states and therefore provide insight for other SEAs, including Ohio’s.
Overall, the current state of SEAs’ capacities and workloads paints a grim picture in terms of their ability to be more involved with school improvement. Not surprisingly, almost 50 percent of the staff at SEAs serves in administrative roles (e.g., human resource, financial management, and technology). On the other hand, the distribution of personnel for performance and improvement ranges from 7 to 28 percent. Colorado and California dedicate less than 10 percent of SEA personnel to performance and improvement, while Texas and Tennessee dedicate more than 25 percent of their staff to these functions.
Largely, the investment in school improvement has been relatively minimal. But many states, via Race to the Top (RTTT), School Improvement Grants (SIGs), or their own school turnaround plans, have committed themselves to transforming the worst schools despite facing budget cuts or being short-staffed. So what can SEAs do to fulfill these school improvement duties? CRPE suggests that agencies shift current federally funded positions to school improvement, as well as contract out these services to consultants.
The Ohio Department of Education would do well to examine its capacity to lead school improvement efforts. As a recipient of RTTT and SIG funds it is more important than ever that they take an internal assessment of their capabilities and re-adjust were necessary to best serve the school improvement needs of the state.
The
Capacity for School Improvement: A First
Look at Agency Resources
Center
for Reinventing Public Education
Patrick
Murphay and Monica Quijdani
August
2011
The Ohio Department of Education recently released performance rankings of all charter authorizers (aka “sponsors”), as part of the new requirement that those ranking in the bottom 20 percent of all authorizers cannot take on new schools for one year.
This is a provision we at Fordham fully supported and in fact helped craft, as a means to ensure better quality and accountability in the charter school sector. The rankings, found here, include 47 authorizers including us (our sister organization, The Thomas B. Fordham Foundation, is an authorizer). On a list of 47 authorizers, we ranked 24th. Nine sponsors fell into the bottom 20 percent and cannot open new schools.
We’ve never shied away from the truth when it comes to our schools. Each year, we publish a comprehensive, public account of our schools’ performance (our 2011 edition will be out next month and you can peruse past editions here). We’ve also been among the first to admit that the work is tough; that more school choice without parallel accountability measures is pointless (kids need better options – not just more of them); and that closing schools is an important part of quality authorizing. Historically we’ve accepted the challenge of closing troubled schools poor academic results.
But because these state rankings are new and high-profile, we wanted to take a moment to put them in context and reiterate our emphasis on continuous improvement for all of our schools.
It’s important to note a handful of facts about the rankings shared by ODE:
That said, while our weakest school pulled down that average, we’re happy to report that it also improved this year, moving up from Academic Emergency to Continuous Improvement. And while we are not in the business of authorizing so as to only take on the highest performing schools, it’s also worth nothing that the school is among the better education options for students in its city.
A little history here is useful. Fordham’s sponsorship portfolio has evolved considerably since we started in 2005. We began sponsorship in July of that year with a total of ten schools (all in the Dayton-Cincinnati area) that collectively served about 2,700 students, and all but three of these schools we inherited from the Ohio Department of Education as they were forced out of sponsorship by state law. For the most part, these schools were troubled academically with five being rated Academic Emergency, one Continuous Improvement and one Excellent (three new start-up schools were unrated). Over the last six years we’ve had six schools leave our portfolio either through closure or by jumping to other sponsors, we’ve opened one new school to see it close after a year, and we’ve birthed two new schools. Two established schools voluntarily joined our portfolio this year. We currently sponsor only four of the ten schools that originally signed with Fordham in 2005. This year, Fordham-sponsored schools serve approximately 2,500 children; three schools carry an Effective (B) rating, four schools have a Continuous Improvement (C) rating, and one is rated Academic Watch (D).
Fordham-sponsored schools have made progress. This is a reflection of the hard work and dedication of the educators, school board members, and students in each building. But, more work remains to be done. We know it and we won’t hide from the challenge, but more importantly the teachers, school leaders, and board members working in the schools we sponsor are committed to making a difference in the lives of children who need it and they are making progress. We are honored to be their allies in this struggle.
A version of this piece originally appeared on Fordham’s Flypaper blog. Matt DiCarlo at Shanker Blog posted a response taking issue with Ohio’s Performance Index score calculation.
The George W. Bush Presidential Center (in Dallas) recently released data on international student achievement in both reading and math, which you can peruse in an interactive tool, the Global Report Card. The report card compares 2007 math and reading achievement levels between districts across the nation and 25 developed nations. The tool does not adjust for differences in race, socioeconomic status, or other classifications; however, the tool is still useful to get an idea of how our nation’s students measure up against their future global competition.
How does Ohio fare? In short, Ohio’s major city school districts don’t stack up well at all against their international counterparts.
Among the eight districts, Akron Public Schools had the best showing, ranking in the 28th percentile in math and the 41st percentile in reading. Dayton Public Schools and Youngstown City Schools struggled even more by comparison. Both cities ranked in the bottom 15 percent in math while ranking in the 24th and 25th percentiles respectively in reading. Chart 1 shows how Ohio’s major urban districts fared in comparison to their international counterparts.
Chart 1: International achievement benchmarking of Ohio’s “Big 8” districts
Source: The Global Report Card, George W. Bush Presidential Center
What is more concerning is that these numbers improve (significantly in math) when these districts are measured against just the rest of the United States, meaning the country as a whole is continuing to fall behind other developed nations. This is largely due to the fact that many wealthier suburban school districts are not doing too well either. Chart 2 depicts examples of some of the high-wealth, high-performing suburban districts surrounding Ohio’s big cities and how they compare to their international counterparts.
Chart 2: International achievement benchmarking of select suburban Ohio districts
Source: The Global Report Card, George W. Bush Presidential Center
Considering that most of these districts are among the highest performing districts in their area, the numbers are rather unimpressive. Not one of these top-end schools ranks even in the top 20 percent in math achievement, and only one district (Ottawa Hills) is barely in the top ten percent in reading achievement.
After examining these figures, it is evident that not only are Ohio city school systems falling by the way-side, but the best public alternatives in the state are struggling to keep pace internationally, especially in mathematics. Despite knowing U.S. schools have been falling behind other nations for some time, academic improvement is still proving to be elusive. In an age of globalization and increased worldwide competition these numbers combined with lackluster improvement paint a bleak picture for our future. Not to be redundant, but if the United States wishes to maintain a sense of global economic and political superiority then we still have work to do.
This analysis originally appeared on Flypaper, Fordham’s blog.
Rigorous studies have been conducted on various school voucher programs – most notably those in Milwaukee, the District of Columbus, and Florida – but this study by CATO’s Matthew Carr is the first of its kind to study Ohio’s EdChoice Scholarship program. Specifically, it examines whether traditional public schools are spurred to improve in the face of a threat of losing students to private schools. It does not examine whether vouchers are effective for students who use them but rather investigates a key school choice theory: whether competition “creates incentives for systemic improvements.”
To test this, Carr collected achievement data from the Ohio Department of Education on EdChoice-eligible schools over three distinct time periods beginning in 2006 (note, eligibility changed multiple times), which creates a unique research design in that there are three “treatment” periods enabling analyses of whether each school changed its own behavior in response to the voucher threat. (In contrast, other studies have compared fundamentally different types of schools, eligible v. non-eligible schools.)
The study measured school improvement by looking at fourth- and sixth-grade reading and math scores; and the percent of students scoring at various levels (limited, proficient, advanced) to gauge the extent to which schools under threat focused on “bubble students” (those just above and below the proficiency cut-off and upon whom a school’s rating depends most heavily). It also controlled for factors such as school quality (rating on A-F scale), and percent of students who are white, disadvantaged, and/or disabled. Unique to this study (and impressive) is that Carr manages to tease out the “scarlet letter” effect, i.e., did schools improve not because of the voucher threat but rather because of the stigma associated with receiving a highly publicized poor rating from the state?
Several significant findings emerged. The voucher threat was correlated with a achievement gains in fourth grade reading (the equivalent of 2200 extra students reaching proficiency) and this did not result primarily from stigma. Second, performance among bubble students didn’t shift much; rather, students in the lowest and highest performing categories made gains. Carr theorizes that voucher-threatened schools may have focused on those students most likely to exit, and calls for more research in this area.
The Impact of
Ohio’s EdChoice on Traditional Public School Performance
CATO Institute
Matthew Carr
August 2011
With a continued emphasis from the federal government on accountability for K-12 schools, specifically among the lowest performers, state education agencies (SEAs) have had to take a more direct role in school improvement. However, do SEAs have the capacity and resources necessary to take on this work? A recent report from the Center on Reinventing Public Education attempts to tackle this question by examining functions currently performed by SEAs, how they distribute their funds to various functions, and whether or not they are capable of more school improvement work.
To answer these questions CRPE examined SEAs in eight states: California, Colorado, Louisiana, Minnesota, New York, Tennessee, Texas, and Washington, and collected data from each SEA’s website as well as contacted staffers about financial data. While strict generalizations cannot be drawn from eight states, it is important to note that their experiences are reflective of that in many other states and therefore provide insight for other SEAs, including Ohio’s.
Overall, the current state of SEAs’ capacities and workloads paints a grim picture in terms of their ability to be more involved with school improvement. Not surprisingly, almost 50 percent of the staff at SEAs serves in administrative roles (e.g., human resource, financial management, and technology). On the other hand, the distribution of personnel for performance and improvement ranges from 7 to 28 percent. Colorado and California dedicate less than 10 percent of SEA personnel to performance and improvement, while Texas and Tennessee dedicate more than 25 percent of their staff to these functions.
Largely, the investment in school improvement has been relatively minimal. But many states, via Race to the Top (RTTT), School Improvement Grants (SIGs), or their own school turnaround plans, have committed themselves to transforming the worst schools despite facing budget cuts or being short-staffed. So what can SEAs do to fulfill these school improvement duties? CRPE suggests that agencies shift current federally funded positions to school improvement, as well as contract out these services to consultants.
The Ohio Department of Education would do well to examine its capacity to lead school improvement efforts. As a recipient of RTTT and SIG funds it is more important than ever that they take an internal assessment of their capabilities and re-adjust were necessary to best serve the school improvement needs of the state.
The
Capacity for School Improvement: A First
Look at Agency Resources
Center
for Reinventing Public Education
Patrick
Murphay and Monica Quijdani
August
2011
