Since 2003, the Thomas B. Fordham Institute has been analyzing the academic performance of schools in our hometown of Dayton and in other Ohio cities. We continued that analysis this year, taking a close look at the local report card data released by the Ohio Department of Education yesterday (see here).
A little history
During the six years we've been analyzing this data we have seen the evolution of the state's assessment and reporting system and have come to appreciate how rich the information it provides is. When we first started to examine student achievement data in August 2003, Governor Taft had just signed into law House Bill 3 (HB3) which put Ohio into compliance with the federal No Child Left Behind law. Prior to HB3, Ohio students in grades four, six, and nine only were required to take proficiency tests in five areas (reading, mathematics, writing, science, and citizenship).
In 2004, the state moved toward testing all students in grades three through eight with achievement tests in reading, mathematics, writing, science, and social studies, and in high school via the Ohio Graduation Test. The state's testing system was rolled out over five years, and by 2008 it was fully operational across all grades and subjects tested (see here).
In August 2008, Ohio became one of a handful of states that not only evaluated students' performance against academic standards, but also measured academic growth over time through a system called "value-added analysis." This system provides another data set for schools that serve students in grades four through eight. With the implementation of value-added analysis we can assess student learning in two broad ways in Ohio for elementary and middle schools: by measuring what students know and can do at a particular point in time (their "achievement" or "proficiency"), and by measuring how much students have improved (their "progress" or "growth") over a period of time. (See Fordham's value-added primer here).
This year's results
In collaboration with Bryan Hassel and Jacob Rosch of the policy and research organization Public Impact, we analyzed and compared the student achievement of charter school students and district school students in Ohio's eight major urban cities (Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo, and Youngstown). These cities are still where most brick-and-mortar charter school students attend school. Separately, Public Impact compared the performance of charter e-schools (also called virtual schools) with that of non-charter public schools statewide (report will be available here later this week).
The story that emerges from the data for brick-and-mortar charter schools is a "bad news/good news" tale. Let's start with the bad news first: the data trend shows us that schools (elementary, middle, and secondary) in the state's largest cities continue to struggle mightily to help students meet basic academic standards, and are nowhere close to achieving the goals set by the state of Ohio or by the federal No Child Left Behind Act. This news is not surprising but is painful nonetheless.
For example, charts 1 and 2 shows that in both charter and district schools serving children in Big 8 cities, only about 57 percent of students are proficient in reading while only about half are proficient in math. Even more disconcerting is that after initial gains achieved in the years immediately following the passage of NCLB, achievement in reading in math for both district and charter schools has largely flat-lined. There is little difference here between charters and district school performance, and sadly both fail to ensure that students achieve the state goal of "proficiency."
Chart 1. Urban Charter School vs. Ohio 8 District School Performance Over Time in Reading
Source: Ohio Department of Education interactive Local Report Card; see methodology in full report here.
Chart 2. Urban Charter School vs. Ohio 8 District School Performance Over Time in Math
Source: Ohio Department of Education interactive Local Report Card; see methodology in full report here.
Further, in looking at the performance ratings of Big 8 charter and district schools we see that about half of the schools were rated Academic Watch (a D) or Academic Emergency (an F) in 2008-09. District officials may take some solace in the fact that only 25 percent of their schools were rated F, while charters had 36 percent of their schools in the lowest rating. But the fact remains that over 125,000 children in the Big 8 cities are in schools rated by the state as failing or on the verge of it. This represents a failure for the state of Ohio and all of us who work in education and care about closing the achievement gap.
Now, to the good news--and fortunately there is some of it in this data. When it comes to student growth in Big 8 schools that serve grades four through eight and have value-added data (504 schools total), 49 percent of charter schools and 54 percent of district schools exceeded expected growth in 2008-09. A further 32 percent of charters and 31 percent of district elementary and middle schools in the Big 8 met expected growth. This means that only 19 percent of charter schools and 15 percent of district schools failed to deliver at least a year's worth of academic progress last year.
In practice, this means that about half of the public schools in the Big 8 cities that serve grades four through eight made some progress in 2008-09 at closing the achievement gap. Unfortunately, the gap is still vast. Chart 3 tells the story in a picture. It shows that most schools, charter and district, that received value-added ratings, met or exceeded academic growth, but only 20 out of 504 in total earned a Performance Index score of 100 or higher. The state considers a school successful if it attains a Performance Index score of 100 or better on a scale of 0 to 120 (see more here). Ohio has 1,134 school buildings with a PI score at or above 100, but only 34 of these are in the Big 8 cities.
Chart 3. 2008-09 Academic Performance of Ohio 8 District and Charter Schools
Source: Ohio Department of Education interactive Local Report Card
Notes: Random variance was used to plot schools horizontally within each value-added rating. This chart includes charter and district schools in the Big 8 cities for which a 2008-09 Performance Index score and value-added rating are available (charter N=123; district N=381).
In sum, Ohio has constructed a fairly rigorous assessment and accountability system that gives analysts, policymakers, journalists, educators, and parents a sophisticated way to look at student achievement and growth. The system makes obvious that the schools (charter and district) serving some of our neediest children in the state's larger cities are struggling mightily to deliver on the goal of helping their students reach academic proficiency in reading and math.
Yet, the system also tells us that many of the elementary and middle schools in the Big 8 cities are making gains with their students, and that learning is happening in these schools. This is an important story to tell. Further, as Ohio moves to redesigning its assessment system in the future (the state's recently passed budget requires changes to the system in 2011 and beyond) it should do so with great caution. There are excellent data and trend lines now available that we should work to keep and build on rather than snuff out.
Other key findings from this year's analysis include:
- More established charter schools, those eight years or older, are far more likely to be both high achieving and high performing than are younger charter schools.
- In the Big 8 as a group, district schools outperformed charter schools in every subject.
- In two cities, Cleveland and Dayton, charter students outperformed district students in both terms of academic achievement and in terms of value-added growth.
- In Dayton, four of the city's top five rated schools in 2009 were charters and 12 of 20 of the top rated schools were charters.
- In Columbus and Toledo, the district schools outperformed the charter schools in both overall student achievement and in value-added growth.
- Statewide student performance outpaced the performance of student in e-schools in all tested subjects.
See our full analyses online here.