The Education Gadfly Weekly

Cassandra M.D. Hart & David Figlio
Education Next
Winter 2010

This study by Northwestern University economists—published in Education Next—teases out findings to a “notoriously difficult” question: do public schools improve when they face the threat of losing students to nearby private schools (in this case, precipitated by tax credit scholarships)? This theoretical “competitive effect” is a main tenet of the school choice movement, but it’s difficult to substantiate that the education market actually works this way. Figlio and Hart’s study bolsters this school choice principle, as it finds that the availability of tax credit scholarships in Florida led to improvements in the average performance of “at-risk” public schools (those that would lose eligible students).

Figlio and Hart examined Florida state test results (FCAT scores) from 1999 to 2007 as well as student demographic variables, and geo-coded data for public and private schools so as to measure the distance, density, and concentration of private school options within a five-mile radius of Florida’s public schools. (They also measured private school diversity, naming ten different types on a religious-secular spectrum.) Next, they measured the effect of “scholarship-induced private school competition” on nearby public schools by comparing student performance before and after the scholarship program was enacted. Meanwhile, they controlled for demographic variables like race and poverty, as well as state-given grades to schools, thus isolating the effects of private school competition and disentangling it from, say, schools’ behavior to avoid accountability sanctions.

The four measures of competition in the study – distance, density, concentration, and diversity – enabled a comparison between schools with varying levels and diversity of private school options. Remarkably, all four measures were positively correlated with student performance in reading and math. Public schools with more, and more varied, private school options nearby experienced small but consistently positive achievement gains in both subjects after the scholarship program was launched in 2001. Further, the performance gains were lasting, and most pronounced in elementary and middle schools (more likely to lose students than high schools) and in schools wherein losing low-income students to tax credit-receiving competitors would put Title I funding at risk.

The study has limits, namely that the school choice market has evolved much since 2001, and Florida-specific findings may not be generalize-able to other states. But the report’s findings have important implications for Florida, for states looking to recreate tax scholarship programs, and for the choice movement itself. Read it here.

Center for Reinventing Public Education
Christine Campbell
November 2010

Kudos to CRPE for its new report (from its National Charter School Research Project) shedding much needed light on an issue critical to the long-term sustainability of charter schools, yet rarely addressed: succession planning. As an authorizer of charter schools, Fordham has seen firsthand how acutely a change in leadership can affect the success or failure of a charter school.

Succession planning isn’t at the forefront of many board agendas, yet it’s just as crucial to a charter school’s viability as other issues that typically garner lots of attention (e.g., academics, fundraising, facilities, and budgets). This report finds a 20 percent turnover rate over two years among the 24 schools it studied. Despite significant turnover, 14 of the 24 schools studied had no succession plan at all. Of the 10 schools that purported to have a plan in place, only five were considered substantive. 

More important than the numbers, though, are the questions that charter school governing boards and school leaders should consider in order to strengthen their organizations. For example, do boards with strong school founders recognize organizational weaknesses/skills the founder doesn’t have? Are the school leader and board - and management company, for that matter - clear on whose responsibility the succession plan is? (It’s the governing board’s responsibility.) Does the school have the bench strength and training capabilities to produce a new leader from within, or does the leader need to come from outside? And, does the new leader need to perform the exact functions as the former leader, or has the school organization changed such that new duties associated with the change in leadership are merited?

The report points out that succession plans need not be complex, and that strong plans:

• Understand where the school is headed, as well as the school’s strengths and weaknesses;
• Account for emergency and longer term circumstances;
• Outline when/how to communicate with staff and train them when the time is right (or ensure job descriptions for outside candidates are updated and fully representative of the duties of the position); and,
• Clarify that final decisions rest with, and are the responsibility of, the school’s board. 

Finally, the report recommends that authorizers make school succession planning part of the charter school application and renewal process. This report is a must read for charter school practitioners -- boards, school leaders, management companies, and authorizers alike.

Public Impact
Julie Kowal and Emily Ayscue Hassel
November 2010

There is undoubtedly no other sector in which talent and performance are more important than in education.  Teachers have a tremendous impact on student learning and improving teacher effectiveness is one of the most important levers in improving student achievement.  Thus, it is imperative that schools leaders have an adequate and efficient way to gauge teacher effectiveness.  This   recent report from Public Impact provides thoughtful insight in this area by comparing performance measurement across sectors, including non-profit organizations and private companies.  Through research of numerous organizations and companies, the report compiles six components necessary for meaningful performance evaluations.

1. Determine the purpose of the performance measurements.  Determine if the measurements are going to be used for planning purposes, for retention or dismissal of employees, or for the purpose of learning.  This is a crucial step because the rest of the design hinges upon this.
2. Choose objectives that align with the organization’s mission.  In education, performance measurements should align with the school’s mission and purpose.
3. Design performance measurements.  Each employee should know their role in the performance review process and what actions they must take to achieve results. 
4. Set performance standards.  Standards must be established in order to understand what a fully satisfied performance measurement looks like.
5. Adopt a performance measurement process.  Leaders must determine who will have input in the evaluation process and how often evaluations will occur. 
6. Use measurement results to take action.  Once the purpose of the performance measurements has been established and the design complete, administrators should use the results to make decisions around hiring, firing, promotion, and retention for their organization. 

In addition to the above steps, Public Impact offers other recommendations to improve performance measurements.  The report suggests that schools measure performance frequently, measure both the “what” and “how” of performance, and use the evaluations to not only improve teacher quality, but also to decide who teaches and how long they are able to teach.  Education leaders must create effective performance mechanisms and use the evaluations as the basis for teacher and principal retention, dismissal, and development in order to provide quality education to all students.  This report provides useful information toward that end.  Read it here.

American Institutes for Research (AIR)
Gary W. Phillips
October 2010 

Analyses comparing US students’ achievement with that of their global peers are widespread (and discouraging), but little has been said about how US educational standards compare to standards in other nations. This new AIR study does just that.

To make the comparison, Phillips begins with the metric the National Center for Education Statistics uses to evaluate state assessments (assumed to be representative of state standards) in relation to the National Assessment of Educational Progress (NAEP). He then uses this NAEP-based evaluation as a bridge to measure state assessments against the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) (two studies assumed to be representative of international standards). Phillips determines that across the grade levels studied (grades four and eight), less than four percent of states have standards on par with or higher than the TIMSS and PIRLS standards in mathematics and reading, respectively. 

Ohioans may be interested to learn that on an A-F scale in which a “B” represents meeting or exceeding international standards, the report gives Ohio’s standards a C for fourth grade mathematics and reading, and a C- for eighth grade mathematics. (The study does not measure grade eight reading.) These “grades” put Ohio slightly below the national average in all subjects and grade levels studied. The study also posits that low testing standards have caused Ohio, among other states, to report artificially high proficiency numbers. For example, although the Ohio Department of Education (ODE) reported that 81.7 percent of Ohio’s grade four students were proficient in reading in 2007, Phillips estimates that only 50 percent would have been proficient by an international standard. (Earlier this year, Fordham reported a similar discrepancy between Ohio students’ scores on the Ohio Achievement Test scores and their scores on the NAEP.)

Considering that Ohio has recognized the inadequacy of its own current standards and has recently adopted the Common Core standards in English language arts and math, the study is more helpful in evaluating where Ohio has been rather than where we are going. Still, it provides a useful comparison to other nations, one that might be worth replicating in the future. Read the full study here.

This national survey of education school professors finds that, even as the U.S. grows more practical and demanding when it comes to K-12 education, most of the professoriate simply isn't there. They see themselves more as philosophers and agents of social change, not as master craftsmen sharing tradecraft. They also resist some promising reforms such as tying teacher pay to student test scores. Still, education professors are reform-minded in some areas, including tougher policies for awarding tenure to teachers and financial incentives for those who teach in tough neighborhoods. Read on to find out more.

Press Release

Much has been written about spectacular charter school blow-ups that have cost taxpayers hundreds of thousands of dollars. Rarely, if ever, do we see headlines that read ???Closed charter school returns $423,421 to state's coffers.??? But that's exactly what happened yesterday when the treasurer for the East End Community School in Dayton hand delivered a check to the Ohio Department of Education. Fordham served as the school's authorizer from 2005 to 2008, when it closed at the end of the school year.

East End had facility problems since its inception in 2002. When the church in which it rented space terminated that arrangement, the school had nowhere satisfactory to conduct class. In its final year of operation the school was one of Dayton's highest performing schools with its students' reading proficiency scores fully 15 points above those of district students. In 2008 the school was rated Continuous Improvement (C) and met AYP. Despite its achievement and an enrollment of 210 students the school couldn't function without a decent facility.

And so, in May 2008, the school's governing board reached an agreement with the Dayton Public School district whereby East End would cease operating as an independent charter and its pupils would be encouraged to enroll in the district's newly built Ruskin Elementary School. Many teachers and staff from East End subsequently took positions in the new school. This arrangement allowed the district to fill an otherwise underutilized new building with students while enabling East End's pupils to move into a suitable facility. Further, community leaders in East Dayton ??? some of whom were on the board of East End ??? wanted to support one school rather than two competing schools. It was sad to see East End close and for our partnership to end, but we understood and agreed with the logic of the school's governing authority. As a good authorizer should, we helped them in the closure and provided various forms of technical assistance along the way.

More than two years later it is a remarkable testament to the school's governing board members, its leadership, and its treasurer that after closing out all costs and expenses, including two state-required audits, the school was able to return nearly half a million dollars to the State of Ohio. The school not only delivered academic results for its students during its six-year run, but it actually did this while building up a cash surplus. This is a remarkable story of how thoughtful community leaders can serve children and their neighborhood well through charter schools, while also providing a bargain to taxpayers.

- Terry Ryan

Taxpayers invest a lot in their teachers, and good ones are worth every penny. Nothing affects student performance more than great teachers. Conversely, weak teachers can do irreparable damage to children and their learning. This alone should prompt Ohio to glean as much information as possible about teacher effectiveness.

But in the face of Ohio's impending budget cliff and the teacher layoffs it will cause, defining teacher effectiveness has become that much more urgent. Consider two pots of federal money that have propped up Ohio's education spending: the 2009 American Recovery and Reinvestment Act and the August 2010 infusion of "Ed Jobs" money. Ohio received nearly a billion dollars for education from the recovery act - funding that dries up in July 2011 - which saved or created upward of 9,000 education jobs. Ed Jobs funding, to expire in 2012, funneled $361 million to Ohio and saved an estimated 5,000 teaching jobs. To say that layoffs will occur en masse is an understatement. Ohio must come up with strategies to keep the most effective teachers in classrooms.

We actually know little about the effectiveness of teachers in the Buckeye State. Current teacher evaluations do not distinguish highly effective teachers from the rest, nor do they weed out poor performers. Further, archaic human-resources practices in public education prevent us from retaining, rewarding and supporting teachers based on their effectiveness. In fact, we pay long-serving, but ineffectual, teachers more than we pay less-senior high-fliers. We reward teachers for their credentials and advanced degrees, but offer the same pay for teachers whether their students thrive or languish. We lay off teachers based solely on seniority.

Since 2007, Ohio has collected value-added data in both reading and mathematics that can be used to help determine teacher performance. These data already are used to measure school and district performance, and play a key role in determining whether persistently low-performing charter schools should face automatic closure. Further, the well-regarded Battelle for Kids has been doing excellent work to help educators use value-added data as a diagnostic tool for improving instruction. So, the data and its power to inform decision-making are not foreign to educators or policy makers in Ohio.

It is time to start using value-added data as a key component of teacher evaluations. With budget shortfalls and subsequent teacher layoffs looming, district leaders need the tools to dismiss or furlough their least productive teachers while keeping their most successful ones. Although not perfect, the best metric for measuring teacher effectiveness is value-added data, and this works in the same fashion as the state's current value-added model for assessing school performance.

According to a recent report by the Brookings Institute, if student test achievement is the outcome, value-added is superior to other existing methods of classifying teachers. Classifications that rely on other measurable characteristics of teachers, considered singly or in aggregate, are not in the same league in terms of predicting future performance as evaluation based on value-added.

Critics argue that since these systems aren't perfect, or since they currently apply only to math and reading in grades 4-8, that they shouldn't be used to measure teacher effectiveness. Such logic is upside down. As value-added measures of teacher effectiveness are the best tool currently available to school districts that teach math and reading (core subjects pivotal to success in other subjects), this is the best time to accelerate adopting them as central components of modern teacher evaluation systems. Value-added measures should be supplemented by other factors when it comes to making decisions about teacher dismissals, tenure, remuneration or school placement.

As Ohio begins debate on how to tackle a historic budget deficit, it is imperative that school districts have the tools necessary for ensuring that they keep their best teachers. Value-added data for gauging teacher effectiveness must be part of the conversation.

This op-ed previously appeared in the Columbus Dispatch.

Center for Reinventing Public Education
Christine Campbell
November 2010

Kudos to CRPE for its new report (from its National Charter School Research Project) shedding much needed light on an issue critical to the long-term sustainability of charter schools, yet rarely addressed: succession planning. As an authorizer of charter schools, Fordham has seen firsthand how acutely a change in leadership can affect the success or failure of a charter school.

Succession planning isn’t at the forefront of many board agendas, yet it’s just as crucial to a charter school’s viability as other issues that typically garner lots of attention (e.g., academics, fundraising, facilities, and budgets). This report finds a 20 percent turnover rate over two years among the 24 schools it studied. Despite significant turnover, 14 of the 24 schools studied had no succession plan at all. Of the 10 schools that purported to have a plan in place, only five were considered substantive. 

More important than the numbers, though, are the questions that charter school governing boards and school leaders should consider in order to strengthen their organizations. For example, do boards with strong school founders recognize organizational weaknesses/skills the founder doesn’t have? Are the school leader and board - and management company, for that matter - clear on whose responsibility the succession plan is? (It’s the governing board’s responsibility.) Does the school have the bench strength and training capabilities to produce a new leader from within, or does the leader need to come from outside? And, does the new leader need to perform the exact functions as the former leader, or has the school organization changed such that new duties associated with the change in leadership are merited?

The report points out that succession plans need not be complex, and that strong plans:

• Understand where the school is headed, as well as the school’s strengths and weaknesses;
• Account for emergency and longer term circumstances;
• Outline when/how to communicate with staff and train them when the time is right (or ensure job descriptions for outside candidates are updated and fully representative of the duties of the position); and,
• Clarify that final decisions rest with, and are the responsibility of, the school’s board. 

Finally, the report recommends that authorizers make school succession planning part of the charter school application and renewal process. This report is a must read for charter school practitioners -- boards, school leaders, management companies, and authorizers alike.

Public Impact
Julie Kowal and Emily Ayscue Hassel
November 2010

There is undoubtedly no other sector in which talent and performance are more important than in education.  Teachers have a tremendous impact on student learning and improving teacher effectiveness is one of the most important levers in improving student achievement.  Thus, it is imperative that schools leaders have an adequate and efficient way to gauge teacher effectiveness.  This   recent report from Public Impact provides thoughtful insight in this area by comparing performance measurement across sectors, including non-profit organizations and private companies.  Through research of numerous organizations and companies, the report compiles six components necessary for meaningful performance evaluations.

1. Determine the purpose of the performance measurements.  Determine if the measurements are going to be used for planning purposes, for retention or dismissal of employees, or for the purpose of learning.  This is a crucial step because the rest of the design hinges upon this.
2. Choose objectives that align with the organization’s mission.  In education, performance measurements should align with the school’s mission and purpose.
3. Design performance measurements.  Each employee should know their role in the performance review process and what actions they must take to achieve results. 
4. Set performance standards.  Standards must be established in order to understand what a fully satisfied performance measurement looks like.
5. Adopt a performance measurement process.  Leaders must determine who will have input in the evaluation process and how often evaluations will occur. 
6. Use measurement results to take action.  Once the purpose of the performance measurements has been established and the design complete, administrators should use the results to make decisions around hiring, firing, promotion, and retention for their organization. 

In addition to the above steps, Public Impact offers other recommendations to improve performance measurements.  The report suggests that schools measure performance frequently, measure both the “what” and “how” of performance, and use the evaluations to not only improve teacher quality, but also to decide who teaches and how long they are able to teach.  Education leaders must create effective performance mechanisms and use the evaluations as the basis for teacher and principal retention, dismissal, and development in order to provide quality education to all students.  This report provides useful information toward that end.  Read it here.

Cassandra M.D. Hart & David Figlio
Education Next
Winter 2010

This study by Northwestern University economists—published in Education Next—teases out findings to a “notoriously difficult” question: do public schools improve when they face the threat of losing students to nearby private schools (in this case, precipitated by tax credit scholarships)? This theoretical “competitive effect” is a main tenet of the school choice movement, but it’s difficult to substantiate that the education market actually works this way. Figlio and Hart’s study bolsters this school choice principle, as it finds that the availability of tax credit scholarships in Florida led to improvements in the average performance of “at-risk” public schools (those that would lose eligible students).

Figlio and Hart examined Florida state test results (FCAT scores) from 1999 to 2007 as well as student demographic variables, and geo-coded data for public and private schools so as to measure the distance, density, and concentration of private school options within a five-mile radius of Florida’s public schools. (They also measured private school diversity, naming ten different types on a religious-secular spectrum.) Next, they measured the effect of “scholarship-induced private school competition” on nearby public schools by comparing student performance before and after the scholarship program was enacted. Meanwhile, they controlled for demographic variables like race and poverty, as well as state-given grades to schools, thus isolating the effects of private school competition and disentangling it from, say, schools’ behavior to avoid accountability sanctions.

The four measures of competition in the study – distance, density, concentration, and diversity – enabled a comparison between schools with varying levels and diversity of private school options. Remarkably, all four measures were positively correlated with student performance in reading and math. Public schools with more, and more varied, private school options nearby experienced small but consistently positive achievement gains in both subjects after the scholarship program was launched in 2001. Further, the performance gains were lasting, and most pronounced in elementary and middle schools (more likely to lose students than high schools) and in schools wherein losing low-income students to tax credit-receiving competitors would put Title I funding at risk.

The study has limits, namely that the school choice market has evolved much since 2001, and Florida-specific findings may not be generalize-able to other states. But the report’s findings have important implications for Florida, for states looking to recreate tax scholarship programs, and for the choice movement itself. Read it here.

American Institutes for Research (AIR)
Gary W. Phillips
October 2010 

Analyses comparing US students’ achievement with that of their global peers are widespread (and discouraging), but little has been said about how US educational standards compare to standards in other nations. This new AIR study does just that.

To make the comparison, Phillips begins with the metric the National Center for Education Statistics uses to evaluate state assessments (assumed to be representative of state standards) in relation to the National Assessment of Educational Progress (NAEP). He then uses this NAEP-based evaluation as a bridge to measure state assessments against the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) (two studies assumed to be representative of international standards). Phillips determines that across the grade levels studied (grades four and eight), less than four percent of states have standards on par with or higher than the TIMSS and PIRLS standards in mathematics and reading, respectively. 

Ohioans may be interested to learn that on an A-F scale in which a “B” represents meeting or exceeding international standards, the report gives Ohio’s standards a C for fourth grade mathematics and reading, and a C- for eighth grade mathematics. (The study does not measure grade eight reading.) These “grades” put Ohio slightly below the national average in all subjects and grade levels studied. The study also posits that low testing standards have caused Ohio, among other states, to report artificially high proficiency numbers. For example, although the Ohio Department of Education (ODE) reported that 81.7 percent of Ohio’s grade four students were proficient in reading in 2007, Phillips estimates that only 50 percent would have been proficient by an international standard. (Earlier this year, Fordham reported a similar discrepancy between Ohio students’ scores on the Ohio Achievement Test scores and their scores on the NAEP.)

Considering that Ohio has recognized the inadequacy of its own current standards and has recently adopted the Common Core standards in English language arts and math, the study is more helpful in evaluating where Ohio has been rather than where we are going. Still, it provides a useful comparison to other nations, one that might be worth replicating in the future. Read the full study here.