The Education Gadfly Weekly

Patrick Murphy, Michael DeArmond, and Kacey Guin, Education Policy Analysis Archives
July 2003

One of the timeless questions in education: Are we facing a teacher shortage? This new report from the Education Policy Analysis Archives uses data from the 1999-2000 federal Schools and Staffing Survey to show that the shortage is nowhere near as bad as everyone fears and that the problem is in fact concentrated in specific regions and subjects. To combat this spotty but still vexing situation, the authors recommend enticing the best teachers into the classrooms that need them most by giving them "combat pay" and other incentives ranging from signing bonuses to freedom from overbearing administrators. (Public Agenda recently concluded much the same in its report Stand by Me. See http://www.edexcellence.net/gadfly/issue.cfm?issue=108#274.) You can find the report at http://epaa.asu.edu/epaa/v11n23/.

Carol Ascher et al., Institute for Education and Social Policy
2003

A comprehensive look at New York state's charter school authorizing and accountability practices, this report is complete with everything from auditing timelines established by authorizing agents; to comparisons of test scores from regular public schools, schools under registration review, and charter schools; to office profiles of the authorizing agents. It helps shed light on New York's complicated double authorization system, which requires many charters to be authorized both by the State University of New York (or New York City) AND by the Board of Regents, leading to extra inspections and a plethora of administrators for relatively few schools. Mostly, it calls for more rigorous accountability and oversight by authorizers. See for yourself at http://www.nyu.edu/iesp/publications/charter/PDF-2003accountability.PDF. Or, for a briefer review, and to compare New York's authorizing practices with other states' methods, take a look at the recent Fordham study Charter School Authorizing: Are States Making the Grade? at http://www.edexcellence.net/doc/Full_report_no_embargo_notice.pdf.

David A. Goslin, The Scarecrow Press, Inc.
2003

"How is it," a Polish education leader asked the late Albert Shanker during a Warsaw conference on education reform in 1995, "that the United States is the world's preeminent democracy and economic power when your education system has all the problems and inequities you and your American colleagues have described over the past two days?" "Because," Shanker replied, "our top 20 percent are as good as any in the world, and that's all we have ever needed." Reading Engaging Minds triggered this memory because its underlying theme is that most American young people never live up to their full academic potential. According to veteran analyst Goslin, "only a small proportion of the nation's students--perhaps as few as 20 or 25 percent--are engaged in learning most of the time." To close the achievement gap and sustain measurable increases in academic performance, Goslin says, get students engaged, and keep them engaged in their own learning. Success in school is largely the result of "hard-work, perseverance, self-discipline, and respect for authority." Yet far too many children think that if something is hard, they simply don't have the ability to learn it, so it's OK to give up. Regrettably, many parents and teachers also buy into this mindset. What can be done? Goslin thinks the standards-based reform movement is a good start for it says that all children can and should meet high academic standards in school. This is a good first step, Goslin writes, but even more important is triggering a societal shift in thinking about education. In short, we all need to get off the couch, douse the TV and spend time helping our children learn. The ISBN is 0810847132 and you can get the book at http://www.scarecroweducation.com/Catalog/SingleBook.shtml?command=Search&db=^DB/CATALOG.db&eqSKUdata=0810847132.

Andrew J. Wayne and Peter Youngs, Review of Educational Research 
Spring 2003, Vol. 73, No. 1, pp. 89-122

This short but important paper reviews the existing research on teacher characteristics to determine what we know about which teachers are most effective. One finding, unfortunately, consistent with the recent ECS report [for Gadfly's review of the ECS report, go to http://www.edexcellence.net/gadfly/issue.cfm?issue=110#1383], is that there isn't a lot of reliable research on this topic. Among the relatively few studies that meet the authors' criteria--primarily, the use of value added analysis and proper controls--they do find some worthwhile insights. Correlates of teacher effectiveness include the quality of the teacher's undergraduate school and the teacher's performance on standardized tests (such as verbal skills or teacher licensure tests). Perhaps more interesting is their examination of degrees, coursework, and certification. With respect to the first two, only in mathematics has a reliable connection been found: Teachers with master's degrees in math do make better math teachers. (In other subjects, the connection has not been demonstrated.) With respect to certification, the lesson is that it's beneficial "only when teachers have certification for the subject taught." Notably absent from existing research is any solid evidence that race, years of experience, or holding an education degree do anybody any good. The implications seem clear: certification processes and hiring decisions need to incorporate what teachers know, and still more research is needed about which teacher characteristics actually help students learn. This paper provides a useful review of familiar ground. Sadly, it's not available online; to order a copy, call 800-521-0600.

The American Federation of Teachers is out with the 2002 edition of its Survey and Analysis of Teacher Salary Trends, and this year's report is even cheekier than usual. Acknowledging that, according to the union's own data, "average teacher salaries improved faster than inflation for the fourth time in five years," the authors demand yet more money for teacher pay. The AFT analysts admit that the teacher "shortage" today is spotty, largely confined to certain specialties and places. (For additional coverage of the teacher shortage question, see below.) What they keep harping on is that beginning teachers get salary offers that remain lower than those for college graduates in general.

But so much is left unsaid. Among the unremarked issues: that teachers have far more generous (and costly) benefits than most college graduates. That (at least after the first few years) teachers enjoy far greater job security than most college graduates. That teachers don't work as many days or hours and that their hourly compensation rate is competitive, if not superior to that of many other skilled fields requiring a college degree. (For a fine analysis of this phenomenon, see Richard Vedder's piece in the spring 2003 issue of Education Next.) Also unsaid--of course--is the folly of salary schedules that persist in paying the same to great instructors as to mediocre pedagogues; that pay the same regardless of whether an individual's teaching field is in shortage or surplus; and that pay the same (within a district anyway) whether one is teaching in a tough or pleasant school.

But it doesn't serve the union's interest to suggest that its members and would-be members are adequately compensated or that the basic arrangements by which they're compensated are screwy. So all of that goes unsaid. You can, however, get some useful numbers from this 75-page document, including state-by-state pay scales and metro-area pay and cost-of-living indices. You will also encounter a laughable effort (Appendix C) to demonstrate that unionized teachers earn more and that states with collective bargaining have higher test scores. Gadfly has insufficient room to debunk these wild claims other than to note that the averages mask huge variations. (Thinly unionized Georgia and North Carolina, for example, pay their teachers better than heavily unionized Iowa and Florida.) Have a look if you like. You can find it at http://www.aft.org/research/survey02/SalarySurvey02.pdf.

In June, Gadfly voiced some skepticism when the Department of Education announced with great fanfare that it had approved the NCLB accountability plans of all fifty states, as well as the District of Columbia, and Puerto Rico. We noted that the "victory" was at best pyrrhic, since many states cut corners to comply and at least one (Iowa) was approved despite lacking state standards. [See http://www.edexcellence.net/gadfly/issue.cfm?issue=2#57.] Well, we now learn that the Education Department has sent follow-up letters to numerous states detailing what they must do for their accountability plans to be "fully approved." In fact, only five state plans were fully approved as of July 1. In addition, some states that thought they were in compliance have had Title I money withheld because of continued problems with their plans. Minnesota, for example, was denied $113,000 because the feds declined to honor a March 2002 waiver that would have allowed the state to use attendance and graduation rates for middle and high schools to determine whether they were making adequate yearly progress. The bottom line is that, before anyone can declare victory, more work awaits.

"'Approved' is relative term for ed. department," by Lynn Olson, Education Week, August 6, 2003

"Minnesota schools: Feds yank $113,000 from state coffers," by John Welbes, Pioneer Press, August 2, 2003

A teacher takes Post education columnist Jay Mathews to task for his rankings of America's best public schools, which he bases on the number of students within a school who take Advanced Placement (AP) or International Baccalaureate (IB) classes. [See http://www.edexcellence.net/gadfly/issue.cfm?issue=23#93.] Tom Schaffer, a former AP teacher in Maryland, has engaged Mathews in a running email conversation on his experience with AP classes, which he believes "should be reserved for just A students and that most B and C students are not ready for such hard work in high school." According to Schaffer, the push to get low- and middle-achieving students into AP or IB classes has not only lowered the standards and atmosphere of those classes, but has also hurt kids who could be learning more in honors or regular classes than by throwing them into deeper educational waters than they're ready for. We think the world of Jay Mathews and his superb journalism, but Mr. Schaffer may have a point. Ranking schools on rates of participation in AP and IB courses seems to put the cart before the horse; wouldn't it be better to raise the standards of classes at every level, rather than just pushing larger numbers of students into AP and IB? And, shouldn't the ultimate distinguishing factor be how well students fare on those tests?

"AP courses not for everyone, educator says," by Jay Mathews, Washington Post, August 5, 2003

When hizzoner Michael Bloomberg gained control of Gotham's crippled school system, many had high hopes. Perhaps at last New York City would muster the strength to free itself from the establishment monopoly over education reform, curriculum, and pedagogy--and the innumerable underperforming schools it has created. Instead, Bloomberg and his chosen schools chancellor, Joel Klein, have moved in the opposite direction, embracing mandatory reading and math curricula rooted in the old-style progressive education ideology that created many of these problems in the first place. In their defense, Klein said, "I don't believe curriculums [sic] are the key to education. I believe teachers are." Well, yes. But teachers, even great ones, have to teach something. In New York, it's clearer with every passing day that the something they're teaching is the wrong thing.

"New York's new approach," by James Traub, New York Times, August 3, 2003

"One curriculum, many skeptics," by Mike McIntire, New York Times, August 6, 2003

On January 8, Indiana became one of five states singled out by the U.S. Department of Education for early approval of its No Child Left Behind accountability plan. These states were depicted as leaders that had set aside excuses and committed themselves to educating all students. Seven short months later, Indiana can, indeed, be characterized as a leader; but it is that state's role in obfuscating NCLB's goals that most deserves recognition.

In an earlier column [see http://www.edexcellence.net/gadfly/issue.cfm?issue=9#367], Checker Finn noted the acceleration of annual benchmarks at the core of Indiana's plan. For the next eight years, Hoosier schools will need, on average, to improve their pass rates on the state's test by seven percentage points every three years; but in the subsequent four years, seven percent gains are expected every single year.

Finn likened this to a "balloon mortgage" that will bequeath half of the state's expected improvement to a future generation of state and local education leaders. Such a balloon is especially unrealistic in a state like Indiana, where state test scores have shown no absolute improvement over the last 12 years.

Yet the "balloon mortgage" may not be the worst of Indiana's NCLB games. A more vexing obfuscation can be found in the state's treatment of expected pass rates on its tests. By applying an inflated "confidence test," state officials have substantially lowered performance expectations and assured that Indiana schools will never be held fully accountable for the 100 percent pass rate touted by NCLB.

Confidence tests are a common tool used in surveys and polls. In brief, they allow the results of a smaller sample of people to be extrapolated, with some degree of confidence, to the broader population. Since academic testing results are considered by many to provide only snapshots or estimates of actual student performance, confidence tests also may be used to identify a range of scores within which true performance actually falls.

Thus, the confidence test provides a "window" around the sample results to represent the area where real results are likely to be found. In public opinion polling, this window is indicated by the "margin of error" that is typically reported with polling data. If, for example, a poll finds that 55 percent of people support the president, and the margin of error--or window--is plus or minus three points, with a confidence level of 95 percent, that means we can be 95 percent confident that 52 to 58 percent of the population supports the president. Of course, wider windows create a higher degree of confidence--accompanied by less precision.

In standardized testing, margins of error are discussed in terms of standard deviations. Statistically speaking, a margin of error ranging one standard deviation above and below the sample results would provide a 68 percent degree of confidence that the actual results are found within that window. A window of two standard deviations would provide a 95 percent degree of confidence while three would yield a 99 percent degree of confidence.

In other words, the bigger the target, the more confident you can be of hitting it. The first of these options, one standard deviation, is quite common in statistical reporting. The second option, two standard deviations, is sometimes suggested for high stakes situations--like NCLB. The third option is extremely rare; yet that is what Indiana education officials opted for.

The results of this decision will have a huge impact on expected pass rates. The baseline pass rates reported this year to federal officials are 57.1 percent in math and 58.8 percent in English/language arts. But due to the large windows created by Indiana's 99 percent confidence test, the smallest groups of Indiana students--30 to 34--will need to reach just 36.7 percent passing levels in math and 40 percent in English/language arts in order to be said to have reached the (current) target. No group of students, no matter how large, will be required to achieve higher than 51.0 percent passing in math or 52.5 percent passing on English/language arts

These pass rates are substantially lower than those reported to the feds; and they also suggest that no school in Indiana, nor any subgroup, will ever be required to reach 100 percent passing--the ultimate goal touted by No Child Left Behind. That target will be lowered to the sill of a window that is three standard deviations wide--just as these initial benchmarks have been reduced.

Even the Council of Chief State School Officers--an organization that, until recently, was headed by Indiana's state superintendent--used only a 68 percent confidence test in its NCLB guidance for states and districts, while suggesting that states might consider the 95 percent test due to the high stakes nature of NCLB decisions. A 99 percent test, it seems, was not even contemplated. The chiefs' group also points to other statistical considerations such as minimum group sizes and multi-year averaging--also adopted by Indiana--as factors that could render NCLB results as invalid when combined with too high a confidence test.

This week's release of Indiana's AYP results [see http://www.doe.state.in.us/reed/newsr/2003August/Gov-NCLB-TitleI.pdf] suggests that these validity questions are, indeed, beginning to emerge. Despite a widely touted toughening of the rigor of Indiana's academic standards, the number of Title I schools identified as needing improvement actually declined this year from 156 to 117. Indeed, one of the schools highlighted by the state for coming off the improvement list actually saw its pass rate fall this year, from 55 percent to 43 percent. A valid accountability system would not have allowed this to happen.

One could hope that Indiana's state officials did not realize the impact of their actions. But using admirable language like "99 percent confidence" provides an awfully convenient cover; and several other manipulations of NCLB goals create a pattern that seems anything but accidental.

As another example, Indiana officials have also redefined some of NCLB's timing requirements. Several states worked hard this year to adjust their testing schedules so that school choice and supplemental services would be offered to qualified parents the year after their schools fail to meet adequate yearly progress, as demanded in NCLB guidelines. But Indiana officials ignored that demand, which means such consequences will be deferred up to one full year for Hoosier youngsters.

The camouflage here is Indiana's fall testing schedule. Given in September, each test actually measures the progress that students made during the previous school year. The test given to 6th graders next month, for example, will actually measure last year's progress against 5th grade standards, the 8th grade test will measure progress against 7th grade standards, and so on.

The results of these tests are returned mid-school year. Thus, Indiana officials will allow schools to delay providing mandatory supplemental services by one-half year and school choice by one full year. In other words, a school that fails repeatedly to make sufficient progress for a specific group of 5th graders will not be required to provide choice until those same students are in 7th grade.

Indiana officials have also played fast and loose with the definition of "dropout." After promising the feds in January that lawmakers would fix the state's flawed graduation reporting, the legislature actually worsened the problem by excluding from the dropout definition "students who have left school and whose location cannot be determined." That would seem to omit an awfully large percentage of what most people think of as dropouts.

There are other issues with Indiana's NCLB implementation, some of them shared with other states. For example, Indiana has no inter-district public school choice law--despite federal warnings that NCLB compliance would likely require such flexibility. Also, Indiana has declared that subgroups of less than 40 pupils need not abide by the law's requirement that 95 percent of all students must take the state tests, thus allowing substantial gamesmanship in the exclusion of students. And like lots of other states, Indiana has interpreted the phrase "persistently dangerous school" in such a ludicrously narrow way that no school is likely ever to get that label. (For California's similar gambit, see http://www.edexcellence.net/gadfly/issue.cfm?issue=26#65.)

Federal officials may well have been unaware of these games when they spotlighted Indiana as a model state. Indeed, among the issues noted above, only the "balloon mortgage" was described explicitly in federal application materials. Other games have surfaced in locally distributed documents or were cloaked in vague terms.

Yet this handful of examples from a single state must raise questions about other games that might emerge from states attempting to avoid new accountability measures under NCLB. Observers would be wise to keep careful watch.

Derek Redelman is a senior fellow and Director of Education Policy at Hudson Institute in Indianapolis. He can be reached at [email protected].

Though the phrase "don't know much about history" is now a clich?, we can't argue with the sentiments of this article in National Journal, which argues that the well-intentioned No Child Left Behind act is perversely marginalizing subjects like history. History already had plenty of problem arising for its submergence into the nebulous field of "social studies." This latest development must surely accelerate the historical and civic illiteracy of American school kids. So, is the answer to add these subjects to the NCLB requirements? In the long run, perhaps, but, as Diane Ravitch aptly remarks in this article, "Basic skills are not enough, but if you don't have the basic skills, you can't learn history or science." For now, a focus on the basics may be necessary, but it would be a catastrophe if that meant history and other subjects are forever shunted aside. Note, though, that NCLB requirements are meant to be a floor, not a ceiling. Alien as this may sound to those grumping about compliance, a school or school system or state is free to do more than the law requires. Stay tuned. Fordham will soon several reports addressing this issue, beginning with an examination of where social studies went wrong and some cogent thoughts on what children need to learn about terrorists, despots, and democracy.

"Don't know much about history," by Brian Friel, National Journal, August 2, 2003 (available to online subscribers only)

Carol Ascher et al., Institute for Education and Social Policy
2003

A comprehensive look at New York state's charter school authorizing and accountability practices, this report is complete with everything from auditing timelines established by authorizing agents; to comparisons of test scores from regular public schools, schools under registration review, and charter schools; to office profiles of the authorizing agents. It helps shed light on New York's complicated double authorization system, which requires many charters to be authorized both by the State University of New York (or New York City) AND by the Board of Regents, leading to extra inspections and a plethora of administrators for relatively few schools. Mostly, it calls for more rigorous accountability and oversight by authorizers. See for yourself at http://www.nyu.edu/iesp/publications/charter/PDF-2003accountability.PDF. Or, for a briefer review, and to compare New York's authorizing practices with other states' methods, take a look at the recent Fordham study Charter School Authorizing: Are States Making the Grade? at http://www.edexcellence.net/doc/Full_report_no_embargo_notice.pdf.

David A. Goslin, The Scarecrow Press, Inc.
2003

"How is it," a Polish education leader asked the late Albert Shanker during a Warsaw conference on education reform in 1995, "that the United States is the world's preeminent democracy and economic power when your education system has all the problems and inequities you and your American colleagues have described over the past two days?" "Because," Shanker replied, "our top 20 percent are as good as any in the world, and that's all we have ever needed." Reading Engaging Minds triggered this memory because its underlying theme is that most American young people never live up to their full academic potential. According to veteran analyst Goslin, "only a small proportion of the nation's students--perhaps as few as 20 or 25 percent--are engaged in learning most of the time." To close the achievement gap and sustain measurable increases in academic performance, Goslin says, get students engaged, and keep them engaged in their own learning. Success in school is largely the result of "hard-work, perseverance, self-discipline, and respect for authority." Yet far too many children think that if something is hard, they simply don't have the ability to learn it, so it's OK to give up. Regrettably, many parents and teachers also buy into this mindset. What can be done? Goslin thinks the standards-based reform movement is a good start for it says that all children can and should meet high academic standards in school. This is a good first step, Goslin writes, but even more important is triggering a societal shift in thinking about education. In short, we all need to get off the couch, douse the TV and spend time helping our children learn. The ISBN is 0810847132 and you can get the book at http://www.scarecroweducation.com/Catalog/SingleBook.shtml?command=Search&db=^DB/CATALOG.db&eqSKUdata=0810847132.

Patrick Murphy, Michael DeArmond, and Kacey Guin, Education Policy Analysis Archives
July 2003

One of the timeless questions in education: Are we facing a teacher shortage? This new report from the Education Policy Analysis Archives uses data from the 1999-2000 federal Schools and Staffing Survey to show that the shortage is nowhere near as bad as everyone fears and that the problem is in fact concentrated in specific regions and subjects. To combat this spotty but still vexing situation, the authors recommend enticing the best teachers into the classrooms that need them most by giving them "combat pay" and other incentives ranging from signing bonuses to freedom from overbearing administrators. (Public Agenda recently concluded much the same in its report Stand by Me. See http://www.edexcellence.net/gadfly/issue.cfm?issue=108#274.) You can find the report at http://epaa.asu.edu/epaa/v11n23/.

Andrew J. Wayne and Peter Youngs, Review of Educational Research 
Spring 2003, Vol. 73, No. 1, pp. 89-122

This short but important paper reviews the existing research on teacher characteristics to determine what we know about which teachers are most effective. One finding, unfortunately, consistent with the recent ECS report [for Gadfly's review of the ECS report, go to http://www.edexcellence.net/gadfly/issue.cfm?issue=110#1383], is that there isn't a lot of reliable research on this topic. Among the relatively few studies that meet the authors' criteria--primarily, the use of value added analysis and proper controls--they do find some worthwhile insights. Correlates of teacher effectiveness include the quality of the teacher's undergraduate school and the teacher's performance on standardized tests (such as verbal skills or teacher licensure tests). Perhaps more interesting is their examination of degrees, coursework, and certification. With respect to the first two, only in mathematics has a reliable connection been found: Teachers with master's degrees in math do make better math teachers. (In other subjects, the connection has not been demonstrated.) With respect to certification, the lesson is that it's beneficial "only when teachers have certification for the subject taught." Notably absent from existing research is any solid evidence that race, years of experience, or holding an education degree do anybody any good. The implications seem clear: certification processes and hiring decisions need to incorporate what teachers know, and still more research is needed about which teacher characteristics actually help students learn. This paper provides a useful review of familiar ground. Sadly, it's not available online; to order a copy, call 800-521-0600.