Ohio’s new teacher-evaluation system requires evaluators to conduct two, formal thirty-minute classroom observations. Yet these legally prescribed observations seem ripe for compliance and rote box-checking; in fact, they may not be quite the impetus for school-wide improvement that policymakers had hoped for.
If this does end up happening in practice, all is not lost. Rather, as I discuss below, informal channels for teacher feedback might actually be more conducive to helping teachers (and their schools) improve than formal procedures.
Consider Will Dobbie and Roland Fryer’s recent work on New York City’s charter schools. The research duo takes great pains to uncover what school-based factors make a great school tick. In my estimation, one of their key findings is how strongly the frequency of informal teacher feedback correlates to school effectiveness.
Dobbie and Fryer measure school effectiveness in two ways. For the full sample of thirty-nine schools, they use a statistical model (a matched student-pair approach) to estimate a school’s impact on achievement. Second, for twenty-nine of the schools, lottery-admissions data were used to estimate school effectiveness. Lottery-based computations are typically considered preferable, because researchers can approximate a random experiment. The researchers then probe the schools’ “inner-workings” during the 2010-11 school year, to gauge which school-based factors differentiate higher- and lower-performing schools.
The study concludes that a “bundle” of practices and attitudes—generally those associated with a “No Excuses” charter-school model—are linked with more-effective schools. Overall, this might be expected, given the powerful research findings on KIPP charters and Boston’s charter schools. They also find that conventional inputs such as class size, per-pupil expenditures, and teachers’ advanced degrees and certification are not systemically linked to higher performance.
Yet reading Dobbie and Fryer’s findings I have become convinced that, among the individual factors examined (separate of the “bundle” of factors),[1] the frequency of informal teacher feedback appears to be one of the strongest marks of an excellent school.
Consider the following points:
When examining the “frequent teacher feedback” variable—one of the five stand-alone variables in the study’s correlation analyses—there is virtually no difference in the amount of formal feedback teachers receive in higher- versus lower-performing schools.[2] For instance, elementary teachers in more-effective schools received, on average, 3.12 instances of formal feedback per semester, compared to 2.73 in other elementary schools. That difference was slight and insignificant.
However, there were significant differences in the frequency of informal feedback. In higher-performing elementary schools, typical teachers received 12.75 instances of informal feedback per semester, compared to just 7.50 in less-effective schools. Meantime, in high-performing middle schools, teachers received, on average, 13.30 instances of informal feedback versus a paltry 4.12 instances in other schools.
The key difference, then, within the “frequent teacher feedback” variable is the amount of informal, not formal, feedback. Keep this in mind as we look at how this variable relates to measures of school effectiveness in math and reading.
Using the matched-pair estimate of effectiveness, teacher feedback was strongly associated with both math and reading results. This was the only variable, except the “bundle,” which demonstrated a strong statistically significant correlation in both content areas (p-value <.05 in math; <.01 in reading). One other variable, “instructional time,” was strongly significant in math but not in reading. Meanwhile, another variable was just weakly correlated with outcomes in both subjects (“high expectations,” p-value <.10).
Meantime, the lottery-based results also support the hypothesis that informal feedback matters. Overall, the link between school-factors and test results weakened under the lottery results due to the smaller sample size. However, the teacher-feedback variable remained somewhat correlated to both math and reading results (p-value <.10), while just one other variable—tutoring—was correlated in both subjects (also at a 10 percent level). Interestingly, tutoring was not correlated in math and only weakly in reading under the matched-pair approach. Neither instructional time nor high expectations were significantly correlated using lottery-based estimates.
Dobbie and Fryer’s study suggests—it is a relatively small-scale study—that the amount of informal teacher feedback is one crucial element of a highly effective school. It may be even more important than formal feedback, and it could also be more important that factors like “high expectations,” instructional time, and tutoring. And perhaps principals giving “informal” feedback—maybe an instructional tip at an opportune time—might be less costly to schools than implementing tutoring or adding instructional time.
It appears, then, that schools, on their own volition and outside of a legal evaluation framework, could increase performance by simply making informal teacher feedback a regular part of everyday life.
[1] The “bundle” is an index of 37 school-based variables that the researchers aggregated (not including “teacher feedback” or any of the other four variables that were examined separately).
[2] The amount of feedback, both informal and formal, per semester was gauged through principal surveys. “Informal” feedback referred to things like a five-minute “drop-in” by a principal. Formal feedback was defined as “written” feedback.
