Education Insights

The Role of Calibration in Determining Educator Effectiveness

Albert “Duffy” Miller, Ed.D., President of Teaching Learning Solutions, has gathered recent research and explored current trends in observer calibration, sharing his findings in a comprehensive new white paper, “The Role of Calibration in Determining Educator Effectiveness.” We’ve shared the first section here, and you can access the full paper below.

Executive Summary

Section 1 – Effective Teachers Result in Higher Student Achievement 

Education leaders and policymakers have grappled with the difficult issue of education reform, a topic that has received only lip service in the past. Because there is now a body of recent research that links student achievement with teaching behavior, the improvement of teaching practice becomes crucial. How do we help all teachers reach their full potential in the classroom, and how do we respond to ineffective teaching?

None of these issues can be addressed without better teacher observation and evaluation systems. Evaluation/observation models need to incorporate elements of effective coaching practices that use evidence from the observations to provide teachers with evidence-based feedback that helps them grow as professionals, whether they are developing or highly effective in practice. The result of the observation processes need to provide school leaders with a consistent, appropriate and authentic way to measure teachers’ practice in order to support each teacher’s development, and to give the schools the data they need to build the strongest possible support and instructional teams.

While we expect that evaluation models provide a vehicle to do all of these things, in most cases, they haven’t even come close. Instead, they were typically perfunctory compliance exercises that rated all teachers “good” or “great” and yielded little useful information. As Secretary of Education Arne Duncan noted in a summer 2010 speech, “our system of teacher evaluation… frustrates teachers who feel that their good work goes unrecognized and ignores other teachers who would benefit from additional support.”

The research that shows the correlation between teacher behavior and student success covers a broad area of the country and is documented throughout many discipline areas. For instance, in the late 1990s, all 184 schools in Milwaukee ‒ private, public, and charter schools ‒ focused on improving quality of instruction and student outcomes across the system. They agreed that one way to improve was through direct observation of instruction and by providing tools for, and support to, principals’ classroom observation.

Their conclusion, as reported by the Brookings Institute, was that “Having a top-quartile teacher rather than a bottom-quartile teacher four years in a row could be enough to close the black-white test score performance.” 1 Researchers looked at a large data set on individuals in grades 3 through 7 in the State of Texas. This data set permitted analysis of the question of differences in teacher quality in the determination of student outcomes in 2005. Their conclusion, as reported in the journal Econometrica, was that “Having a high-quality teacher throughout elementary school can substantially offset or even eliminate the disadvantage of low socio-economic background. Teachers and, therefore, schools matter importantly for student achievement.”2

Section 2 – Value of Evaluations and the Impact of Observations

Section 3 – Fairness, Reliability and Validity = Defensible Evaluations

Section 4 – Research-based Education Methodologies Support Evidence

Section 5 – Aligning to District-defined Framework and Expectations

Section 6 – Improving Inter-Rater Reliability Among Observers Using a Calibration Platform

Download to Continue Reading

1 Gordon, R.., Kane, T. J., & Staiger, D. O. (April, 2006). Identifying Effective Teachers Using Performance on the Job . Hamilton Project Discussion Paper, Brookings Institute

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. Econometrica, (March, 2005), Teachers, Schools, and Academic Achievement. Vol. 73, No. 2 417– 458