Calibration Scoring to Foster Inter-rater Reliability

Valid, Defensible Observations

Fairness in the evaluation process occurs when observers follow similar processes, accurately interpret district-defined rubrics, and have the necessary skills to gather objective and representative evidence of teaching practice.

Administrators instill confidence in the observation and evaluation process by ensuring district-wide inter-rater reliability.  Educators trust the observation process and know they are being evaluated fairly and consistently.

Truenorthlogic Calibration is an easy-to-use, reliable tool for scoring and certifying those who conduct teacher, principal and staff observations. With Truenorthlogic Calibration, districts create consistency and fairness in the evaluation process and foster inter-rater reliability.

Used to inform and drive professional development, Truenorthlogic Calibration provides a rich source of information on how to prioritize limited budgets for leadership training. Truenorthlogic Calibration manages calibration events with strong workflow and access control capabilities, precisely measuring an individual observer’s skills and accuracies.

Truenorthlogic Calibration was built in collaboration with Teaching Learning Solutions, an esteemed organization that conducts observer certification trainings and analyzes data to determine degrees of inter-rater reliability based on research, best practices and statistical significance.

Truenorthlogic Calibration is a key component of the unified Performance Matters Educator Professional Growth Platform, offering online solutions for professional development management, performance evaluations, and more, fully supporting continuous educator improvement.

  • Relevant, high-quality, master-scored videos aligned with district rubrics, which provide observers with realistic teaching scenarios. Districts may also add their own scored videos.
  • An easy-to-use scripting tool to help observers demonstrate their ability to collect defensible evidence and select valid ratings.
  • Three distinct accuracy measures which explain the observer’s accuracy, how the observer compares to his or her cohort, and how closely aligned the cohort is to the accuracy measure.
  • Automated scoring provides instant access to individual results.
  • Individual proficiency-level scores identifying observer strengths and weaknesses, driving professional growth decisions for observers.

Georgia’s Leader Effectiveness Story

How the Georgia DOE offers all districts in the state calibration training in support of inter-rater reliability.

Read more

“I found Truenorthlogic Calibration to be a great tool for our state office. We used this tool to facilitate statewide principal reliability practice and the final certification assessment. It allowed the participants a platform to uniformly enter data. I was then able to review individual results to analyze data trends and easily track completion rates and areas needing support.”

—Jeannie Rowland, Observation Calibration Specialist, Utah State Office of Education