This is Part 1 of a two-part series, sharing Los Angeles Unified School District’s experience with PM Calibration.
What role do school site administrators play in improving teaching practice? How can a district’s central office build the capacity of administrators to provide quality feedback?
Improving teaching practice within the Los Angeles Unified School District (LAUSD) is contingent on the ability of school site administrators to provide quality and timely feedback to the 26,000+ teachers we employ. Providing this type of feedback to teachers requires a commitment from administrators to frequently visit classrooms and collect accurate and reliable evidence of teaching practice. To support these efforts, since 2011, LAUSD has invested time, funding, and staffing to provide administrators with a multi-day observer training and certification assessment on the use of the district’s instructional framework, the Teaching and Learning Framework (TLF). Currently, 2,059 LAUSD administrators are certified to observe teaching practice using the TLF.
While rigorous, the initial observer certification training is a stand-alone professional development opportunity designed to introduce participants to the skills and knowledge needed to collect, align, and accurately rate teaching practice. Once in the field, administrators, without ongoing support, may begin to exhibit certain behaviors that inhibit their ability to accurately differentiate between levels of practice. These shifts in accuracy can ultimately impact the quality of feedback administrators provide to their teachers.
Frequently measuring our administrators’ abilities to accurately observe and rate teaching practice can inform the development of our district-wide supports for improving the quality of feedback. Beginning summer 2015, central office staff from the Professional Learning and Leadership Development (PLLD) branch have annually invited its administrators to participate in a calibration event. Calibration serves three main purposes:
- Useful for detecting rating errors that occur over time
- A means for providing observers feedback about their accuracy performance
- Inspires confidence in the observation process for teachers and administrators
In previous years, prior to the beginning of the school year, over a thousand administrators attended an in-person calibration event. During the calibration event, administrators deepened their understanding of the TLF, discussed with colleagues their rating practices, and completed an accuracy assessment. Given the size of LAUSD (geographical and population density), preparing for this in-person event required months of immense logistical coordination. Anticipated reduction in PLLD staff as well as feedback from the field informed the shift to providing the calibration event online using the Performance Matters calibration solution for the first time this summer. From the comfort of their own home (or office!), administrators had one week to log into LAUSD’s online evaluation platform, the My Professional Growth System (MyPGS), to view a prerecorded classroom lesson, review scripted evidence, align evidence to selected elements of the TLF, and rate the evidence using the TLF rubrics. Once completed, administrators electronically submitted their results via MyPGS. As submissions rolled in, PLLD staff monitored completion data to inform real-time decision-making for improving this year’s online participation rates.
Would fewer administrators complete the calibration event now that there was no in-person monitoring or would there be an increase in participation due to improved flexibilities?
Stay tuned for the 2017 LAUSD calibration participation results and lessons learned!
 Joe, J., Kosa, J., Tierney, J., Tocci, C. Educational Testing Service. Observer Calibration: A Tool for Maintaining Accurate and Reliable Classroom Observations. San Francisco, CA: Teachscape. Online.
Authored by Martha Cortes and the members of the Professional Learning and Leadership Development (PLLD) team – Los Angeles, CA
A focus on quality teaching and learning practices drives the short- and long-term goals of the PLLD team. For over five years, this team has worked together on the development, implementation, and refinement of the Los Angeles Unified School District’s educator growth and development policies and programs.