What is Return on Learning?
In the business world, Return on Investment (ROI) is a key metric in that determines whether the programs you’re investing in are working. It tells you what programs are worthwhile and which should be reconsidered and reevaluated.
In K12, we invest in learning outcomes for all students, which is a result of both teacher effectiveness and student growth. Doesn’t it make sense to be able to measure which programs, initiatives, or actions are having the greatest impact on learning?
At Performance Matters, we call this Return on Learning, or ROL. Many districts have taken bold steps to begin to analyze the impacts of specific purchases or implementations on a defined metric. For example, attempting to calculate the learning benefit for students when a school purchased and began using iPads or attempting to determine if a specific professional development offering yielded larger learning gains. Answers to these questions are certainly important in understanding effective practice in districts where time, talent, and treasure are limiting factors. Frank and Hovey (2014) state, “But this process misses some of the biggest cost drivers in the district’s budget and never addresses some of the fundamental resource issues that may actually drive student success.” We tend to agree.
What are we measuring?
This equation forces you to really consider your biggest cost drivers and funding sources in a district. The biggest cost by far is personnel expense. Ever since the public report “A Nation At Risk” was published, the United States has discussed, mandated, and reported on the progress of student achievement. From AYP, NCLB, and, most recently, RTTT, federal spending and legislation attempted to bring resolution to the challenges American students face competing in a global economy. The efforts did not stop there. The Feds also recognized that student achievement wasn’t the only thing we should be looking at.
“With the recent advent of teaching effectiveness data and improved data systems, our ability to estimate impact and cost objectively and accurately has grown tremendously.” Frank and Hovey
Through Race to the Top funding, states attempted to highlight and bring resolution to the challenges around comprehensive and research based evaluation for teachers and administrators. This call to action, motivated by the receipt of federal dollars, certainly forced many school systems to find ways better ways to evaluate classroom and leadership practice. The work at national and state levels has provided a perfect platform to now look at multiple measures to initiate that vital deep dive into system reforms for public education entities.
How do we use student and educator data together to measure ROL?
Students are required to show mastery on standards through assessment. Do your assessments measure the standards?
Can your teachers quickly identify students who need assistance on specific standards to target instruction to weak standards?
In an environment where less testing is now the call to action, how can teachers get the biggest meaning from assessment practice while holding students accountable for learning essential content?
Changes in climate for better measuring teacher effectiveness and student growth have forced terms like inter-rater reliability, calibration, student learning outcomes, student growth measures, response to intervention to the forefront of educational conversation. Where many school systems struggle is in determining what to focus on first and how to measure efforts and outcomes in strategic planning.
Disparate data is one of the many roadblocks many school districts must overcome to clearly see outcomes and value resources required to support. At some point a school district must lay student growth measures next to teacher effectiveness ratings and evaluate outcomes. Schools must be able to answer questions like:
Is there a positive correlation between investment in PD and student growth?
Are we improving the quality and effectiveness of teachers, principals, and other school leaders?
Is scheduling equitable for struggling students among our experienced teachers?
Finding answers to these questions is where the largest expenditure a district makes and the metric through which they are held accountable meet. Using an integrated and reliable statistical analytics tool such as Performance Matters Analytics ensures the validity of your measures. Because the most important question is, “Are you really measuring what you think you are?”