Calibrating District-Level Innovation

Collaboration is an invaluable tool for building and maintaining great teaching practices. That’s why I’m always surprised when I talk to groups of teachers about their collective practices, beliefs, and attitudes about pedagogy, curriculum and assessment. It turns out, they often have no idea that they’re on totally different pages around these fundamental building blocks of education. For anyone working to transform schools, this is where the work begins.

Research shows that professional development that builds cultures of practice must place a heavy emphasis on teachers collaborating and talking about their practices, beliefs, and attitudes. One method that’s been gaining traction in the US is authentic lesson study. This is an excellent way to challenge assumptions and get teachers collaborating. Emily Hanford’s 2015 article in American RadioWorks points out that many innovative practices that have been developed in the US are being studied by teams of Japanese teachers, using lesson study. Jo Earp’s 2016 article in Teacher Magazine situates lesson study at the intersection of model lessons and critical feedback, providing teachers with authentic learning experiences and hands-on opportunities to learn. While there are certainly cultural barriers to bringing lesson study to the US, its increased popularity and success in helping to change practice demonstrates the strength in approaches that meet teachers where they are, provide opportunities for authentic collaboration with colleagues and are situated in the work teachers are doing.

Another such approach is calibration training, with a twist. Many schools and districts offer calibration training when undertaking new performance assessments or common rubrics. These trainings typically focus on teachers grading papers together or comparing grades on commonly-assessed assignments. While the focus is often on training teachers to give aligned ratings, this type of training represents another opportunity for teachers to engage with colleagues, work together on authentic practice and evolve their thinking around assessment, instruction and curriculum. I (relatively) recently had the chance to facilitate such a training, and I was pleasantly surprised by the results. Here’s how it worked.

The district was small and rural. They had a long history of school autonomy, with little decision-making coming from the Central Office. They had a written curriculum and district assessments, but both of these were viewed by principals and teachers as “suggestions” from district personnel. Into this dynamic, the district moved to establish a proficiency-based graduation system, and the schools decided to band together to develop common materials and assessments. As they had little history in proficiency-based grading, we needed to develop in-depth training while simultaneously developing well-articulated curricular expectations that were vertically aligned. As we always stated that our values were to be a “true learning community” that values the inputs and diversity of all learners (including staff), we wanted to make the necessary shifts in a collaborative, egalitarian way, rather than by employing expert presenters to teach our staff what to do.

To accomplish this, we began a series of graded performance tasks to get students and teachers used to both new expectations and new procedures. These assessments were created by district staff, in collaboration with teachers from multiple schools. We also adapted existing rubrics, but made sure they were vertically aligned and in student-friendly language. Then we built and executed our new writing assessment protocol:

1. Students completed the performance task in-class and responses were collected at Central Office
2. Each teacher (regardless of specialty area) was assigned to a grade level
3. Papers were randomly assigned to teachers across the grade level, with each student’s paper being assessed by three different teachers.
4. Teacher ratings were collected (note, there was no “overall” rating, but each teacher assigned ratings in several areas from the rubric)
5. Each paper was analyzed for ratings disagreement.
6. Central Office staff generated a list of the 10-15 most disagreed-upon papers for each grade level.
7. Each grade level team would then convene for a 3-4 hour facilitated meeting. At this meeting, papers from the list would be discussed, with discussion topics focusing on:

• Why was there disagreement?
• What are appropriate grade-level expectations for this work?
• How do we support student growth?
• Do we need to change our rubrics to reflect our expectations?
• How can we support each other to make the needed changes in curriculum, instruction and assessment?

Fortunately, we were able to automate much of the assignment, completion, collection, distribution, grading and rating collection through Google Classroom, Google Spreadsheets, and Google Forms. These tools streamlined the process and provided a no-new-cost solution, given that we were already using Google for our district and were able to redirect Central Office staff time to do the programming and set up work.

The results were outstanding. While our intent in the collaborative meetings was to have rich dialogue about practices and to establish common expectations and grading standards, it ended up going so much further than that. Over the course of 6 semi-annual meetings in three years, teachers started using the district curriculum documents and editing them together. We developed the rubrics and adapted them into student-friendly language, so teachers started using them in classrooms. In addition, we noticed two unexpected benefits:

• Over the course of the 6 meetings, every grade-level significantly reduced the amount of disagreement between raters (by an average of 92%). The program was so successful that by the end of the third year, we had a hard time finding papers to debate!
• Teams became so good at collaboration and discussions of student work that we no longer needed facilitators, and groups could get through papers very quickly. During the first year, we would often only get through 3-5 papers during a 3-4 hour session. In the last year, most groups could get through their entire packet in under 2 hours, and move on to more pointed discussions of curriculum reform, which they did with great enthusiasm.

This experience was one of the most transformative PD investments I’ve ever been a part of. As we moved into project-based learning, this, combined with our PD Academy, which provided teachers with personalized PD “courses” with cohorts of colleagues, helped our teachers become more comfortable with new ways of grading, developing shared curriculum and expectations, and utilizing district staff and colleagues to develop standards of practice. Teachers began using authentic, embedded assessments and collaborating with district colleagues in new and exciting ways.

In addition, this structure became the basis for discussions of student achievement data. Teachers became more comfortable with district-based assessments and our way of talking about them. They began aligning their grading practices to our initiative. As a result, they began to see this data as supportive and helpful, rather than a source of conflict with classroom assessment data. Teachers also found the district assessment useful as a “gut check” against their classroom assessments, as they could see how their students were progressing compared to those in the other schools across the district. This work also smoothed the introduction of additional benchmark tests, because we had established a culture that knew how district and classroom data could support each other.

Classroom teachers know that  “those who do the work do the learning.” Our program showed how well this model can work for teachers and building leaders, as well. We provided our staff with authentic ways to engage with curriculum, assessments and data in a manner that was non-evaluative and gave them the opportunity to work with colleagues. They responded by doing the difficult, but important, work of authentic collaboration. It truly transformed practice AND culture in our district.

Finally, it should be noted that this entire initiative cost less than $5000 per year. This, as part of an overall new professional development strategy based on a peer learning model, enabled us to lower our district PD costs by 60% while raising both effectiveness and teacher satisfaction ratings by over 80%.

Schools across the country are learning how important it is to teachers that we take their professional skills and insights seriously. Programs like lesson study and Teacher’s College are excellent examples of how we can leverage internal expertise to learn together and amplify knowledge.

As administrators, we want our teachers to let the students take active roles in their own learning. We want teachers to cede control – to become “guides on the side” and have faith in their students. A crucial step in making this happen is to model this attitude with our staff. We must allow our teachers to take the reins in their own development. We must prepare PD that is active and lets the teachers engage where they are. Teachers are dedicated professionals who want to improve their practice. They are unafraid to roll up their sleeves and get to work. Providing them with meaningful opportunities to do so is crucial to staff development. These opportunities exist, they really work, and they are not very resource-intensive. Together, let’s get to work!

For more, see:

• Find a Way to Yes: 9 Leadership Lessons from Pam Moran
• Because Being a “Good School” Isn’t Good Enough
• School Models: Pick Your Favorite Flavor

David Adler is co-founder of ReContext Data Solutions, where he develops data-informed leadership practices and strategies. He spent 20 years as a public school teacher and administrator and his accomplishments include the largest 2-year MCAS score increase in Massachusetts history. Connect with him on Twitter: @DataAdler 

Stay in-the-know with all things EdTech and innovations in learning by signing up to receive our weekly newsletter, Smart Update.