On Promoting Efficacy: A Conversation with John Tweeddale

John Tweeddale (@JTweeddale) is Senior Vice President of Efficacy and Quality at Pearson. He works in Boston, MA and his team supports customers in North America.
Tom Vander Ark (TVA): What does your team do?
John Tweeddale (JT): Our team is customer-facing, both K-12 and higher education, and focused on efficacy and quality across North America.  We support Pearson’s efficacy strategy and study how it is applied. We help customers define measurable outcomes and observe how Pearson products and services are meeting those outcomes and how we can improve learner outcomes.
TVA: Tell us about the higher ed team.
JT: Our higher education team is ten years old and immersed in what needs to happen to improve outcomes for the end user. We gather evidence on how products are used with particular attention to implementation models. This year we will gather data for about 180 case studies on use, implementation and outcomes and will publish 10 to 13 papers. Our recent papers include:

• MyLab & Mastering 10 Best Practices
• 2014 Science and Engineering Efficacy Report
• 16 Proven Ways to Help Your Course Redesign Succeed

Our customers find it helpful to see the challenges of similar institutions, and the models that are being used to address them. We want to build product and service capabilities that help our educational partners improve their outcomes based on specific learner needs.
TVA: What about K-12?
JT: In January, we launched an initiative to conduct 20 school and district case studies this year.  We have a team studying the impact that implementation models have on outcomes including variability between the classrooms, across a district and statewide.
Our Quality team tries to gauge the quality of the experience, in order to understand the learner journey while using our products and services particularly if they are using multiple products and services.  We want to identify the moments of truth and moments of difficulty — we want to know at what point in the journey can customers expect to achieve the desired outcome.
Similar to the higher education space, we want to understand more about areas where our products and services are making an impact on the path to helping customers achieving the desired outcomes, in order to inform the development of new products and services.
TVA: Sometimes you just provide a textbook and in other cases you run an online school for a customer. Does your approach vary based on the extent to which Pearson is involved in implementation?  
JT: There is wide variability of involvement in implementation. We differentiate our approach based on how fundamental the products are to the course and how far along the digital continuum a professor, teacher or school is. It could be as simple as providing a great digital tool where it’s challenging to gather evidence of outcomes.
We generally don’t conduct full randomized control trials on the products adopted. We are cautious of making causal rather than correlational claims.
TVA: What about implementation support?
JT: In higher education, we’re putting a higher degree of focus on implementation support. We are providing more “how do I?” support with each product. We know strategic implementation is key to delivering better outcomes. We are building best practice models based on our extensive evidence to help instructors identify the implementation model that is more likely to help achieve intended learner outcomes. For K-12, we are identifying the opportunities to build and scale these capabilities.
TVA: How is this line of inquiry into customer experience leading to changes in the company?
JT: We’ve made a commitment to outcomes and reporting against our efficacy framework by 2018 and that is fundamentally changing product and service life cycle development. This commitment also positively changes the culture of the organization.
The Quality teams developed around the Efficacy work have really improved understanding the customer’s journey and viewpoint. That includes better appreciation of the wide range around skills that students need/want for college readiness. (For more, see The Incomplete Guide to Delivering Learning Outcomes)
TVA: Should the efficacy framework be external?
JT: The framework is not unique to Pearson. We encourage widespread use in the educator sector, which would provide consistency across all institutions.
Some of our customers have used the framework independently. We believe it could lead to the development of a strategic plan, and a formal process for implementation and improved learner outcomes.
TVA: Would some kind of third party validation, like GAAP and audited financials in accounting, be useful when it comes to efficacy?
JT: We could see that if a framework becomes widely adopted. We’d need to think about standards of reporting for the external community to understand those standards.
TVA. Will this efficacy work drive the company toward more implementation support?
JT: It could. It’s clear that quality implementation support and professional development are important.
TVA: What value is your group adding to the company?
JT: First, a robust understanding of customer data and how it ties back to product development. Second, we add value to customers by helping them use the efficacy framework and sharing implementation best practices.
When you’re in a classroom and looking into the eyes of a student, you see evidence of all the work you are doing. It’s why I have been in the business for as long as I have.
For more on Pearson, check out:

• Pearson Creates Leading Curriculum, Apps for Digital Learning Environments
• Pearson Partnerships for Principal Development
• Infographic: Pearson on College Readiness

 
Pearson is a Getting Smart Advocacy Partner