On Pilots & Proof Points

Pilots and proof points show what’s possible.  They are useful milestones as part of an investment strategy.  They are useful examples as part of an advocacy strategy.
Pilot projects demonstrate proof of concept—show that it works, post some early indicators.  Proof points are comprehensive interventions that move the needle on desired outcomes.
The challenge in education is quality at scale.  There are thousands of good schools, but few communities that put it all together an sustain it over time.  Quality at scale is expected in the private sector—businesses need to execute at consistently high levels in lots of places at the same time.  But U.S. education execution—specifically the quality of classroom instruction—varies wildly in the same school, across the same city, as well as nationwide.
In most cities and states, schools simply replicate social class.  What we need are schools that prepare the vast majority of students for the idea economy—at the minimum that means being able to pass  a community college entrance exam and earn college credit without remediation.
Problems associated with achieving quality at scale in America include 1) a history of local control with politically elected leadership and 2) a history of individual teacher practice conducted in private.
Local control is ineffective at best and, in urban America, is corrupt behind repair. Politically elected leadership results in a regression to a low mean. As argued in a recent blog, coherence can only be created by sustained effective and mission-focused leadership.
Challenges of governance and practice have been best dealt with in purpose built networks where schools and systems have been designed from scratch to be responsive and effective.  Two examples include charter management organizations and online learning providers.  Organizations like Aspire Public Schools and Florida Virtual School have achieved impressive results but have not taken responsibility
Producing quality at scale across geography, particularly one with any poverty, as been more elusive.  A portfolio approach (as described in this 2004 paper) with sustained leadership (most often exhibited under mayoral control) produces the best results.  But even promising decade long efforts in New York, Boston, and Chicago fell well short of the goal of quality at scale.  New Orleans, with the horrible opportunity to start over, will be the best portfolio proof point.
One challenge with efforts to create proof points is confounding variables.  Cities and education are complex and efforts to influence them are hard to measure.  It’s tough to know exactly what worked and what didn’t—causality is tough to pinpoint. This is particularly true if the investment strategy is iterative and adjusted based on leading indicators and early lessons learned.
Time is another challenge.  The initiative to open 400 new schools in New York took a decade as did the evaluation of the first 200.  New schools take a year to plan and are often opened one grade level per year, so it can be a half a dozen years before you can measure graduation and college attendance rates for a new high school.
Proof point design. Important proof points would include all students in a diverse geography with high level of challenge (>50% students in/near poverty) with some level of scale (>20,000 students), and would be sustainable after the intervention and replicable in other locations. 
Important proof point geographies would show significant improvement in achievement for nearly all students sustained over time.  In the first two years there would be significant improvement in leading indicators.  After five years, terminal outcomes would begin to emerge.
Pilot projects. Short of full proof points, demonstration projects can be illustrate the possible benefit of new strategies or components.  The summer school debut of School of One is a great example of a low stakes demonstration that illustrated customized learning and introduced the concept of a personal learning ‘playlist.’   Components worth demonstrating include:
1) blended school models including:

  • · customized learning with multiple modes of instruction driven by a smart recommendation engine and comprehensive learner profiles
  • · school designs that double productive learning time
  • · talented and well trained teams in a differentiated (different levels) and distributed (different locations) staffing models with supporting productivity tools

2) student supports including:

  • · demand development activities that increase student and family engagement and academic press
  • · self management, guidance, career awareness, and decision support systems

3) policy innovations including:

  • · learning certification systems that allow students to show what they know and represent a breakthrough competency-based learning
  • · weighted student funding with choice to the course (or unit)

When you put all of these together, efforts to create community proof points that significantly shift the achievement distribution should include school improvement, new school development, community engagement, talent development, systems development and an innovation strand.
Demonstration projects take a lot of work and some smart risk capital.  Attacking the big challenge of quality at scale and creating real proof points is an order of magnitude harder and riskier.  Charter networks and online learning providers have achieved quality at scale where choice exists, but it’s a tougher assignment to lift a whole community. That will require new tools, new schools, and a new governance strategies.

Tom - Speaking Engagements

Tom Vander Ark

Tom Vander Ark is the CEO of Getting Smart. He has written or co-authored more than 50 books and papers including Getting Smart, Smart Cities, Smart Parents, Better Together, The Power of Place and Difference Making. He served as a public school superintendent and the first Executive Director of Education for the Bill & Melinda Gates Foundation.

Discover the latest in learning innovations

Sign up for our weekly newsletter.


Leave a Comment

Your email address will not be published. All fields are required.