Are you ready for the era of ‘big data’?

“Radical customization, constant experimentation, and novel business models will be new hallmarks of competition as companies capture and analyze huge volumes of data. “  The McKinsey Quarterly ran a great article last month calling the question, Are you ready for the era of ‘big data’?
The article opened with a comparison of two retailers, one that had invested in data:

The competitor had made massive investments in its ability to collect, integrate, and analyze data from each store and every sales unit and had used this ability to run myriad real-world experiments. At the same time, it had linked this information to suppliers’ databases, making it possible to adjust prices in real time, to reorder hot-selling items automatically, and to shift items from store to store easily. By constantly testing, bundling, synthesizing, and making information instantly available across the organization—from the store floor to the CFO’s office—the rival company had become a different, far nimbler type of business.

The shift to personal digital learning will allow schools to become knowledge creating institutions that are constantly testing, bundling, synthesizing, and making information instantly available across the organization. 
McKinsey asks five big questions about big data.  They are adapted here for K-12:
1. What happens in a world of radical transparency, with data widely available?
The ongoing challenge will be to figure out what to do with all of the data—how much to collect? how much to share?  what data to use for what decisions?
It’s likely that data pyramids will emerge with lots of data shared between teachers and students, a big extract shared with an instructional team, and key performance indicators with the district/network and state.
We’ll need to invent strategies and tools to compare big data sets (i.e., how does my Khan score compare to your enVision, Power of U, and Mangahigh score?).
Standards-based gradebooks will need ability to extract data from multiple apps (all producing feedback on a common data standard).  They will also need to house (or link to) portfolios of evidence.
Student records (probably in SIS) will be expanded to capture not only performance feedback but keystroke data that will be key to understanding persistence.
2. If you could test all of your decisions, how would that change the way you educate?
Like Rocketship Education does with their learning lab, big data will allow a continual bake-off of learning products and services to make sure we’re giving students the best possible sequence of learning experiences.
3. How would your school change if you used big data for widespread, real-time customization?
Individual pathways for students and competency-based progress have huge implications for how schools are organized and operated.  For hundreds of years schools have been organized by age cohorts in classrooms.  Customized competency-based learning will require a whole new operating system for schools (i.e., policies, organization, staffing patterns, and tools).
4. How can big data leverage great teachers?
We’ve been telling teachers to differentiate but haven’t given very good tools. Big data will give them better information, help customize learning for each student, and enable differentiated staffing models where great teachers can impact more students.
5. Are there new ways to organize schools around big data?
Big data will power new achievement recognition (badging) systems that will use multiple forms of assessment to certify mastery.  When kids can move at their own pace, we can rethink how and when and where they work in groups.
I think we’ll see more flex schools, as described in the Rise of Blended Learning, with an online core and lots of flexibility to add projects, field trips, and application opportunities.
I’m glad that Aimee Guidera and the Data Quality Campaign has been encouraging states to use what we’ve got and get ready for more data.