An Odd Couple of Measuring 21st Century Skills

The morning session at the Technology Enhanced Assessments conference hosted by the K12 Center at ETS and CCSSO featured what Michael Russell, Measured Progress, called a formative assessment odd couple—a messy science virtual world and a math homework helper.
Ken Koedinger, CMU, discussed the question of whether teaching on the test hurts validity. He tested ASSISTments, an instructional assessment and homework platform. He determined that ASSISTments, a project led by Neil Heffernan at Worcester Polytechnic Institute, was both a valid assessment and produced significant learning value from using the platform in homework, spaced practice, and connecting with parents.  There’s lots of evidence on the benefits of assisted homework on the ASSISTments site.
ASSISTments (funded by Gates and Carnegie) is also ranking the quality of educational content. For example, a SchoolTube video on area of a perimeter had greater learning benefit than a static page.

Immersive Science

Chris Dede, Harvard, has spent the last two decades studying the intersection of instruction and assessment in immersive environments.
With a grant from IES, Chris’ team created EcoMUVE, an immersive one month middle school ecosystem science curriculum that encourages students to engage in collaborative inquiry and to explore (post-secondary level) causal complexity.
A multi-user virtual visit to a pond yields thick logfiles noting events, observations, and measurements. Midway through the month of virtual visits, the fish in the pond die. Students conduct a CSI-like analysis and plot time series data to attempt to identify environmental changes that may have contributed to the fish kill.
EcoMuve is going mobile ( with an augmented reality (AR) fieldtrip app for Boston area kids. I love the idea of AR science field trips with hotspots that augment the experience. One day a pond could be a bio-experience, the next a geo-experience, and the following day a history and social studies experience complete with three logbooks of data and observations.
The team is capturing information about the exploratory paths, the use of guidance systems, and the transferability of learning.
This summer EcoMUVE will be released as open content with supplemental teacher supports. They will also release an authoring shell for localized augmented reality fieldtrips.

State of Play

Gilbert Andrada, Connecticut State Department of Education, acknowledged the importance of state leadership to develop tech infrastructure for digital learning and assessment. 
He noted that both of these presentations suggested a “re-conceptualized relationship between students and teachers” and new strategies for combining instruction and assessment.
Remember this was a testing crowd—vendors and state assessment people—so these papers resulted in a bit of head scratching about “assessment of, for, or as learning.”
The last two days included a lot of discussion about testing hard to measure KSA (that’s knowledge, skills, and abilities). I suppose that does pose a challenge for summative assessments, these two projects suggest that if we focus R&D efforts on engaging Core-aligned learning experiences, we’ll advance assessment as well as achievement.

Tom Vander Ark

Tom Vander Ark is the CEO of Getting Smart. He has written or co-authored more than 50 books and papers including Getting Smart, Smart Cities, Smart Parents, Better Together, The Power of Place and Difference Making. He served as a public school superintendent and the first Executive Director of Education for the Bill & Melinda Gates Foundation.

Discover the latest in learning innovations

Sign up for our weekly newsletter.


Paul leather

The formative/summative divide in the digital world of assessment is another unmapped frontier. Perhaps a national challenge is needed to figure it out. This is
A major problem for those of us interested in scaling competency education in the public sector.

Matt Dunleavy

Great post, Tom. My colleagues and I have been working on an AR development platform called FreshAiR that enables anyone to create mobile and interactive AR learning experiences: Currently, FreshAiR is being used by educational technology research groups including the EcoMOBILE project at the Harvard Graduate School of Education referenced above. There is a lot to research in this emergent field and I am interested what other projects or teams (e.g., UW-M, MIT) are currently working in this area. Any ideas?


Tom Vander Ark

Hi Matt, we're excited about the potential of AR, great to know about folks in the field. Would like to know more and will connect you to other folks working in the space.

Leave a Comment

Your email address will not be published. All fields are required.