Town Hall: Experience Matters

In this town hall, Getting Smart’s Mason Pashia and Education Design Lab’s Meghan Raftery discuss the findings of their collaborative report, Experience Matters, which explores how work-based learning experiences can be better designed, documented, and communicated across education and employment. The conversation centers on a shared framework built around three dimensions of experience quality — autonomy, complexity, and contribution — and how this common language can help educators, employers, and learners tell richer, more credible stories about skill development. Joined by researchers Michael Crawford and Chris Unger of America Succeeds, as well as practitioners Kimberly Hauser and Andy Hepburn, the group examines what it takes to make experiences legible: from intentional experience design and structured reflection to the role of digital credentials in capturing context that traditional resumes and degrees often miss.

Introduction and Overview

Mason Pashia: Hello everyone.

I’m Mason with Getting Smart. We’ve done a town hall on this specific wheel before, which came out of a publication we did about two years ago. What we were interrogating with that publication was the different types of experiences that would be called valuable, or that we would consider something to document in a person’s life journey to demonstrate what they know, what they’re able to do, what they’ve done before.

So we put together this little wheel — it’s going to come back again a bit later. This whole conversation started because we were thinking about the landscape of skills validation and skills credentialing, which Meghan’s going to talk a lot about here in a second.

We were trying to figure out how you would actually show that someone had these skills, whether they be technical skills or the more durable and transferable skills, which I think famously are a little bit hard to prove, but also famously some of the most important skills that a person can have.

So we’re going to come back to this to talk more about experience types. For the purposes of today’s conversation and the report, we really zeroed in on internships and client-connected projects — something that’s a little bit more work-based, a little bit more directly engaged with a mentor. But there’s a ton of other places where you can extrapolate this out. I just wanted to prime the conversation today: a lot of experiences can follow what Meghan and I are talking about today, even though our conversation, for the sake of understanding, may be a little bit more narrow.

So, Meghan, hello.

Meghan Raftery: Hello.

Mason Pashia: It’s great to see you.

Meghan Raftery: Yeah, thank you.

Mason Pashia: You’ve been doing this work for a really long time, so just walk us through a little bit of the work of Education Design Lab at large and then specifically on this subject.

Education Design Lab’s Work on Durable Skills

Meghan Raftery: Yeah, so Education Design Lab as an organization is focused on the future of education for the future of work. We’re an ecosystem convener, kind of in the spaces in between work and learning. A lot of our work tends to be with what we call new majority learners — the folks that are in our institutions now that look different than what people would assume. An 18-year-old who goes to college and graduates in four years is rarely the story of who our learners are. We have similar stories about workers.

Education Design Lab uses a human-centered design process to really get to know who is actually interacting in these spaces, what they’re doing, and what they need. The work that I do at the lab is specifically on skills validation. We’re interested in looking at how somebody has the skills that they have, and it tends to be more on the workforce side than the student side. We are looking at those skills and how they play out in the employment context, especially for people who don’t have formal degrees that signal their work in the ways that we’re used to seeing it.

Our quest in this particular project was to take the lab’s — now I would think over a decade old — durable skills framework, which, like many frameworks, contains the skills you would expect: communication, collaboration, intercultural fluency, self-directed learning, and some of those soft skills.

We wanted to know: could you take a durable skill and level it? That was the initial question. What we really wanted to know was, if you have critical thinking, do you just have critical thinking, or is there more critical thinking or better critical thinking? So we took our educator brains and tried to build a framework around our framework to ask: is it possible to level durable skills? Do we need to? Is it something that we want to do?

We started a bunch of different experiments in trying to level the skills — a traditional four-by-four grid rubric of what it looks like to demonstrate a skill at the novice, intermediate, and expert level. We tried to do more of a holistic version. Ultimately, the traditional methods of leveling presented some real challenges for durable skills, because durable skills exist across context but also manifest themselves in contextual ways.

If you want to talk about a 10th grader’s critical thinking skills and a CEO of a small business’s critical thinking skills, they don’t feel even. But we also weren’t quite sure how to say they were uneven, and that’s really what we wanted to try to figure out.

We went through multiple iterations of different leveling frameworks. One of the things we thought was really important is that we wanted to emphasize that all the levels actually matter. When you see something like novice, intermediate, expert — especially in a K–12 setting — it’s tempting to think expert is the best and that’s where I’m striving for, without honoring the work that it takes to be the novice. There’s actually skill demonstration there, and to transition to intermediate requires a lot of skill demonstration too.

To value each of those levels is difficult. If you put bronze, silver, and gold on it and someone shows you their bronze, your first question is, why aren’t you a silver? So we wanted to be really clear that those experiences mattered at all the levels.

The other thing we started to discover is that a linear rubric that just says “this is what it looks like to be demonstrating this skill” was not enough. It was an evaluation layer that we could look at and say, but the evidence used to generate that evaluation really matters. The more evidence you have creates more trust. Being able to have multiple ways to demonstrate the skill — with different levels of maybe self-assertion and maybe performance review and maybe an artifact you’ve analyzed — the more you have, the more those signals are trusted.

We’ve also talked a lot about how different methods of validation matter. A lot of times people think of assessment as the validation method, but assessment is only one way. We like to think more of assessment-plus and get really creative about how you can assess somebody’s skills — different things you can do to source evidence, from a skills demonstration to maybe an experience translation where it’s already happened and we’re going to put this layer on top of it, to different sources of information that help us get that evidence.

And then ultimately the biggest learning of all was that context really matters. How and where the person demonstrated the skill is a huge key to being able to not only level, but just evaluate a durable skill at all. It’s something that matters to employers. It’s something that matters to educators. When you demonstrate something in a classroom, is that enough? If you say you demonstrated something in a workplace environment, how close is that workplace environment to the one in which you want them to demonstrate it? People really care about that context, which is very difficult to summarize in a universal way in a traditional rubric. By making it generalized, you take away the context, and then that makes the rubric feel less usable.

So we really grappled with this, and that’s really the publication we put out that initially connected us to Getting Smart. This project was a publication that said we’re still figuring this out — it wasn’t one that said, here’s our leveling concept, please go use it. That’s where we entered into this work and this collaboration. Mason and I started to untangle some of the things that yes, we learned, and also what we learned that we didn’t know yet.

Introducing the Experience Matters Report

Mason Pashia: I think that’s a great summary. I’ll also say that Education Design Lab has been doing a lot of really great work in the larger LER credentialing space. One way to think of today’s conversation is through the actual lens of the design of experiences, and we’ll talk more about that later. But another part of it is really this capture component — what is the metadata, how do you actually capture this context going forward such that it can be scaled, communicated, or trusted?

One key takeaway from our multiple years of research on this is that the technology is pretty much here for doing that. We have the ability to create a digital record that has a ton of data attached to it — contextual variables, artifacts, portfolios — that looks very different depending on the experience type. But the technology and infrastructure has actually been being developed in the background this whole time, which is wonderful. Now, as education designers and intermediaries and employers, we really have to take the next step to leverage it well.

So, want to go ahead and get into the report a little bit? This report is called Experience Matters.

A lot of people who have been following Getting Smart, following Education Design Lab, following Ryan Craig — who I think coined this phrase — think a lot about this idea of the experience gap: employers are demanding validated work experience for entry-level jobs, yet both AI and other emergent trends are making this really hard for learners to actually get that experience. So they’re saying, you need to have experience to do this job, and at the same time, the learners are saying, well, where would I get that experience? That’s so much of the work that you all are doing in this work-based learning intermediary space. It’s a really tricky issue.

At the same time, what it kind of fails to recognize is that people are learning all the time, as you all just shared in the chat. A huge part of this is really asking: how can we start to bring those types of experiences into a record that matters and communicates something of depth to employers?

The Framework: Autonomy, Complexity, and Contribution

Mason Pashia: So we basically created this framework together. The funny thing about this is we both were working on this problem completely separately and essentially identified the same three buckets — the qualifying markers of a quality experience. Those would be autonomy, complexity, and contribution. What we’re saying here is that if an experience increases in these domains, it is likely a more rigorous or more expressive-of-skill, confidence-type experience. As Meghan said earlier, it doesn’t necessarily mean that autonomy one is any worse than an autonomy two. It just means it allows you to right-size it for the right time for the right person.

Meghan, do you want to talk a little bit about where this came from for you all and specifically the progression of the one through three and onward?

Meghan Raftery: Yeah. It’s a little bit complex to explain the origins because, like I said, we swirled, and even being able to explain it to Mason was a challenge for us. But ultimately, this feels like one of those ideas that exists in the universe that people are grabbing for because it just feels right.

There are some origins for us in how we encountered it. My colleague Tara Laughlin is here with us today too — she was part of this swirl. I think originally she saw a presentation at a conference by a gentleman named Marcus Bowles, who wrote the Human Capability Index, which is used mostly in Australia but has international implications. It was developed simultaneously to some others, like the Sophia Framework, which is used to evaluate technology professions. It has this same kind of thing — the words might change, but the notion of autonomy, sometimes called influence, and complexity was a really attractive anchor for us. It let us start thinking about our skills differently, to say, instead of saying someone is a better oral communicator than somebody else, we started asking questions like: within Education Design Lab, how is the entry-level contractor communicating versus the CEO who is communicating with funders?

That was this contribution-influence layer that said a mistake at layer one and a mistake at layer four has different consequences. Then as the work gets more complex, how do you communicate complex ideas to an audience who doesn’t know about them? That’s a complexity of the skill of oral communication — can you continue to use your communication skills as the situation becomes more intricate, as the situation is more novel? That was another lever.

And then the last one, which I think is the focus of a lot of employment frameworks, is autonomy. This is where you start to see those more traditional things: you are an individual contributor, you are a member of a team, you are leading a project, you are leading a team, you are leading a department, you are in charge of a company. That more linear type of thing that you see in org charts was missing something for us, which was the level of independence — the degree to which you are acting as an independent agent with your skills versus being directed to use them, being told how to use them, and then executing them.

So these different levels in combination started to make this kind of follow, assist, apply framework. It doesn’t feel right or perfect yet, but it feels closer to something dimensional that’s not just flat — that shows how experiences can be communicated in a way that captures them a little bit better than just saying you’re a level two.

Mason Pashia: Yeah, and I think ultimately the subtitle for this report is about identifying a common language between education, workforce, and learners. So much of this framework, and as my understanding goes with the Sophia Framework specifically, had to do with internal mobility and navigation — so you could create a job description that had sort of these metrics across it, like a level three autonomy, a level two complexity, a level four contribution. That would be a very visible and transparent way for someone to say, I need to build that skill or build more confidence in that domain in order to move up in this ladder.

What that kind of inspired in us is: this framework works within the computer science domains of the CFI framework right now. But it’s a really useful one for actually stacking experiences for young people — starting them in the early three, which are follow, assist, and apply — things that feel like they could be done in an internship, an apprenticeship, and some of these other types of real-world experiences. What if we were actually able to start this now and then continue it into their career? So they’re already using the language, they’re understanding that visibility, and it actually helps put some flesh on the bones of the experiences that educators are trying to design.

There’s such a gap between what an educator knows to ask and what an employer knows to provide in order to serve the learner at the place they’re at. That is a really messy middle. I want to dig in a little bit more to that here in a bit, but this is just kind of the overarching image of this framework.

Since this is Zoom, you should be able to zoom on your end, but I’m just going to walk through these phases really quickly because we understand this is a bit of a framework soup — kind of like a framework inside of a framework inside of a framework. This visual in theory helps.

You have these key stakeholders: an employer, an educator, and a learner. Those are really the people we’re trying to serve through this framework. An experience is had by the learner along these different indicators we talked about earlier — autonomy, complexity, and contribution. The experience can increase in rigor, increase in each of these domains, and then it goes up to the actual storytelling layer. Already in that experience design is the place where the employer, the educator, and hopefully the learner have a conversation about what this experience should look like.

They apply these levers to build it, and the learner has the experience. Then it goes into the storytelling layer, where essentially the learner is starting to collect artifacts over time. They’re doing the reflection work — as was called out in the chat, we just didn’t have the time to do the reflection work at the time. So there are some examples of what that might look like, and the verification evidence up here.

Then it goes through the evaluation phase, which is much easier to do if you have this kind of structured domains framework. Essentially it gets disaggregated into those skills we’re talking about. So it starts to say, because they had this experience, it actually indicates that they have a pretty high ability to do collaboration. I think that’s where the AI layer starts to make inferences — because of the evidence provided, we can make an inference that this skill could actually be applied in multiple domains rather than just the one domain we’re currently setting in.

And ultimately, once all this goes together, that stacks toward enrollment and engagement and the kinds of things we’re actually talking about with these badges. Imagine you have a learner wallet that has 30 different experiences, each with all of this metadata stacking into it and culminating into this picture of skills and readiness. That’s kind of where we’re trying to get to, both with these internship-type experiences, but also with courses and classes, with service learning, with entrepreneurial and independent studies. That’s kind of where we’re at today, and we’re going to talk about each of those a little bit more in depth here in a second.

Research on Durable Skills Development: Michael Crawford and Chris Unger

Mason Pashia: Most of the research that’s been done on this — and Michael and Chris, I’m going to come to you here in a second — is about the storytelling or the confidence that comes out of a learner being able to talk about their experience. I think one of the most heartbreaking things is when people are providing these experiences and the young person either doesn’t know how to talk about it or they don’t really recognize what part of the experience was valuable to them. Then it just became this thing that kind of happened, and hopefully they build social capital in the process and that’s able to carry them onto their next thing. But without that metacognitive layer, you really start to struggle to identify what was valuable and transferable. Applying these different levels helps a learner navigate that. Ensuring the process of artifact collection really helps with that.

But Michael and Chris, you guys have been doing some really meaningful research lately around durable skills and applied skills — doing some amazing work conversing with young people about the experiences that have led to those skills. What have you learned in that process about how to get someone to talk about an experience and identify what is valuable in it?

Michael Crawford: Thanks, Mason. I’ll jump in and then toss it over to Chris. Just for a little bit of background and context: with America Succeeds we do work at the intersection of education and employment. About 20 months ago we embarked on a project to really understand what high schools were doing in support of durable skills development — what kinds of pedagogical approaches, what leadership looked like, how they were leveraging technology, whether they were interfacing with the broader community and in what ways, and what the experience was like for leaders, staff, students, and alumni who could reflect on their experiences in these programs.

Just two weeks ago we released a report on our findings. We had over 200 interviews with students, staff, leaders, and alumni, and really tried to make sense of what was happening at the schools that were part of the research project.

A couple things we found right off the top — I’ll give the top lines and then toss it over to Chris. These schools were making skills explicit. They were being intentional about what it was that they were aiming to cultivate in young folks — what their projects, internships, and other kinds of offerings were aiming at, and what skills are important.

Secondly, it was very valuable that these experiences were authentic. I think I saw somebody mention that they weren’t simulations — these were young folks doing work with outside community organizations, or self-directing projects that mattered to them, which made a big deal in terms of their own confidence and the way they had experiences.

And third, these schools were integrating skill development across the experience. It wasn’t just a one-off career class, it wasn’t just an internship in one area of their school experience, but these skills were on the wall, were part of rubrics, were part of reflection prompts, and were part of conversations that young people were having.

As you’re describing what kinds of experiences young people are having — in their reflection journals, for example, it would start with very limited description of the experience they had. But over time, as they had more repetitions and more opportunities to engage in that experience, their ability to articulate what it was that they learned, what they were struggling with, and what they were taking with them into the next year or after graduation expanded.

There’s something powerful there. Chris, you want to jump in?

Chris Unger: Yeah, just a couple highlights — things that I picked up personally as well as professionally. I was very thankful to have been able to work with these 12 or 14 schools over the course of two years, really thinking and talking with them and with the students — the learners themselves — about their experiences in the schools.

A couple things I really hang my hat on when I think about skill development with youth and learners: it often is interest-driven. I know that probably everybody on this call would recognize that when you’re engaging people in authentic, interest-driven learning, a lot more learning happens because you’re paying attention to it, you want to pay attention, you get into the weeds, you struggle with it, you think about it, and you engage.

So, student interest-driven learning. But the piece I’m going to build on from what Michael was saying is that even if we try to do experiences within schools, they’re usually proxies for the real thing.

For example, if you really want a kid to learn how to drive a car, you don’t take them to the mall, put them on a fake dummy car at an arcade, and say, now you know how to drive a car. It’s not the real thing. I’m a big fan of situated learning — you live in situ and you learn in context, which is what Meghan mentioned: context is so important. If you really want to learn how to drive a car, you need to drive a car, and you build the skills that you actually need by driving an actual car.

One of the things these schools have been able to figure out is that instead of creating proxies for experiences, they’re actually giving real-world, contextualized experiences. But the part that Michael also referenced is that you are naming the skills and engaging in a relationship with students to reflect on the skills they’re learning through those experiences.

My last quick version of this — which some of you have probably heard before — you take a group of seven or eight kids to Disney World, you have a great time. Three days later you come back home and you say, hey, what did you guys think about customer service at Disney World? And they’re all looking at you sideways like, were we supposed to be paying attention to customer service? Was that part of the program? I was just there to have a good time.

But if we had basically said before we traveled there, we’re going to pay attention to customer service, we’re going to talk about customer service, we’re going to identify customer service, we’re going to sit at lunch and see where we see it, and we’re connecting the dots about what customer service looks like — when you come back, the push to reflect on, to observe, to talk about, to be in a social relationship with one another talking about what that thing looks like in the real world is going to build your knowledge and your capacity and competency to identify those skills and recognize their utility.

So often kids are going through school getting these proxies of experiences, not really learning much — in one ear and out the other — because they’re never engaged in real-world, authentic activities that are interest-driven. And if they’re not naming and talking about the actual experiences, they’re not gaining the actual tools and skills and the metacognition to recognize which skills are useful and in what context.

Michael Crawford: Just the last couple seconds on that. I think one of the most valuable elements of being intentional, being explicit, having opportunities for reflection, is that it allows young people to point at things — whether they are particular challenges in an experience, a skill that they developed, or an emotion that they had when so-and-so said X, Y, Z. Reflecting and being able to point intentionally at certain things allows young people to then take the shell of those things and try them out and apply them in different places.

If you aren’t aware that certain things are happening, that you’re developing certain skills, that experiences have different dimensions, then it’s much less likely that you’re going to be able to take that and utilize it in other contexts. You might do a version of this sort of automatically and unconsciously, but not necessarily effectively, or not necessarily in ways that are pro-human, pro-social, supportive, or generative. Being intentional about it then allows you to reap the benefits of intentional application and improvement over time in other contexts — which I think is something that is lost in the way that conventional schools typically operate.

Mason Pashia: That’s a great point, Michael, and I love in your report how one of your key findings is about this skills visibility — the ability to name it and claim it. I think competency-based education has been beating this drum for a long time, and there has to be more of an integration between those two disciplines of real-world learning and CBE in order for this to really be sticky.

Also, Chris, that was the first director-of-photography moment we’ve ever had on a town hall — where you picked up your computer and went sideways. That was pretty exceptional. Oscar goes to you on that one.

Applying the Framework with Employers

Mason Pashia: Meghan, do you want to talk a little bit about how this might work for employers? If an employer were actually to apply the logic of what we’ve put into this report, what would that do?

Meghan Raftery: Yeah, I think this conversation feels very similar to the one we have at the lab a lot, which is about the degree being a proxy for skills — when somebody has a bachelor’s degree, you make assumptions about all kinds of things about how they gained that degree.

I think for employers, students are used to just being able to say, I had an internship at X, Y, Z place, and that’s a holder for lots of things that are invisible in that. What we wanted to try to communicate is that in this world of internship-leads-to-employment, students need to be able to describe that experience in a way that matters to employers, but also employers need to be able to ask questions in ways that unlock that experience. There’s a reciprocal relationship between how the storyteller tells the story and how the asker asks the question.

What we wanted to do is make this bidirectional so that employers could also start thinking about this. If any of you have ever had the experience of trying to match kids to internships, you kind of just let anybody who’s willing to take somebody do it, and the experience after the match is kind of up to the place that hosts them. Some of the supports we’re starting to imagine would be not only how do you match, but also how do you decide what the student will actually be doing? How does the student describe what they’ll be doing? How does it become a currency? Not just being there, but also actually having these things to aim for could be really helpful.

A really simple one would be an interview question — you know, tell me about a time you worked on a team, new question, describe a project where you — and then using these levels as a way to actually ask the question. While the student might be tracking these during the internship or during whatever the experience may be (and we keep saying internship as just a tangible one people understand), the employer could start to source that information because the language is common.

Kids could be able to articulate the experience in a way that resonates with the employer. The employer could start to say, here is the level of autonomy and contribution that I’m expecting from this person. I want to see experiences that either match that expectation or are indicators that they’re approaching the level that I would need for this job.

What we’re trying to do is increase those signals between the two sides, better using common language, using resources that might make it easier for people to have conversations with young people that are closer to what they’re experiencing and not based on assumptions about what that would be. There’s an equity piece to that, a teaching piece to that, and then also just a “you need something, how do you get what you need” piece to that, which we’re trying to capitalize on here with these tools.

Mason Pashia: Yeah, I think that’s really important. There’s a comment in the chat about the difficulty in making time for this reflection and growth. Part of the reason this conversation is so important right now is that these are the kinds of things it would be really easy for AI to take over and do — basically submit experiences and have AI do all the cognitive work to figure out what skills you demonstrated, et cetera. But as we just talked about with Michael and Chris, it is so important for the learner to be able to name those things themselves and do that metacognitive work of recognizing, oh, I demonstrated collaboration in order to make this change in this relationship at my internship.

I think it’s really important to have this conversation now, as these tools are no doubt going to be emerging, to make sure that we empower learners to actually do that cognitive work rather than offload a lot of it to specific tools that’ll get developed to do it.

Designing Experiences in Practice: Kimberly Hauser and Andy Hepburn

Mason Pashia: So then — this is just an example of the internship design by level. I’m actually curious — we’ve got two people here. This is just a hail Mary, so if neither of you want to contribute on this one, that’s fine. But Kimberly with Collegiate Education is here, and Andy Hepburn with GPS Education Partners is here. Both of you are doing really meaningful work at this intersection with the employers and the schools. I’m just curious: how do you think about either designing these experiences or advising the design of these experiences?

Kimberly Hauser: Thank you for the shout-out for CEN. We do work with small and rural districts, mainly in Texas, but we’re branching out to New Mexico and some other areas across the United States. I think what we do really well is we start with the community and let them tell us where those gaps are. Then we backward plan. We find out what it is they need, what skills they think are lacking. Then we go back into the schools. In Texas we have the PTECH program, which really allows for those industry-based certifications, and we follow the latest labor market information so that we’re really building those skills that are in demand in real time in our neighborhoods and communities. We kind of try to make it a whole community effort.

Mason Pashia: Love that. I think that’s super important. Andy, how about you — any tips for how you all suggest designing experiences, or how you play that intermediary role so well?

Andy Hepburn: Yeah, we’ve done a lot of work building out growth layers within our youth apprenticeship programming — setting out those competency maps to say, here are the points in time where we’ve intentionally built in reflection and 360 reviews, so that students and employers have to sit down with us and say, here’s where we’re going to check in on your level of development. Then intentionally identify those gaps for next-stage conversations, and really focus on the students starting to own those conversations.

That’s the really critical piece, so that by, you know, 12 months into a two-year internship or apprenticeship, they’re saying, here’s what I want to work on next, here’s what I see as my next opportunity. They’re pointing out what they want to see in the relationship with the employer. They’re identifying their next steps — that ownership, that agency, that you want to see develop as they go through that struggle.

I think that’s the thing you’re looking for. And really, even tying that into helping the employer recognize where to increase compensation and things like that, so that there are incentives on both sides and so they don’t get lost in structures that are alien to them when they move into the workforce.

It’s all about being very proactive and recognizing that these are first jobs, these are first experiences. They’re not going to come in and advocate for themselves with fidelity the first time out of the gate. You’ve got to build those structures in place from the beginning so that they know how to handle those conversations appropriately — and then give them the space, the confidence, and the coaching to be able to have those initial conversations so they can do that on their own.

Mason Pashia: Have you seen that friction around capacity or just the time you can dedicate to this? Or have you all found a way —

Andy Hepburn: Oh, absolutely. You’ve got to put the work in. We do a ton of coaching with the employers to make sure that they’re present, that they show up, and a lot of wrangling to make sure that they’re ready and able to be in the room the right way the first time. But once you build those repeatable processes and they see the return on that, it can be a wonderful experience. Once they own it, it’s a great experience for students and employers alike.

Mason Pashia: Yeah. I love what you’re saying about student ownership in this. That is so critical. And I think oftentimes that’s the layer that gets left out of a lot of these proposals, where it’s like, oh, let’s figure out how to design an internship, and there’s not a lot of thought about the ways in which the student will own that, either through it or at the end of it. So I really appreciate the work you’re doing.

Applying the Framework with Educators

Mason Pashia: Okay, so we’ve talked a lot about employers. Another obvious key stakeholder here is educators. We’ve talked about them a little bit, but Meghan, do you want to shed some light on how you’re thinking about educators using this tool?

Meghan Raftery: Yeah. I think this comes down to the fundamental challenge of school being able to translate out into work — another one of these translation issues of people using the right language to describe what’s happening.

We’re mostly clear in schools about what evaluation is and how it should be used. But this idea — and I know this is something I’m looking back on my time as a teacher thinking I didn’t do a good enough job with — is simulating real-life experiences. And then thinking about the currency with which students will take that outside of school into other environments was not a responsibility that I took on as the educator. Thinking about how they will communicate this outside of school — my job was to give them feedback on how well they did, but not necessarily helping them understand how to speak about the experience outside.

I volunteer a lot in my local school system here in Virginia Beach, and one of the things that’s increasingly common is that at the end of internship experiences, they’ll host employer interviews and I’ll be asked to come in and be an employer to interview the students. I notice a lot of times that even though the point of it is a final activity in their experience — having just done an internship — they won’t mention the internship in the questions I ask unless I directly ask them. So the teacher might not understand that the student isn’t creating this cohesive narrative for themselves between the experience and the goal that they have.

Part of it is being really articulate about what the student knows, understands, and is able to do. Part of it is putting that in the language that an employer can understand. Part of it is helping to coach the student in how to explain what they did and why it matters. Part of it is doing a really good job of matching the partnerships, training the employers, and finding the right people — hopefully people who are really committed to having these young people in their workforce eventually, who take a sense of responsibility, so that it feels less like charity and more like getting an asset.

The student is increasingly able to do things that actually matter and have value to your organization. There are lots of different pieces here. I think for educators — and I’ve understood certain parts of this through my own journey — authentic learning experiences mattered to me, but maybe I didn’t quite realize that the growth piece also includes this extra step, which is translation outside of the school environment to other audiences.

And also, not assuming that people taking on a student in an internship experience know how to do that — that they understand what the student can do in a way that makes it actionable for them once the person lands in their space, that the student isn’t just present, isn’t just following along, isn’t just doing grunt work. Although that could matter at certain points, there are more things they might be able to do if we’re better able to explain the skills they walk in with and the skills they’re trying to develop.

Next Steps and Closing Discussion

Mason Pashia: Yeah, I think that’s a great summary, Meghan. I’d also just say that I personally think this work has implications for less employer-connected work as well. I think you could pilot this with project-based learning — this is a useful way to think about how do you measure the really tricky, dynamic nature of so many of these experiences that we design and provide to young people. So if anybody here is dabbling in some upcoming PBL and you want to try some of this, please reach out to Meghan and me. We’d love to work with you to help either implement that and learn from you as you roll it out.

There are a couple of tools in this publication similar to what we’re just talking about — supervisor evaluation tools as well as a learner self-evaluation tool — for either the evaluator or the young person to actually assess along these domains. You can find those in the appendix of the report.

There are also a number of other recommendations specifically with regards to the LER — the learner employment record landscape we’re talking about. This might be about what metadata to collect along an experience, outside of the normal ones like industry and team size. There’s a bunch of contextual variables that are much easier to identify, but then there’s a bunch that are a little bit more challenging and they change dramatically from discipline to discipline.

If you are a nursing intern or shadow in some capacity, you can’t take photos of the work that you’re doing most of the time due to HIPAA. That looks really different than if you are a graphic designer and your whole portfolio is visuals and the things you’re building. So how do we make this communication consistent across these different domains and disciplines, where we have a number of good but challenging restrictions that accompany them?

That’s kind of this report — it’s a way to think about experiences, leveling them up, and identifying a language for people to talk to each other. Our next steps are to try to get out into some pilot territory and map how this would work, and identify the points of friction. I think we’ve named a few already — number one being just time. That is a really hard thing to do, especially if a key part of your work is getting employer validation along a process.

And Michael and Chris, I’d actually be pretty curious to know this. I imagine that most of the work you did in these surveys or conversations with young people happened kind of at the end of a project rather than incrementally over time. I could be wrong in that assumption, but I’m curious if you saw different results when you talked at the end versus during, and what your recommendation would be to really capture that dynamic, longitudinal learning-progression information as they move along.

Chris Unger: I’ll make this one comment and then, Michael, if you want to chime in.

Those of you who know Gibson Eck at Big Picture School in Issaquah — I know this is going to sound a certain way, but it’s kind of like a free-for-all. They’re thrown into these internships and personal learning projects, and they’re trying to figure it all out, as freshmen. So when you think about beginning and end, scale is interesting. We are finding that there’s a huge benefit to doing this over four years and not over six weeks or a project.

At Gibson Eck, you have to engage in personal learning projects and internships, and there are a bunch of other things that they’re doing there. But what’s happening is it’s very clumsy and difficult and challenging, because none of these students, through middle school, were ever even given the opportunity to be independent actors, take control of their learning, and become self-directed. It is quite a journey.

STEM School Chattanooga is a different beast altogether — I’m not saying one is better than the other, they’re just different. They actually have a very specific set of experiences, projects, and activities they do over four years. It’s very scaffolded. They are very precise in how they scaffold the kinds of experiences, the complexity of experiences, and the degree to which they’re self-directed over time.

It’s interesting to see how students feel incredibly well supported both at STEM School Chattanooga, because of the nature of the scaffolding of the projects they engage in over time, and at Gibson Eck, where the advisors and mentors are also providing support, just through two different kinds of pathways.

I do want to say one quick thing before we leave here: all this work has identified something super important, which is that when you get students to really focus on this and see themselves as agents of their own learning and agents of building their own skills, the sense of agency for students is huge. It’s a huge transformation. Right now, school happens to you. But if you are put in positions to be self-directed, to be engaged in these activities, to reflect on your experiences and on the skills you’re gaining from them, the sense of agency changes dramatically in a student.

I don’t think we should overlook that — we need to identify it, recognize it, and hold it up. If there’s one thing we want to do, it’s to increase their sense of agency and opportunity in the new future. It’s not just a good thing to do — the development of student and learner agency is a big, big deal when you do this work the right way.

Mason Pashia: I love that. Totally agree, Chris. Michael, you’re shrugging like he said it all.

Michael Crawford: Just — yeah.

Mason Pashia: Perfect. If there was a mic to drop, he would have. Amazing.

If anybody finds any value in applying this going forward, please let us know. We would love to follow that story, share about it, and learn from you as you learn with us. So thank you all — it was really great to see you. We really appreciate all your contributions in the chat and just in the field — everything you’re doing. Meghan, thanks so much for your support on this project as well.

Speakers

Meghan Raftery is an Education Designer at Education Design Lab. Specifically, she work on assessment and skills validation.