Town Hall Recap: Badging & Credentialing

On this episode of the Getting Smart podcast, we are airing a recording of our last town hall of 2022, all about badging and credentialing. On this town hall, we were joined by Project Director of Project Zero, David Kidd, as well as Director of Product Management at XQ, Rachel Safferstone. These two guests are doing incredible work in the space of credentialing and badging,

and both have the vision and the experience necessary to help us better understand the broader landscape. We really hope you enjoy this episode. And remember, if you’ve never been to a Getting Smart town hall, you can always find the next one at GettingSmart.com slash town halls.

We hope you’ll register and attend. All right, let’s jump in. Okay, well, welcome everybody. I think we’ll get started as others join on. So my name is Nate McLennan, VP for strategy and innovation at Getting Smart, and welcome

to our Getting Smart town hall on badging and credentialing, how students show what they know. So we’re super excited. We are over our heads or neck deep or head deep or whatever the way you want to describe it in badging and credentialing and thinking about how do we really shift the ecosystem,

the landscape of education to better serve learners of all ages. So a quick roadmap for today. I’m going to share just a few slides just about a basic overview of our thinking right now around credentialing. We then have two awesome experts who I’ll introduce in a minute who are going to talk

about our credential model, like a big picture credential model, how this could work. So David will talk about that. And then Rachel’s going to talk about a specific initiative at XQ really looking at how this is happening in real time. So that’ll take up the first half of the town hall.

And then we’ll open up to a bunch of discussion questions, which we always do, which will be soliciting information from you all questions from you all and really having a conversation back and forth. So that’s our roadmap for today. So a couple of things on credentialing that we believe are true at this moment.

So number one, there’s a whole lot of building blocks that are out there. And these building blocks need to operate together in some way in order for a truly credential system to work. So the building blocks that we’re articulating on this side is you need to have some articulating of competencies, i.e. what you are credentialing.

There have to be linked learning experiences so that learners either co-design, they build or educators build or designers build experiences that allow learners to demonstrate that they’ve reached competency. Some sort of performance assessment that’s linked into the learning experience that allows us to verify the badge and or credential.

And we recognize and acknowledge that there’s a lot of different uses of badges, the word badge and the word credential in the ecosystem. So we’re not at this point, we tend to say that badges may lead, a stack of badges may lead to credentials, but there are others who believe differently and there’s no real accepted definition for one or more along the lines of credentialing, less along the

lines of badging. All of those, the badges and or credentials are part of a learning and employment record, a lifetime record for learners from cradle to grave about what they’ve learned and where they’ve been, they’ve worked. And then that then links into workplace and higher ed readiness.

How can we better match learners as they’re moving into, say, higher ed or any post-secondary experience, learning experience, and then eventually into a workplace? So we have the best match for, again, all learners to access family sustaining wage and all workplaces to have access to the best possible talent in order to maximize sort of progress in the world.

So those are some building blocks. One thing that we want to acknowledge for those of you who remember your statistics one on one courses from either high school, hopefully, but more likely college if you were privileged enough to take a statistics course. But right now we have a number of existing measures.

And if we look at standardized tests and all the things that are associated with them is that they are highly reliable, but we would pause it, they’re not valid. So they may be really, they can cluster and they’re not super divergent in how they push results, but they’re not necessarily valid in describing the entire learner. And so where we believe credentialing heads or leads us to is something that’s both reliable

and it’s valid. And that idea that we have a better measure of student success and student learning using a credential and badging system. We also recognize I didn’t put it on the slide, but we may in that process may produce something that has a little bit less reliability, but is more valid.

And we believe that’s better for the human endeavor in the long run. So just a statement on validity. So when we think about credentials in a broad way is we’re trying to find a better way to accurately document human talent and proficiency for all learners. That’s the goal here for us.

And so in that ecosystem, there is a ton of work going on that we won’t get into today. But I just want to acknowledge that in a credentialing system, you have someone who issues the credential, you have someone who receives the credential and you have someone who verifies that credential, possibly all stored in a central place, wallet, etc. One Ed Tech and the World Wide Web Consortium and IEEE and a lot of associated working

groups. I don’t know if Greg Nadoos on the line, but he is leading up a working group on really trying to define all the infrastructure, language, interoperability, the complexity that lies when we try to build a credentialing ecosystem, as well as a number of technology providers, many of them building on blockchain around the Web3 world that are trying to figure out the technology that’s associated with this.

So I’m just acknowledging there’s a technology piece and there’s an interoperability piece that are really, really highly linked to this endeavor of trying to create a badging and credentialing ecosystem in the world of education. And then finally, there’s a ton of challenges that all of you who are interested in this, I’m sure are working on some or all of these parts and I’m sure we’ve missed a lot in

here. But just this idea that there are barriers in K-12 and post-secondary and that what they are actually transcripting may not match to what employers necessarily need or accurately describe what the learners can demonstrate that can do. Also, they’re stuck in time-based models, the Carnegie unit restrictions. Those two pieces are huge barriers, so skills mismatched and time-based models.

In the connection between K-12 and post-secondary, you have admissions policies, even NCAA requirements get in the way of some of this work. Policies around what’s accepted in terms of admission and then just verification and value. If every learner has credentials and it’s on a transcript of some sort, does that mean something for the employers? Does it mean something for post-secondary?

For the employer side, they have a huge amount of work to get HR resources and HR capacity to actually link into these systems of credentials so that they can find the right match for the talent that they want. So that skills matching piece. Just a handful of barriers that are out there. With that in mind, I’m going to turn it over to a few awesome experts who agreed to come on

and talk today. I know there’s a lot of chat are coming through, so I appreciate that and I’ll take a look at those in a minute. But David Kidd, welcome Project Director for Project Zero out of Harvard and then Rachel Saffersow and Director of Project Management for XQ. And we’re just so grateful that you all joined us today. The way this is going to work is that David’s going to talk a little bit about the landscape

view and an idea about how this thing could work. And then Rachel is going to give us a specific example of how it is possibly working right now in four states that they’re piloting and working in. So that’s where we’re going to go and then we’ll open up the second half of the discussion. So David, welcome. You get to kick us off.

Thank you so much, Nate. And thank you all for being here with us today and honored by the opportunity to talk a little bit about some of the work that I’ve been doing with my colleagues at the Democratic Knowledge Project, an initiative of the Edmund Lilly Saffers Center for Ethics at Harvard. Earlier this year, we released a white paper titled A Call to More Equitable Learning, How Next Generation Badging Improves Education for All.

So that’s an ambitious subtitle. So I owe you a definition of what we mean by this next generation badging on the next slide. We define it simply as measuring and recording learner mastery of competence. These and reporting on that mastery by awarding learners badges that are accredited by an independent body of assessment experts. And, Nate, if you could click about three times,

it’ll pull up these things. So we believe that what we can do with this new system is realize some of the opportunities promised by badging to increase transparency about what learners are actually achieving, what competencies they’re really attaining. Right now, letter grades and course titles are pretty much black boxes for anybody reviewing a transcript. And we think that badging can get us beyond that by showing by focusing on actual competencies.

We also believe that we can introduce more flexibility into our current learning ecosystems with badging. And this comes from really two things. First of all, moving away from the Carnegie unit as our primary unit of learning, which as Nate mentioned is based on seat time in a limited location. But in order to achieve that, breaking away from the Carnegie unit, we also need to have a system for ensuring credibility that allows for cross context

comparability of badges or indicators of achievement. We believe this will come together to make our education system more equitable. And that will lead to better outcomes for all learners as they work towards achieving their personal goals. So I want to go into a little bit more detail about the question of credibility. Thank you, Nate. So one of the questions here is how do we ensure that people know that a badge means what it purports to mean? As Nate mentioned,

when he was talking about validity, there’s a difference between reliability, getting the same thing over and over again, and true validity, which is actually getting a read on the thing that you’re looking for. In order to achieve validity, we believe that we should orient around something called principal assessment design, which presented very simply, calls our attention to three elements of valid assessment. The first is a clear understanding of the underlying competency or cognition

that we want learners to be able to demonstrate. The second challenge is once we have a clear understanding of that competency, we need to think realistically about how we’re going to observe learners demonstrate that competency. This might involve identifying an existing opportunity. It might also involve engineering designing an opportunity to observe authentically their competencies. And then we need processes for transparently, reliably, and fairly evaluating

what we observe and linking that interpretation back to the underlying cognition. In order to determine that an assessment process is valid using principal assessment design, you have to evaluate the strength of the evidence supporting the links between cognition, observation, interpretation, and back around to cognition. This can be done by an external board, which we would call the badging board, that really just looks at the strength of the evidence supporting the

assessment system. This allows us to have authentic assessments that differ across contexts, while still providing a credible indicator of a commonly understood competency. So we don’t believe that we should be standardizing assessment in terms of specific items on the test or something like that, but we should be standardizing around a shared understanding of the learning outcome and a shared understanding of what it means to have a rigorous valid assessment.

When we can achieve this cross context validity of assessments, that allows us to really achieve flexibility. And if we look to the next slide, what we’ve done is we’ve tried to think about what it means to break apart the Carnegie unit and look at learning in terms of competencies and how that works with a badging system. So we’ve proposed three tiers of badges at the smallest level or micro badges. These reflect sort of task level performances. We would use them as formative

assessments. They allow the learner and the educator to see where a learner is on their pathway. Once they’re ready to attempt what we call a mesobadge, this is when they’re ready to demonstrate a full competency. And it’s the assessment that backs up the awarding of the mesobadge that needs to be externally validated. And the reason for that is that it’s the mesobadges that we believe aggregate up into macro badges that reflect more curriculum level achievements

in a traditional system. So these are collections of competencies that genuinely prepare someone to succeed in a particular domain of practice. The importance of the macro badges is that they’re much easier to digest for people reviewing transcripts. And they also help ensure that learners are on coherent learning pathways as they collect mesobadges across learning contexts. There’s a lot of questions that we still have about how to implement this sort of strategy,

and we’re continuing to learn from people in the field. So on that note, I would like to join all of you in hearing from Rachel, the next presenter. Well, thanks, David. Super helpful. And I just want to pause for a second because Elise asked a great question as you were talking, is that we’re talking about assessment and we’re talking about competencies. And Elise asked, is there consensus on which competencies? Is there a master set of competencies

that are out there? And I think a couple of different answers. Rachel will give an example of how they tackled that. From a competency perspective, the switch my slide here. So XQ has an example of a competency set, a set of learner outcomes that are, there’s a progression that’s associated with them that all of their network schools are using, as do states who have adopted portraits of graduates. So South Carolina has a portrait of graduate, Utah has a portrait of

a graduate, etc. So there are states that have done this. And of course, schools that have done this and districts that have done this. So there’s a lot of work. I think the best message that I heard from the person in Kentucky or a group in Kentucky that was working on this is that they’ve asked a number of districts, 40 or 50 districts to do this work, divine a portrait of a graduate, and a set of master competencies. And they found that the overlap was like 90 to 95% or

something like that, even though the language might be different, the overlap comes to some sort of common consensus. So something to think about in terms of, is there a master set or a agreed upon set of competencies? So just as an intro to Rachel, just XQ is doing a lot of work around, how do we rethink high school as a whole? How do we disrupt the Carnegie unit? And part of this is a set of learner outcomes that I just alluded to. Rachel’s work is a specific strand that’s

content based on how do we do badging in the math world as part of a large number of XQ, different XQ work streams. So Rachel, you can explain it better than I can. So you’re up. Great. Thanks, Nate. Hi, all. I’m Rachel Safferstone. I work with educators, researchers, and engineers to build resources for high school students and systems and schools at the XQ Institute. If you’re not familiar with XQ, we work in communities throughout the country,

both schools and systems, to help them dream big about what high school could be, how to turn their innovative ideas into action, and how to create more rigorous and equitable schools. In addition to providing grants to schools to support these efforts, we also build open source tools in places that we feel like aren’t being well addressed in the market. This badging work is interesting to us because we think it has the potential to really improve

college and career outcomes for high school students through greater flexibility and pathways and progressions through high school math. So what are our math badges? Working with our colleagues at Student Achievement Partners, we’ve developed a set of 21 badges that cover the material traditionally associated with Algebra I, Geometry Algebra II. We’ve also tossed in some data science there also to give them the growing interest in that terrain. Each of these

badges comprise material that is substantially bigger than a single standard, but much smaller than a full year-long course. So for example, and you can see sort of outlined there in the hot pink, a traditional course like Algebra I would actually comprise six badges on that set. You’ll notice on this slide that there are some lines that look like precursors, but there are actually very few of them. We’re trying as much as possible to really only identify prerequisite

badges where it feels truly warranted by the material to give educators and students as much flexibility as we can to have the badges aligned to their interests and the pathways they’re looking at. So just we’ll talk more about the pilots we’re doing, but right now for most of our pilot, students will be engaging with these badges in traditional class settings, but we believe that breaking badges, the high school math that is down into these different smaller badges,

will promote more flexibility for students and allow them to progress through math at their own pace and to support differentiated pathways and quite likely learning outside the classroom. This is just an example badge. So you’ll see in this case exponential modeling. Just as a brief note, you’ll see that we emphasize real world application. So we’re really trying to address that question that many students ask in high school math, which is when will I ever use this?

So we’re trying to actually make it very clear when you would. And also when these math concepts would show up in real careers. We didn’t want to build out this badging system in a vacuum. So we actually really enlisted a series of states very early in the design process to provide feedback on the badges and actually to pilot them in classrooms. We like to think about these badges as really being building blocks that states and systems can

use to configure their own initiatives and priorities. So each of these different states has a somewhat different area of focus, but they all share in common and interest in promoting equity and student engagement. This is from Illinois. So what they are doing is actually to align the badges to a program that they call transitional math. The goal of that program is to actually promote college readiness for students who wouldn’t actually usually take a senior year math course.

And so what you’ll see is that each of those pathways of those three that you see outlined there are actually pulling together a few badges and students will actually take coursework, align with those badges, allowing them to progress on to college at a much more career-ready standpoint. They’re also in the middle of creating a virtual course that aligns to those badges, also too. So students can take them in their own schools or they can actually take a virtual course

via a statewide symposium. So just to give you a sense of our timelines here at the states, they’ve all recruited pilot districts and are working to engage a range of stakeholders in this process, so higher ed, school leaders, teachers, and ultimately students and their families. They have roughly five to eight or so pilot districts in each state. And right now, those teachers in those schools are working to map the badges to their current standards.

They’re aligning coursework to those badges and thinking about how to involve more engaging activities in their curriculum. And then within XQ, with our partners, we’re working to build up assessments for each badge, both some traditional computer-based assessments and also more innovative ones like portfolio reviews and performance tasks. For those performance, portfolio reviews, we’re actually building on exemplars with teachers so that every

portfolio will actually have an example of what student work looks like in terms of mastery in each badge. And we’re also building out, again, performance tasks where for many of the badges that have a modeling focus or thinking about real representations of math, like viral spread, for example, in the world, we’re building on these performance tasks that students would actually engage in as part of an assessment. Rather like in David’s words, I think you said engineering

an opportunity. So these performance tasks would be actually engineering an opportunity for that kind of assessment. And then we’re also building out some professional learning opportunities for teachers also too. So we’re very excited to see this work unfold across the next school year in 2023, 2024 and beyond. Thanks so much. I’m not going to let you off the hook just yet because there’s at least asked another great question, which is the work, so standards are embedded in state

legislation across the board in all 50 states. How in the, specifically in that math work that you’re doing in those four states, how was that mitigated? Was that an easy process because it’s very content focused? Or do you have any thoughts on standards use plus badging and the integration of those? Yeah, I would just say that we have worked, the initial badge set kind of looks back at traditional standards at a cluster level. So we have had an initial mapping of the badges to

traditional clusters, which makes it fairly easy for each state to then go through and actually do that mapping themselves. So each of the states as part of their process actually has done a mapping of their state standards to the badges. And I think that brings up the topic of where, and Rachel, you’ve been part of these conversations. It’s easier to think about matching standards and badges when the badges are very content specific, like what you all have done. It’s

more difficult when we start introducing these portrait of a graduate or learner outcomes framework, frameworks that XQ has done, because they don’t direct that some of them may connect to standards, but they don’t directly connect to standards. So we’ve been having these really interesting discussions with XQ and others about how do we make sure that the standards work, which is so embedded, can be also parallel to this competency work and where are the intersections between

those in those areas. So I just think it’s a really good question that we get asked a lot. David, one question that I also think is important is just can you just state one more time that you throughout the word badging board and we didn’t get any response in the chat, which was surprising to me, but can you just talk again about the distinction between a badging board that you said the badging board is not going to sort of assess individually students, but it’s going to, I think assess those

who award badges. Can you just make sure you state that one more time for everybody? Absolutely. And thank you, Nate, because this is really something that sets the strategy we’re proposing apart from like a portfolio based approach. So the difference here is that we’re really looking to ensure that there’s external review of an assessment process rather than an external review of a student’s performance on an assessment. So if we think

about it in the context of research where different researchers might study something like self-esteem and depending on the population they’re working with, the context in which they work, they might measure self-esteem in different ways, but their colleagues would want to ensure that they have evidence that the way they’re measuring self-esteem has validity. And so what we’re proposing is that we don’t need an external body to grade each student’s responses. What we need is an

external body to look at the process that’s used to award badges and evaluate whether or not that process itself has credibility. If the process for awarding the badges has credibility, if the sort of classroom based evaluation that’s happening has credibility meets external standards of validity, then that means that a badge awarded using that system can be honored by people outside that immediate context. So we do believe that assessment is best done by those closest to

the learners and those who are aware of the learning context. That means that we really don’t want single national assessments. We want assessment to vary across contexts, but we want the rigor to stay the same so that when somebody looks at a badge they know that there’s a real assessment behind it. Yeah, that’s super helpful. And I think that addresses Tony’s question, which was rightfully asked about it seems very daunting about the badge creation. But Rachel, you’ve done a

ton of work on badge creation. That took a lot of people and a lot of time to do that, but it’s making sure that it’s adhering to a set of concrete principles is the idea that David’s presenting, is that their research base is authentic and so that everybody is being using those same set of principles. And I think that’s super important. Just addressing Rachel, this is a question for you, I think just are you at the stage yet where badges have been awarded

to students? And if so, Colin’s asking like, are they being integrated into a technology yet or digital wallet stuff, things like that? They are not being awarded yet. They will start to be awarded across the 23, 24 school year. At present, we don’t have a technology platform that we are kind of promoting as a mechanism for tracking that. It’s a very kind of local state decision about how to do that. We are providing though the assessments that can be used to help to support

that. And I know Colin, you’re involved. There’s a lot of conversation about that, like I said in the beginning about the technical infrastructure. And North Dakota has just announced there as an example, North Dakota has just announced a statewide release of a statewide digital wallet, which I think is the first state to have done that. So if you’re interested in that, there’s tons of work being done in that area. One more question, another one for you, Rachel,

is the question about content. I know, having looked at the badges that you’ve thought about this, but considering what’s relevant and what’s not relevant to the future in terms of what’s the right math, what was the right math to include in the badges and how much leeway did you have to say, data systems are important or whatever and exclude some things and include other things. To be really clear, we’re not specifying a curriculum and we’re not saying what badges

must be adopted anywhere. We’re simply giving people blocks to use as they see fit is kind of the first point. So like we’re not saying that this actually is the regimented work that you have to do in any given area. We are saying if you are looking to actually look at linearity, here’s what linearity actually means. So I guess that’s point A. Point B is that within those, what we’re doing is looking much more at what I’d call traditional cluster level. So we’re trying to get away from

kind of the atomized focus on the most minute level of standard mathematics. So I think part of what we’re trying to say is what matters is the forest, not the trees when we’re looking at any given concept area. And then finally, we actually are looking to imbue, I think a lot of what people are thinking is much more important for students or for many students anyway, in the realm of data science. So we actually have a whole series of badges around data science, which we all know is kind

of important for today’s world. There are so many good questions coming in. Thanks, Rachel. I’m scanning questions and listening at the same time. I’m going to move from math for a second and take us back up to the credentialing concept as a whole. And so Joe asked a great question about what’s the connection between current work and higher ed and then higher ed to industry. Tom, I wonder if you could just take a quick take on that because you and I and the rest of the

Getting Smart team have been really thinking about that a lot. So thoughts on that and then we’ll open it up to others. Yeah, thanks, Nate. It’s a great time in this dialogue to remember that what we’re trying to do is create a two-sided market, right? A send helping learners send valuable information to, to facilitate talent transactions with universities and scholarship organizations and ultimately with employers. And so there has to be value for all of those parties

and for, for credentials to become part of. And then eventually we think the full communication system for human development, it will need to be valuable for learners and teachers and, and businesses and higher education. And so the, the badge board that David described would have to be have some representation from the business community because they’re ultimately one of the recipients of, of these credentials. I’ll note that Dan Tesfaye is on and Dan worked with the

Kauffman Foundation in Kansas City. Dan is helping four 31 systems there that are credentialing valuable experiences, not competencies, but experiences like client connected projects and internships and entrepreneurial experiences, sort of bundles of competencies that the business community values. And so they’re hoping that, that by badging those experiences, it’ll be a signal that’s useful to the business community. We do think that skill credentialing is going to follow that, but we

think it’s an interesting development to credential experiences as well as skills. But, but ultimately we’ll find out if that’s valued by the recipients of those credentials. Right. There’s a demand issue that everybody’s wrestling with. So, so in order to, in order for a credentialing system to be in demand, a lot of barriers need to get lower. And so employers need to make it easy for them. Their current HR system, it’s, it’s great.

They, someone sends in, they use a service or whatever, someone sends in a piece of paper documenting or call the CV or a resume, and then they can assess from there, whatever we replace that with has to be easier than that system. And it has to be more accurate and allow people to get more talent. Admissions from say higher ed, if someone’s moving on to higher ed, has to realize that they’re going to get a better fit student body by looking at schools in K-12 that give them

credential type experiences. And, and I think the one other thing to acknowledge, I think here is that credentialing is not a new piece. Most, many of you might be, have been involved in CTE world and credentialing still exists and has been existing in the CTE world for a long, long time. But it’s been very, very specific to some percentage of students and to some percentage of industries. And so what we’re proposing as a collective whole here is that that concept should be

broadened to the entire ecosystem because it’s a better descriptor of long-term human performance and long-term sort of who a, an individual is. A lot of chat here. I want to, I want to pose a question. We have a bunch of questions, but a lot of them are being answered. What barriers have we missed? So we’ve talked about a few different barriers, but either David or Rachel, any, anything that you want to point out that’s gotten in the way, David, in your,

the theoretical work that you proposed and Rachel in the tangible work that you, that you didn’t expect or that you wouldn’t expect? I don’t, this wasn’t a thing I didn’t expect, but it’s a thing we’re encountering. And I will say encountering with, with educators who embrace it with open arms, but it’s, it’s a little daunting to think about moving from a course to a smaller unit. Because you’re, you’re putting a greater emphasis then on a piece that you, that you wouldn’t have

seen quite in the same way. And so a lot of the work that we’re really engaging in with educators now is thinking about how to do that kind of mapping and think about how to break things down in that way. And also, I think it kind of puts a spotlight on places where they wouldn’t have felt their curriculum is currently meeting the bars they wanted to. So that’s a challenge also, I think. Yeah, that’s helpful. That sort of granular change, step down change, right? Which is the irony is that

educators have been in this level of granularity for a long time. They’ve built modules. If they’re a PBL type, project-based learning type educator, they’ve built projects. And those are linked to, in this case, standard or in their case standards. But it’s the systems themselves that aren’t necessarily used to that level of modularity. Is that, would that be a true statement, Rachel? Yeah. David, any thoughts on that question? Barriers that you either uncovered in your work?

What were thoughts? I will say that one thing that’s been a little surprising to me is almost an illusory barrier that I had imagined, which was that the credibility argument was really key. And I think it’s become clear that the reason that big standardized tests and GPAs have so much stain power is not because people are just bowled over by the evidence of their validity. It’s because they’re really convenient. And so the sort of barrier, I think, is actually less

one of showing that we can have a more valid system. It’s showing that we can have a system that’s equally convenient for the people making decisions. So that’s just been sort of an interesting shift of perspective for me. Not to say credibility isn’t important. I think it’s really key. But I sort of neglected the convenience factor. I think it wasn’t as clear to me at the start. Yeah. That’s super helpful. Equally convenient is a good way to describe it.

I like people that push back on us. So Michael, I’m going to give you the mic for a second. Briefly, can you say what you’re talking about? We need to go beyond badges. So what’s beyond badges as succinctly as possible? Yeah. So succinctly is that badges themselves are an aggregation of information that then needs to be both assembled in a way that makes sense and then disaggregated in a way that’s consumable. And in fact, if we’re using the tools that are available

now in data science, we can disaggregate this and instead take the same kind of approach that we take with accounting and take a ledger based approach where there’s a series of entries. That’s what I put in the link in the chat, an article there that I wrote about a year ago, ideating on this. I worked on badges. I ran the Cities of Learning Initiative that I started here in Washington, D.C. for several years. And I just hope that people can learn from

what we’ve learned there and save themselves just a lot of headache. Because I too believed in badges for a long, long time. And I think that we have to take a step beyond. That’s just my own perspective here. Thanks, Michael. Yeah, appreciate that. Take a look at his article that he linked into the chat. It’ll also be in the notes as well. Tom, you talked about convenience and then you had a few other statements there. Share with us what you’re talking about there.

Well, to David’s point, convenience has ruled the day. And I think it’s part of why tradition, our tradition of courses and grades has persisted. But what we’re beginning to see in colleges and now even high schools are learners that are building learning and employment records and then being able to permission those to some colleges, some scholarship organizations, some employers, and allow those people to have access to all or parts of a learner record.

And so instead of this hundreds of years, we’ve helped learners build one transcript that they sent to everyone. We’re now going to be in the business of helping learners curate records with a lot more fine-grained data, including credentials that they can permission and then receive inbound offers of employment work, scholarship. And so I do think we’re at a new stage where there’s an opportunity to populate a record with a lot more and better

information. And I think by the end of the decade, we’ll think differently about convenience of communication. Yeah. And there’s a lot of technology work going into that. As we know, someone, Mark mentioned, LearnCard. Tom, you and I had a demo for Territorium the other day, which has a digital wallet embedded into an LERA. Greenlight credentials is doing this in Texas. Smart Resume is now being used in Kansas City. It’s a big footprint in Arkansas. So we are seeing

pretty rapid growth of these of these LERs and increasingly being populated with credentials. Yeah. And MIT has an open source one, and that’s downloadable online if you go to their lab. Mastery transcript is developing a learner record system to complement their transcript tool. And I think just as a, if you’re not in the sort of the learner record digital wallet space, I think ultimately the user experience is you’re going to be able to choose a wallet,

and that wallet’s going to be able to store anything that you’ve learned and anything that you’ve been like your employment record, and you’ll be able to interchange between wallets. So it’s not going to be one winner of the wallet race. It’s going to be multiple winners, and it’s going to be a convenience in terms of who uses it. I want to, there’s a lot of good stuff here. James, you had a really good preamble, and I, if you could just talk briefly about what you’re

talking about there, because it’s a really important question. So if you could give us a quick thought on that. Well, truly just the question about including the things that take place more peripherally to what we’re looking at measuring here, and even with the people we normally entrust to measure it, looking past the advisors really to the person who’s leading the improv class that the student goes to twice a week, which should probably be included in this

mix. And so, you know, are we, are we able to include that, especially as it broadens the question about validity? Like, is this certifiably accurate? Are they following a good standard? That kind of thing. I think maybe it’s another barrier. I’d love for us to include that now in the 21st century, and how we look at learners. I think there might be a practical question in the near term between the assessment and the acquisition of knowledge. We would love over the, I’ll call medium term

even, for that acquisition to happen outside the classroom. I think in terms of the types of assessments we’re looking at at least, we have some computer based assessments that are more traditional, but for some of the performance tasks, for example, and a portfolio assemblage, I think we would still expect to see that being assembled by a mathematics teacher, for example. But certainly it could be learning the knowledge and gaining that outside the classroom. But then

maybe over time it becomes much more open also too. And I would like to just add to what Rachel said. First of all, to echo that, yes, we think that out of school learning is a very rich area and needs to be better incorporated. Right now, students who are, who have the coaching and the skills to write a really compelling college essay, who do the right extracurriculars that are well recognized, they get credit for that. But for somebody who’s mastering a second or third language,

a volunteering in a community center or who has a job after school, they’re not necessarily getting credit for that unless they’re really good at showing it. And we think that badging is a way to recognize those competencies being developed outside of school. And I also wanted to point out that in our system, the goal is for out of school educators, they might not be called educators, they might be mentors, to be able to at least guide a learner towards a valid assessment practice.

So one of the things we’re working on right now is working without a school educators to think about how we could help them develop assessments. They don’t have the resources to do that themselves. But the fact is, these are people who work with kids every day, they have a really good sense of whether or not kids are learning or not, they just don’t have the tools to formally document that. So we think that there’s a need for more professional development, there’s a need to invest in resources

for these educators, but we don’t see any reason why a learner cannot be evaluated just because they’re not on a school campus or in a classroom. So in principle, that’s what we’re aiming for, is that vision of the future, it’s going to be hard to achieve. There is another whole body of work about this unbundled ecosystem, and Mason maybe can drop any information on that into the chat, but we’re thinking a lot about how does the system get

unbundled so that these opportunities are out there for all learners. Rachel, you asked a great question, who designs the performance assessment around that? Out of ecosystem folks have been really good at creating experiences and have done so for years and years, and it’s connecting these ecosystems together, I think is really important. So you can see Mason dropped in a publication we just produced on Unbundled Ecosystems has a ton of policy

places in the country where this is already happening, different parts and pieces. I also just want to acknowledge, I think Rachel, you reinforced it, but Eric’s statement about Carnegie is that the Carnegie Foundation for Advancement of Teaching is who originally created this Carnegie unit concept is really trying to disrupt the Carnegie unit right now, and they’re on the other side of that issue, which is fantastic and really thinking about how that all works

together to advance this credentialing work and badging work, etc. So we are at the end of our time, so many good questions, and we didn’t, there’s a lot of questions about it, like how do you assess things like self-esteem? Should we be assessing those that are topics for another conversation? Please reach out and connect, look at our show notes that we’ll publish, and please reach out to us if you want connections to anybody that we’ve referenced on this particular town hall.

Thanks everybody, have a great afternoon, evening, morning, wherever you are in the world, and we look forward to seeing you on the next town hall.