Who Should Own a Learner’s Data in an AI World? | Catching Up

In this episode of Catching Up, Nate McClennen and Mason Pashia talk about LearnKit, a proposal for learner-owned data and learner wallets in an AI world. The conversation also covers the second season of Sold a Story and what schools can keep from direct instruction while still making room for inquiry and student agency, what students say they want from school, and how pathway schools in Tacoma and Issaquah are connecting learning to community. Nate also explains how NAEP works and shares a school shout-out for Harwood Union High School’s competency-based transcript, and the episode closes with an AI-made song and a challenge for listeners. A useful listen for leaders and teachers thinking about who controls learner records and which approaches fit their students.

Transcript

Introduction

Nate McClennen: Mason, great to see you today. Excited to catch up.

Mason Pashia: Nate, likewise.

Nate McClennen: All right. What are we gonna learn?

Mason Pashia: We’ve got a lot and a super deep dive. We got a proposal for data integrity and ownership called LearnKit that we recently put up on our website. We ask some questions about belonging and really the role of pedagogy in the classroom. And at the end, we talk a little bit about what is human about the new movie, You Can See Everything.

Nate McClennen: We’re gonna do a little bit more on testing and talk about the NAEP test. Not too long, only one to two minutes, so if you don’t like standardized testing, don’t worry, the whole episode’s not about it. We’re gonna dive really deep into this new season of Sold a Story, and it’s really unpacking direct instruction, and I think there’s a lot that’s fascinating, not only about direct instruction as an approach, but also the meta effect of storytelling. We’re gonna throw in some AI tidbits. We talk about a few different schools this time, so stay tuned for that. And as always, we’ve got a great song at the end, and my challenge for you all is to see if you can spot the convergence in lyrics over our last almost 20 episodes now.

Mason Pashia: I hope you brought your vest and suspenders for this song.

Nate McClennen: Your vest and suspenders, absolutely.

School Visits in Tacoma and Issaquah: Pathways and Community Learning

Mason Pashia: Oh man, it’s been a busy few weeks. We hosted some visits up in this area for some Kansas City folks to come check out some schools, which was super cool. And we got to see some awesome examples of authentic learning experiences, some pathways. This was in Tacoma Public Schools and Issaquah Public Schools.

And it’s just really inspiring the ways these schools are finding their footing in and of community, I think, is really the takeaway. That you have Issaquah, which has a Big Picture Learning school called Gibson Ek, which has internships that go out into community, and always has, and they do a ton of them over the course of a year, which is amazing.

And then in Tacoma Public Schools, you really are having a ton of schools built in partnership with industry, so they really involve the partners in the development of the school, which keeps accountability kind of tied on both sides of the equation in a really unique way. And I think it was really helpful for a lot of these KC leaders to see as they’re thinking about real-world learning in their own context.

Nate McClennen: Yeah, so Issaquah has a couple different high schools. BPL, Big Picture, is one of those. It’s an option for students, right?

Mason Pashia: They have a couple different high schools. I think Gibson Ek is the only Big Picture high school though in Issaquah. Yes, yes. But this is one of the Issaquah School District high schools. Correct.

Nate McClennen: And in Tacoma, where they have a bunch of pathway schools, there’s probably some comprehensives as well. Or are the pathway schools… I know I’ve been out to a few of the ones in Tacoma, the one at the waterfront, SAMi and the arts one, et cetera. But those are choice schools within the district, correct? And then there’s comprehensive schools as well that have their own pathways, like the maritime pathway or whatever the case may be.

Mason Pashia: Yes. So there’s a couple of those… There’s the three kind of original, I think they’re technically called contract schools, but these are like their innovation hubs. So there’s SAMi, SOTA, and iDEA, ranging from arts, science to, like, industrial design and engineering. And now they have this new maritime facility that is just dedicated to maritime pathways.

And right now they’re also housing cybersecurity in there, but they have plans to kind of move that out into a new campus. So they’re really making these kind of hubs for each of these pathways, and then their comprehensives also have some really great pathways. I know one of their comprehensive high schools in particular is renowned for their arts program, but they also have an arts and theater school in the district.

So it’s kind of cool to see how they are able to distribute them, but also have a focused concentration if students want to either learn differently or get more hands-on industry experience.

Nate McClennen: Yeah. It’s super interesting. I think that there’s this interesting… I don’t know if it’s a tension, but it seems to be a model choice. I know that some of the teams are gonna be down in Texas this week at some sites that are wall-to-wall, meaning every student is in a pathway, and some sites which are comprehensive, but a choice academy or a choice school, whatever the case may be.

And I don’t know if I have an opinion on that. I think if every student gets the pathway they want, if they want a pathway, that’s good. And we know that in a lot of places, there’s not enough… there’s more applicants for, say, CTE pathways than there are available spots. But it is interesting how some models go wall-to-wall and say, “Everybody must do this,” which in some ways students get to graduate with a credential of some sort, every student. Or more choice. There are pathways and you don’t have to join the pathways. So I don’t know if you have an opinion.

Mason Pashia: I have a lot of opinions and thoughts. I don’t know how many of them are informed versus a knee-jerk. I mean, I think in some ways those two districts represent, especially Gibson Ek versus these pathway schools, not to put them in opposition, but they represent very different approaches to this challenge of essentially matching students with authentic and personalized learning experiences.

Some of that is through just complete interest, self-selection into a pathway, and some of that is through this more, like, rotating cast of experiences that would be internships or project-based learning. I know which one I would’ve done better in, but I don’t think that’s necessarily true for all kids.

And so there’s, like, this really interesting moment where I don’t know what is the answer. If the answer is to be more specific and precise about what you’re about, my personal belief is if that’s the case, I think you owe it to your community to serve more than your K-12 students. Like, if you’ve developed a giant maritime center, that’s a huge opportunity for anybody who wants to get into maritime, not just the public school kids. I don’t know if they’re doing that, but I believe that is hopefully on the roadmap for a place like this. And we saw that, last thought on this maybe, we saw that in Chicago recently at the Academy for Global Citizenship, where they have a bunch of solar panels out back powering the school, and they host some community solar installing workshops, like, in their cafeteria for community members.

So they say, like, “Come use our facilities to learn how to be a solar installer and get licensed.” And so I do think there’s, like, this really cool way where you can start to use the facility as training, and if you hyper-specialize a pathway, I think that just opens up your opportunities even more to do that.

Nate McClennen: Right. It’s where schools become learning centers, community learning centers, which I do think is a future-forward vision. And I think we see that especially in rural places, you know, opportunities for night classes or other opportunities where you have multi-generational experiences. Cool. Well, good.

We love doing school visits here, and I’m headed out… I think we’re both going to Kansas City. We’re doing different things next week. But I’ll be at Liberty and taking a tour of Liberty, which we all like. And down in some of our work in Virginia, we had Julie do a presentation on Liberty for some of the folks in the VaLIN network, and it was really well-received.

They’ve just done a good job around alignment of saying, “This is who we are, this is what we do, and this is how we do it really well.” So, what else on your short list here in the beginning?

LearnKit: A Proposal for Learner-Owned Data and Learner Wallets

Mason Pashia: Yeah. Short list. So we recently published something on gettingsmart.com that I’m just excited to see live in the world. So this is a thing called LearnKit, which was put together for the Gates Foundation by a few close colleagues and friends of Getting Smart, Karen Cator, Valerie Greenhill.

And it’s basically a proposal for governance and usability of learner wallets in an AI world. So Gates Foundation and others are really interested in this idea of AI portability, and specifically with an eye toward security. So this is really like how do you take something with you as a learner and make sure that it is secure and interpreted well in your own kind of service, rather than being interpreted by something else for them about you.

So they just put together this pretty useful potential framework for how to build this, and I think the thing that stood out to me was they just had a really nice distillation of how things tend to work with regard to data. So I’m not gonna go through all this over audio. I’ll put the link in the show notes, and we put together some super fun future scenarios about what it would look like if a learner wallet was developed and implemented well.

But I think the real shift here is most data ecosystems exist at the level where a platform kind of owns the data, and nobody can manipulate it from there. So it puts a lot of onus on the platform to be secure and operating within the best data principles. What this proposal talks about is that ultimately, the learner would own and operate this data.

So while a platform might help them demonstrate the data, or show the data, or manipulate the data in terms of who can see it and who cannot, the learner ultimately gets to make that decision and choose where that lives. And so I just think as we’re continuing to think about these classic pressure points of, like, if I want a transcript, I still have to email the university I went to, and getting your transcript from high school is so hard, or showing what an A means is so hard, I think it’s really important that this becomes something that the learner and their families can own, and then eventually that transitions into just full learner ownership.

It’s an important report. I would check it out. It is a proposal for how to solve this problem. So definitely check it out, and we’d love to know if you have any thoughts on it.

Nate McClennen: Yeah, it’s a cool report, and I briefly looked at it when you shared it with me, and I love some interesting storytelling techniques as well for that. So I encourage everybody to take a look at that. And, you know, we’ve been preaching this idea of self-sovereignty for a long time, that learners own their own data, and I think that’s gonna become more and more important when these LERs and these wallets become more and more prevalent as a way to connect learner achievement and learner proficiency to jobs, to higher ed, whatever the case may be. So a lot to read there, which is great.

How the NAEP Test Works and Why We Test So Much

Nate McClennen: Let me see. I might just jump in here with a quick one. I’m gonna geek out for just one minute on my continuing to unpack how these national tests work. And so last time on Catching Up, we talked about PISA, the international test, and how statistically you’re really testing a small number of students, but it’s representative of the whole. So NAEP, the National Assessment of Educational Progress, is one that’s administered by the US. So as we know, most listeners know, PISA is global, so whatever, 90-plus countries. NAEP is federally administered, and then every state has their own standardized testing as well.

So just a quick review of NAEP, because people always talk about NAEP, and then you see the headlines, it’s like, “Hey, NAEP scores have gone down, gone up.” But schools are selected. Two sessions per school, 25 students per session, grades four, eight, and 12. So that’s the general context, grades four, eight, and 12. Generally testing reading and mathematics, and this is always the part, the statistical part, that fascinates me. 2024 was the last main assessment, and so 117,000 students in grade four, and about 114,000 in grade eight, and 24,000 in grade 12, out of the 3.5 million per grade level. So you have 100,000-plus students in grade four out of the 3.5 million, and that’s what’s being assessed, and the data’s being extrapolated from. And this goes on, it’s a very similar statistically relevant method, just like PISA. And I continue to just wonder how are we using these tests?

And they are effective for broad-scale change, but the testing is never disaggregated down to the student level. So these students take these assessments, but they actually don’t help that particular student grow. And I guess I wish, in my dream state, that assessment always started with the student and was aggregated up and never duplicative. So we shouldn’t have a reason to do a PISA assessment, a NAEP assessment, a state assessment, and then a district assessment. So that’s starting from the top and coming down. We always should start from the bottom and say, “How can students be assessed in their classrooms around a broader set of outcomes that are important?” And then how is that aggregated up into these measures that are important? So I just wanted to, again, put a placeholder in there that we haven’t solved the testing issue, and we test probably far too much, in my humble opinion.

Mason Pashia: Definitely agree with that. I think we’ll talk about this in a different way in the deep dive today, about a couple of different approaches to scale. So I’ll save some of my thoughts for that. But I appreciate these deep dives into tests. They’re things that are the water we swim in that I honestly know less about than I should in terms of how they actually are rolled out and measured. So I appreciate the information. Yeah, do you wanna share another one?

What Students Want From School: The Speak Tomorrow Data

Nate McClennen: No, you have the Speak Tomorrow data.

Mason Pashia: Yeah. I’m gonna probably end up having to add, like, an editor’s note to this one because I reached out for a comment on how this question was asked, because I think that’s really… I’m very curious. Basically Speak Tomorrow has some new data that just came out, I believe this week. Whenever this goes live, I think it’ll go out this week. And basically they did an assessment on student perspectives on edtech in general and technology, kinda with a bent towards AI. But one of the questions they asked was around the student wish list for modern classroom learning experiences, which is something you and I are always passionate about.

It’s the demand side for so much of the work that we’re doing. And I think the thing that was interesting to me is they broke this down into grades six through eight and grades nine through 12 students. In general, they kinda mapped to each other. It wasn’t a huge disparity between the age bands. But there was no specific experience that students wanted over 50%, which is really interesting. And so the top one was connecting what we’re learning to real-life stuff. For middle school age students, that’s about 41% of students want that. For grades nine through 12, 48% of students. So I think it really matters how this question’s asked.

If it’s like rank your top three or something, maybe. But I just found it, like, fascinating. And then the least selected one, at least within this particular chart, was to personalize learning to fit student preferences and strengths, which is really surprising. Only 30% of students grade six through eight wanted that, and 36% grade nine through 12 wanted that. So again, I’m gonna put a caveat that I either might strip this out of the conversation depending on what they give me, or I’ll do it like a little voiceover interjection of what I learn on the question. But I think it matters that students desire the things that we’re building, and I think it’s hard for students to imagine something without experiencing it.

And so those are probably the two tensions that surround this challenge. But I know this kinda raised an eyebrow for you as well. Like, do you just agree that the question formulation matters a lot, or does this echo anything that you’ve seen or experienced in your own life with students?

Nate McClennen: Yeah, I think question formulation will help inform this data. It is interesting. I’ve seen other surveys where students generally want things to be more personalized or more connected to real-world stuff, or help build skills. And the fact that there’s only 45% that would say that leads me to believe that the question was asked like a top-three question or something like that. So it’s actually a much higher data point. But I also think, from the demand side, as you and I continue to uncover and sort of peel back the layers of the onion, that just like leaders and teachers don’t know what they don’t know in terms of models, students are the same way, and probably more dramatically so.

So if you’ve been in a system, we talked about this, if you’re in the system and this is all you know, then it’s really hard to say what else could… How could it feel different, right? Other than, “Hey, can we chew gum in class?” Or, “Could we do a project instead of a test?” Or something like that. But seeing really outside the box. So the idea, for example, of Big Picture Learning schools, where a kid on Tuesdays and Thursdays or whatever, two days a week, is going for a three-hour internship, that’s really unusual for probably a lot of students if you shared that idea, right? Like, “Yeah, this is what we do. This is the water we swim in.” So I think this goes back to we need to build more demand, and we need to have students involved in building that demand and understanding the options that are out there.

Mason Pashia: I think in that student project that we ran in Issaquah School District, just to kind of extrapolate this further, so just for the listeners, part of that project was to take student survey data that was conducted by and for students, take it into the middle school, and have the middle schoolers design a high school that basically solves the problem that the high schoolers are describing. So that was the challenge. The interesting thing was when you asked the middle schoolers to dream about a new school model, they were like, passing period times. It was very rooted in their own daily life, and Gibson Ek is one of their high schools. Somebody in that district could be communicating, “Hey, this high school does internships three days a week, or two days a week.”

That matters. And it’s just interesting to me about, like, what trickles down and what doesn’t in a school community, in a district. Obviously, that’s a big task to get information to, like, sit and resonate with middle schoolers. That’s not always easy to do on any topic, let alone school related. But I don’t know. There’s just something there. And then just one other thing this data brought up as I was looking at it. It’s really interesting to me that in the different tiers that the students identified, connect what we’re learning to real stuff, help us build skills we’ll use in jobs someday, and create projects based on real-world problems are three different ideas within this, and to me, those are one idea.

Like, I wonder sometimes if we fracture skills from life stuff to real problems into these different buckets in a way that’s actually not helpful to learners, because it’s like, oh, I could be learning something relevant, or I could be working on something that’s like a problem in the world. And, like, there’s actually not a delta there to me. It’s like that’s sort of the same thing. So I don’t know. Again, I have questions about the question. But I wanna look into it, and just curious.

Nate McClennen: Yeah, I mean, there’s that fine, that sort of discernment question. If someone is doing something that’s relevant to them, like researching craters on the moon, that’s not necessarily a real-world problem to be solved. They could turn it into potentially a real-world problem, but I think that there is a distinction between some of these questions, and I again wonder what you’ll learn when you ask the survey folks, what you hear back.

AI Chatbots, Artificial Intimacy and Human Connection

Nate McClennen: So anyway, let’s jump in. I’ve gotta do a quick plug. We don’t have a whole lot of AI conversation going on today, so I’m just gonna do a quick one. Ezra Klein and Sherry Turkle had a podcast last week, and super interesting. Sherry Turkle is the author of a new book, Artificial Intimacy: Who We Become When We Talk to Machines, which is something that’s been in my mind as we move from this period of authentic human-to-human connection, then communicating via technology, and then communicating with technology, right?

By “with,” I mean the technology is the person that we’re talking to. So this moving from human-to-human, using technology to communicate with one another, to humans just communicating with technology, and eventually maybe my technology communicating with your technology is fascinating. But there were two things that came out in the conversation. Listeners, definitely take a look at this one if you’re an Ezra Klein fan. But two facts. One is that Google’s own researchers have found that some children are now struggling to distinguish between talking with chatbots and interacting with humans, right? So that’s fascinating. If children can’t discern the difference, yet we know there’s a difference, whether it’s how the chatbot is responding and et cetera, et cetera, what does that mean for those children as they become adults?

We don’t have to answer this question. Posing it out there to the universe. And second is, one in five US adults are using a chatbot for a romantic partner in some form or another. So this continued very fast emergence of a human-like partner is really, really interesting, and I think it’s gonna have profound effects on our society, as Sherry Turkle was writing about.

Mason Pashia: I think this is interesting. Like, in some ways this is a re-imagining of a familiar problem, which is, like, generational communication differences and expectations. Like, in some ways, bear with me as I make this loose connection. So I think every generation has a slightly different way of communicating.

You and I run into it a lot ’cause I write hey and y’all in my emails and you do not. And so, like, there’s a version of that. But also, like, if I get a text or a message from someone, I’m expecting a level of, like, communication exclamation point something. I think that’s, like, a very millennial trait to expect some sort of, like, excitement or, like, emotion in a communication. And depending on the workplace I’ve been in and who I’m communicating with, that is not how they communicate. And so there’s almost, like, this bracing or readying yourself for this more, like, blunt real world that I think is gonna be really true with how sycophantic chatbots are, right?

Like, if I’m talking to Claude and then I go talk to a human, I’m like, “Whoa, you are, like, not as nice.” And that is good. I would rather be among people who are more real. But I think that that might be, like, the first place where we really see it, is it’s not that somebody is like, “I actually don’t wanna… I don’t talk to humans.” It’s like, “Man, when I talk to humans, it’s just hard, and, like, I’d rather not.” And, like, that I think is where we’re actually gonna start to see the gaps. And I think in that Ezra Klein conversation, she made a funny comment about, like, how people are practicing to ask people out or something like that. It was a part of the conversation. Which is, like, so much of the charm in being asked out is the nervousness. It’s, like, the way that the person just kind of fumbled the ball and you’re like, “Oh, this really matters to you.” Like, if I asked someone out like it was a TED Talk, that would be, like, pretty gross.

So I do think there’s something in this that’s like, don’t have to be perfect, but also you can’t expect people to be, like… I mean, again, sycophantic as a chatbot is. So I really don’t know what to do with that. But I’m wondering, like, I think we experience this in small ways already just based on generational differences or talking to a close friend versus talking to a stranger, and I think that’s a good mirror of what this might feel like to young people in the future.

Nate McClennen: Yeah, yeah. No, it’s interesting. And it reminds me of the show, The Rehearsal, that you and I talked about, which we won’t go into details about. But I guess I wonder, we’ve talked before about individualistic aims and the commons, aims for the greater good: if the commons depends on collaboration between humans, actual humans working together to do something, are young people, or anyone who’s exposed to this AI in ways that they’re overly dependent on sycophantic partners, does that work become harder, right? Does altruistic work that helps humanity become harder because everybody kind of gets what they want from these technology partners?

Mason Pashia: It’ll be an interesting world to see if this continues, if this rises, like how some of the early collaborative experiences change, right? Like sports at school, where students have to have, like, rapid, pretty blunt conversation with each other about, like, “You messed that up.” Like, “Hey, this didn’t go well because of X, Y, and Z.”

Like, I think people are in those collaborative environments, and they will be just because that’s, like, the way that kids are and the way that our systems are set up. And I think those will be really early indicators of, like, we’re actually losing the ability to collaborate as a species, and that could be because of this thing. So I don’t know, just super interesting indicators to keep an eye on.

Nate McClennen: Yeah, we’ll pay attention and see. It is a brave new world for sure. All right, you’ve got one more AI one here.

Mason Pashia: I’ll save it for next time.

Nate McClennen: Okay.

Sold a Story, Direct Instruction and Project Follow Through

Mason Pashia: Yep. So let’s jump into our deep dives.

Nate McClennen: Into the deep dive. Let me give you a little context. So as you and I talked about, Sold a Story is a new podcast. The first one, season one, came out, mm, two years ago, and it was about the teaching of reading at the elementary school level. And it was sort of an extraction of a deep-seated belief around whole language instruction versus what’s being called the science of reading, which is really around phonemic awareness and reading comprehension combined.

I won’t go into the details of the actual argument, but I think season one had some effect of now 43 states having science-of-reading laws in place, which means for all the elementary schools that are teaching reading, you must have an established program that is based in the science of reading, which has strong evidence around phonemic awareness and reading comprehension. So my opinion on that is that we should teach kids to read, and if this is the way to teach kids to read, then that’s fine. Like, I actually don’t have any issues with that. So that was interesting, and there’s a big exposé, tons and tons of listens. The second season is now coming out.

Before I go into the second season, the science of reading, a lot of us are talking about, or you’re hearing a lot about this Mississippi miracle. And it was a great podcast where they were interviewing one of the women who helped implement it, and she actually said it’s really the Mississippi marathon, because they didn’t just implement a science-of-reading curriculum. What they did was long-term. It was clear, top-down professional development, which is actually very different. We have very decentralized schools. So in Mississippi, they said, “You are all gonna do, if you’re teaching third grade, K through 3 reading, you’re all going to have the same instruction around the science of reading. That’s what you’re doing.” So it’s top-down, and the fact that many ed schools aren’t teaching this skill, so they had to upskill the teachers to do it. They made a very specific third grade holdback. So if you are not mastering particular skills, you will be held back a grade. And they started in kindergarten giving parents letters saying that your student is behind, your child is behind the mark. And then finally, the broader context was that you can’t teach reading in the long run without the context, the content understanding about the richness of content along the way. So the point is that this wasn’t just implement science of reading. This was a series of things. The reaction has been across states, implement the science of reading and a particular program or curriculum. But Mississippi did a lot of things to make this happen, and it wasn’t just one thing. Okay, so that’s Sold a Story Season 1 and the ensuing science of reading in the 43 states. Okay, so the next season, as you and I know, focused on this guy, Zig Engelmann, who was the founder of direct instruction.

And we’ve all heard about direct instruction. Mostly, direct instruction imagines that we are in a classroom and someone’s lecturing to us. That’s the way, at least I think of direct instruction. I did some deep dive on direct instruction as in capital D, capital I Direct Instruction because I was interested. I was like, “Really? What is this?” And as a pedagogical tool. And it is actually a very specific program and defined in short as scripted lessons. The amount of review is greater than new material in any given classroom. Active student response, and so active student response is the call and response.

Like, “2 plus 2 is,” the kids say, “4.” “3 plus 3 is,” the kids say, “6,” right? Skills-based grouping, subjects leading to big ideas. So what are the big ideas, which I think is around priority standards or priority competencies. Immediate correction, so don’t let errors continue, and then mastery-based learning.

And so interesting about this is that the general understanding of direct instruction is that it’s someone lecturing students. In actual direct instruction, there’s some things about personalized learning, mastery-based, and skills-based that are actually part of innovation agendas of a lot of schools. What I wonder about, because this podcast is very clearly contrasting direct instruction with inquiry and discovery-based teaching, that’s the dichotomy they’re setting up, just like in the first season they set up whole language instruction versus phonemic awareness and reading comprehension, the science of reading. And I wonder what is going to happen when this podcast reaches school boards, superintendents, and everyone suddenly starts talking about, “Hey, we’re gonna do away with inquiry or learning by discovery because we’ve gotta do direct instruction.” Because the research that they unpacked on this is that direct instruction, according to the measures that some people articulated in a pretty large study, which was in the late ’60s and ’70s, actually had strong efficacy. Now, there was strong pushback against that as well, saying, “Hey, the statistics are wrong,” et cetera. So I’m not taking a side on this. What I am asking you from a storytelling perspective is saying, when suddenly there’s a podcast that’s very popular and suddenly talks about direct instruction, can it move school boards, superintendents to suddenly say, “Hey, we gotta get rid of all these things around inquiry, and learner agency, and student choice, and all this, because this is the only thing that works”? And so my nervousness is around, again, the pendulum swing that seems to either clock on one side or clock on the other side and never talks about the center.

Mason Pashia: Yeah, I mean, I think that’s a really… It’s a good question. I think any media can fan a flame, right? Like, I was reading the comments on the Spotify podcast episodes the other day, and it was all people that were very, very pro-DI just being like, “Oh my gosh, this is like… Can you believe that person said this thing about inquiry-based learning? That’s crazy.” And so, like, there’s a little bit of the internet that’s just gonna inherently polarize people and have them find their camps. I think the first season really did demonstrate an ability to change minds. I think that there’s so much unrest around education right now in terms of what works and who’s in charge of saying what works, that I can’t tell if that makes this a vulnerable time or kind of like a stonewalled time.

Like, if you actually take this somewhere, I could see school boards being receptive because their test scores are going down, and I could see a lot of, like, the microschool community be like, “Hard pass. That’s why we decided to break off in the first place.” Like, I could actually see a further fracturing of some of these places. I think the first season had, if I remember correctly, a year-in-review episode after it came out, which was a lot of interviews with educators about essentially their feelings after listening to the first season, and it was really cathartic to them to hear somebody kinda put this other form of reading on blast. It felt like a reason or a rationale for why their teaching wasn’t working, and I think that made a lot of them feel good. And so I also wonder about that. Like, if enough teachers in a district co-op and they’re like, “This explains my feelings,” to a school board, like, I think you get stories told twice, right?

You get, like, this compounding effect of storytelling, which starts to be mobilization and starts to be a movement. So that would be probably where I think this really takes on power, and I think those are the people who would probably bring it to their school board. Like, educators love this podcast because it explores what they’ve been doing and why it has or hasn’t been working, and I think a lot of people have been looking for that answer for a long time, when the answer is likely much more complicated than this, right? Like, we’ll probably talk about the study a little bit, but there was a study on these different teaching methods, and basically, the study found that a lot of them are viable. And technically, DI was the most consistently viable version, and so they were not able to, like, back one up necessarily.

They were sort of just like, “Oh, there’s pockets where things work, and there’s pockets where things don’t work, and DI is the most consistently working one, but we can’t really say that as the federal government.” So there’s, like, this interesting inability to get behind the thing that, quote-unquote, “works.” But to further complicate this, I think it’s really easy to miss, in this case, like the trees for the forest, actually, where it’s like the components of DI that are working, I think matter, because some of them actually could be represented in other learning formats. Like I think this might be what you’re getting at with the mastery and the skills-based grouping.

But like by doing a call and response type of approach, there is a radical amount of transparency for the student as to where they are compared to their peers. Like you get to see pretty fast if something’s not making sense to you, and it’s making sense to, like, Joey sitting next to you, which I think is actually, at its best, what mastery-based learning does. Like it is a real-time calibration of what do you know and where do you need to grow? And that is probably a piece of this that’s implicit within the DI model, but may or may not be actually one of the things that they studied when they think about it. They were maybe focusing more on the content, the scripted lessons.

They were maybe focusing more on the pace, and really it might just be about transparency, and communal learning is powerful. And, like, it’d be almost impossible to extrapolate that from the data though. So I have a general hesitancy to throw the baby out with the bathwater on some of this stuff. But I don’t know. So sorry.

Nate McClennen: That’s super interesting. And I think I have a couple responses. So one, just for our listeners, if you don’t have time to listen to the podcast, they are unpacking this large initiative that was federally funded called Project Follow Through. It started in 1968, went through most of the ’70s, so a 10-year sort of period, and they were basically trying to answer the question, what pedagogical approach works best? That was the goal. And they were measuring things on problem-solving, self-esteem, as well as all the core academic skills.

So it wasn’t just academic skills. It wasn’t like Hattie, for example. When you look at Hattie’s work, he’s looking at sort of efficacy around academic skills. This was probably broader than that in terms of the study. And like you said, the initial analysis showed DI was ahead on problem-solving, self-esteem, academic skills, by a lot. I think the things that I’ve learned through the first three episodes that have been released are, a couple quotes first. One is that their approach with DI, this particular guy, Zig Engelmann, was that if a student doesn’t get it, it’s a teacher issue. And that to me makes sense, right? Like, I think we oftentimes point fingers saying, “This student can’t learn, this student can’t learn, this student can’t learn.” And I think often when I was a school leader, I’d always say, “Reflect back. What can you do differently as an educator to reach that particular student?” And that goes back to the other conclusion, at least after the first three episodes, that when you look across the United States at many different programs, you are always going to find a program that works somewhere with some cohort of students. And so the quote was, “Every program works somewhere.” And I think my addition is there’s probably not one program that works everywhere. And I think sometimes we’re looking, probably all the times we’re looking, for a silver bullet, and when something comes out like this, we suddenly say, “All right, DI is a silver bullet. Get rid of all the inquiry work. No student agency. Make sure that doesn’t happen anymore.” That’s not what DI is necessarily saying, but that’s what happens as a result, because people jump on bandwagons all the time in education. I guess I would love to see more of us thinking about, our friend George Philhower says, “I wanna build schools that are exactly right for every kid.” So trying to create schools that are right for that kid who’s in that school, and that may be a different type of school for every kid. It might be the same type of school for a group of kids, but it’s not a one-size-fits-all model. And I think this will probably emerge as a, “Hey, everyone who’s been doing inquiry learning and you didn’t like it, you’re gonna jump on a bandwagon and say direct instruction works,” and then you’re gonna take the parts that you want of direct instruction and only use those parts, because fidelity of implementation’s tough.

And that may be the same… that’s the same for inquiry, it’s the same for project-based learning, it’s the same for skills-based grouping, mastery learning, competency-based. All these different types of approaches are all valid in some place, and it’s our job as educators and school leaders to say, “What’s the learning model that’s the best for our students?” And it’s probably not the same. It is definitely not the same for all students.

Pedagogy, Belonging and Why Call and Response Works

Mason Pashia: I mean, I think this connects exactly back to the school visits at the beginning of this conversation, right? I would say Tacoma’s approach to this problem is make sure that every kid has a school that they can see themselves in. Like, building a variety of school experiences.

I would say the Issaquah School District example right now, partially due to serving a smaller student population, but, like, Gibson Ek is basically a hub model where the students can branch out from there and find out the type of learning and the thing that they wanna learn that serves them best. And so I think in some ways, like, one of them is school-out, and one of them is district-out, in terms of exploration.

One other thing I wanna just say about this DI conversation, actually two things. Connecting back to the NAEP conversation, you were talking about the analysis showing DI ahead on problem-solving, self-esteem, and academic skills. To me, what I took away from that podcast is that self-esteem and problem-solving were actually very surprising to Zig. Like, he didn’t know. So he was like, “Yeah, academic skills, huge improvements.” But until this study, they actually weren’t sure that those things were happening, and they just weren’t measuring them.

So I think that connecting back to NAEP, it’s like, what you measure matters, and I think, like, if we don’t try to account for some of this other stuff, we really can’t holistically gauge what works. So I think self-esteem on this is really interesting. The other piece I would say is, like, it’s possible this works well for K-5 and then stops being as relevant or something like that, and then it’s really hard. How do you help those learners transition from a DI approach to, like, inquiry-based? If that is actually something that we’re positing, like, as you get older, you need more agency, you wanna do more self-directed learning, that’s a hard jump to make. And so it is both how do you build the academic skills, but also how do you build the skills of learning? And I think that that is probably implied by problem-solving but is not maybe directly assessed.

And maybe my last comment on this is just back to the thing I was saying about transparency. I think call and response is just inherently like a human technology that works. If you think of concerts, if you think of church, if you think of all these places, like, you have somebody saying something, people responding, and that both signifies, like, belonging because you’re a part of a congregation or an audience. That signifies, like, you are actively engaged because you’re responding to something that’s happening in real time, and it signifies sort of over time this, like, general buy-in in the direction of the thing. Which is like, if the general buy-in in the direction of a church is belief, the general buy-in in the direction of a school is learning, right?

And so there is, like, this… I just think that that is a beautiful technology in itself, and I’m just really curious how you could isolate component parts to be like, “This is the thing that makes people feel seen. Let’s pull that into our context and make it work in a more self-directed way,” or something like that. And that could be, like, the way that Jeffco Open School does their, like, capstone celebrations where they get all the peers in a room. The student talks about their progression through the thing, and everybody, like, vouches for them and asks them questions. Like, that’s a really beautiful call and response-like methodology of an inquiry-based learning model. But I don’t know. I think it’s fascinating. I’ll be really curious to see where this season goes. I think you and I should do another check-in at the end to see kinda what their take is, ’cause I think it does matter ultimately where they land on this thing. It’s got a lot of ears, and they’re hungry ears. They’re not just passive ears.

Nate McClennen: Yeah. And I was thinking about that call and response piece, that I have seen classrooms that are call and response that feel very compliance-based, simply doing something. Okay, so you hear the judgment in my voice ’cause I used the word simply. On the other hand, a student knowing two plus two is four, memorizing single-digit addition, memorizing multiplication facts is something that is really, really important.

And I’ve seen a number of progressive schools say, “Hey, you don’t have to memorize those.” And having taught math for many, many years at the high school level, I’m like, “Yeah, you actually do need to memorize those. Those are really, really important.” And so call and response may be a solution there, and it could be, you know, another modern version is quick response software where you’re typing a response in your computer, and it’s showing up on the board and maybe not per the individual student, but they can see if they’re right or wrong right away, which is what you’re alluding to, is there’s many versions of this. And so I think there’s the pedagogical piece that when we think deeply about this is let’s find things that help students learn the core content and develop the ability to think critically, problem solve, be agentic, et cetera. And maybe there is a different balance in elementary school, middle school, and high school. And I have seen project-based learning high schools where the students don’t have the foundational knowledge they need, and then they go and try to do projects, and I’ve talked to you before about this, but they become Swiss cheese projects, right? Like they’re really excited, they’re engaged in it, they really love it, but the learning is not so great, right? So I think we, again, let’s find the pendulum in the middle where it takes the best of all these different pieces and build models that are best for the particular cohort of students that we’re using. My fear is, when a very popular media channel puts out something like this, which was very well done and important to learn, is people take it the way they wanna see it.

They take it as, “Oh, this is in support that I never liked inquiry learning anyway, and now I have proof that we’re never gonna do inquiry learning again,” which would be a bad outcome for this.

Well, okay. That’s a long deep dive. I think we’ll stick on that for our deep dive today and finish out. I wanna do a quick school shout-out and then jump into human expression, if that’s okay.

Mason Pashia: Can I say one last thing on the DI piece? Just final shout on the DI conversation. Sold a Story did interview some students about their experience with direct instruction. They put a link in the general podcast feed. So always an advocate for hearing from learners, but these are learners that had DI in the past and now are adults and reflecting on that journey. They didn’t get a lot of voices, but the voices they got were favorable and kind of supportive of the claims. So I don’t know. I think it’s always just worth hearing from the actual students rather than just the data. So just putting that out as additional reading for your pleasure.

Nate McClennen: Yeah, and that syncs with a podcast you did with Andrew Frishman, right, from BPL, about his Met experience. Interviewing adult students, adults who were students, you know, 20 years ago or whatever the case may be, is really, really fascinating because you find out what’s really sticky. So definitely encourage people to listen to that as well. So I wish, as we think about student voice, you know, you and I are really bullish on we need way more student voice that’s authentic, and how do we do that? And we’ve got all sorts of project ideas out there with partners, and I wonder if there’s a missing piece, which is adult perspectives on their student experience. That would be an interesting addition to this conversation.

Mason Pashia: Agree. ‘Cause you really don’t… I mean, I don’t know when learning was valuable until it emerges again, right? Like, there’s a lot of times today where I’m like, “Oh my gosh, that project I did when I was in high school,” like I can see that in this moment, but I didn’t see it for 20 years. So I think that, like, there’s a lot of ways learning, if it’s good, it’s just sticky. It’s not even inherently visibly formative until it is, and then you’re like, “Dang. Crazy.” Super interesting.

Nate McClennen: The question I ask of all our podcast guests is what was your most memorable learning experience, strongest outcomes, felt like you belonged, strongest engagement? And everybody has one. And very rarely are they direct instruction, I just wanna say. As the lowercase D, the lowercase I, meaning someone lecturing at you. So that may be a perception issue. I mean, who knows? It’s an interesting question. So okay, we’re gonna pay attention to it. Stay tuned for our next episode or probably the next one after that where Mason and I will listen to a few more of these and see what we come up with.

School Shout-Out: Harwood Union and Competency-Based Transcripts

Nate McClennen: All right, quick school shout-out as we finish up here. So I’m in Vermont right now for the fall, which has been great, and the leaves are finally almost all changed here. But just southeast of Burlington, in South Duxbury, Vermont, Harwood Union High School was featured at Full Scale. And so they were one of the schools that was out there talking about competency-based learning and how they do things. And so I put it in our notes just as a way for you and I to visualize this, but I’m gonna describe it for people.

So it’s just an official transcript of Harwood Union High School, it’s a test transcript, and it has on the top courses. It has no letter grades, but it has zero through, it appears to be four, scores, which means they’re grading across a rubric rather than grading across averages of tests and exams. And then flexible pathways is the next section, so they obviously have some dual enrollment and/or some personalized experience. There’s a third section, which is on exhibitions of learning, which doesn’t have scores in it, but that’s where you actually do your exhibitions of learning in order to graduate, and then that would be documented there. And then there’s the learning expectation summary scores, which looks like a set of competencies.

So this is the scores across multiple experiences, across multiple classes, how they’re doing on communication and creative expression, integrative thinking, literacy, personal and global engagement, problem-solving, and self-direction. I put this up here because this is an example when people ask all the time, “Who is doing this in the world? Like, who is truly doing competency-based all the way through to the signals, the signals they’re sending out to the world?” And this transcript’s a great example of that. You can find it on their website ’cause they broadcast it pretty clearly as like, “This is what we’re doing. This is what our transcript looks like.”

And many of their students go to post-secondary experiences in higher education. So worth looking at. We’ll put the link on our website, but I just wanna call out Harwood for being brave and saying, “Hey, this is how we can better help students demonstrate proficiency in a variety of ways, both on core content and broader competencies.”

Mason Pashia: Love it. Love some evidence in the world. Thanks for sharing.

Human Expression: You Can See Everything and What Matters Most

Nate McClennen: All right, let’s talk about humans, our favorite species to talk about.

Mason Pashia: Thank you for saying “to talk about.” Because we did invoke starfish earlier in this conversation, I would hate to do a disservice to the starfish. Mine we already talked about briefly, so I’ll keep it short. But last week I went and saw the viral film, You Can See Everything, which is the Elizabeth Holmes documentary.

I think the reason this makes it into human expression is in some ways a little bit meta. Like, one of them is it’s kind of a miracle of a movie. Human expression in that sense is very, I think, thoughtfully and well done. But second of all, I think it’s really a testament to the complexities of being human in the world.

I think, like, the most generous read of the movie, I think, asks questions like what do we owe each other? And, like, how can you even… How can you trust, period? Like, there’s just something in it that is so fundamentally human because of the ways it is so nuanced, and you just leave feeling so confused in a way that only a person can make you feel. Like a person that you know well. Like the more you know a person, the more you’re just kinda like, “Oh, there’s really good things and there’s some things that drive me nuts.” Like, that’s crazy. So I don’t know. I don’t think everybody has to rush out and see it in theaters, but it is a really fascinating look at a fascinating person by a fascinating person.

Nate McClennen: And from what you said, I haven’t watched it yet, but just a sort of a study of humans, right?

Mason Pashia: Yeah.

Nate McClennen: A particular human, and what happens when you are in that space of real strong intimacy of being close to someone and being in someone’s space for so long, and what do you learn, and what can you learn, and how confusing it can be. So, all right. Well, it’s on my list. I’m gonna watch it, probably not in the theaters, but I will watch it.

Mason Pashia: Noted.

Nate McClennen: My human expression is we were on a long drive the other day listening to a podcast called Diary of a CEO. Fun podcast. There’s a lot going on. He does interviews with all sorts of interesting people. But he was interviewing a hospice physician and neurobiologist, this guy Dr. Christopher Kerr. And this particular scientist, he did a lot of interviews with people in their last one to two weeks of life and recorded a lot of these interviews and tried to understand patterns, and he specifically was looking at dreams they had.

So, what are their sleep patterns and what kind of dreams and what are they dreaming about? And it was worth listening to if you’re interested in that part. But I think the human expression part for me is that the pattern he saw is that at death, it’s about relationships with others, belonging, connection, and helping others. So it’s very, very commons focused at the end of people’s lives rather than self-focused. Very few people talked about their accomplishments. Very few people talked about themselves, and everybody was talking about the commons, the bigger piece, and that’s what mattered most to them in the last one to two weeks of life. Even though we push accomplishment all over the place and, “What are you gonna do to change the world?” But in the end, what matters the most is relationships between humans and belonging, and I feel connected to someone else.

The AI Song and Closing

Mason Pashia: Beautiful note to end on. Thank you, Nate.

Nate McClennen: Speaking of notes, we are gonna go one more note and play a song. Do you have it queued up for us?

Mason Pashia: Yeah. I sure do. All right. Let me share with audio. Can you hear it?

Nate McClennen: It’s actually really nice to finish with a song. Always chalk. It’s so convergent.

Mason Pashia: Gotcha. Nice.

Nate McClennen: That good? There we go.

Mason Pashia: That is some wild AI phrasing. There’s a lot of crazy… I like the new rules, new something. New tools. New rules, new tools.

Nate McClennen: New rules, new tools. Yeah. It is so funny that it always converges. Like, we talked a lot about the gray goo of AI, and it’s like our songs are becoming more convergent. It’s always talking about tablets or laptops in contrary to chalkboards and chalk. So, who is my inspiration?

Mason Pashia: I’m gonna say it’s kinda stomp-clappy and has a little banjo, so I’m thinking like a Lumineers or something.

Nate McClennen: Yeah, it’s like a… And I’ve done this one before, but I felt very Mumford & Sons in the… here, so I was thinking that would be a good finish for us.

Mason Pashia: There you go. Yes, it definitely has it.

Nate McClennen: Moving forward, the question is how do we make songs that have… I might guide it to say you can’t use chalk, chalkboard. I’m gonna give it some limiters so they can’t use the different words.

Mason Pashia: I like it. Yeah, I think we might have to keep education out. I think education might be the word that is triggering a lot of things, which actually goes back to our Hollywood, our problem with Hollywood, right?

Nate McClennen: Absolutely, ’cause Hollywood looks at education a certain way. Mason, awesome to catch up today.

Mason Pashia: Great to see you, Nate.

Nate McClennen: Good to chat. We uncovered a lot, unpacked a lot, and hopefully got people thinking. So send us questions, listeners, if you have them.

Mason Pashia: Absolutely. Good to see you.

Links

LearnKit
Harwood Union High School profile and transcript
NAEP reading
NAEP mathematics
NAEP long-term trend
NPR on the 2025 NAEP long-term trend results
Artificial Intimacy by Sherry Turkle
Weak Bundle, Strong Bundle: How AI Redraws Job Boundaries (CEPR discussion paper)
The emerging fight against grade inflation in K-12 classrooms (The Hechinger Report)