LISTEN — Class Disrupted S4 E7: Beneath the AI Hype

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Class Disrupted is a bi-weekly education podcast featuring author Michael Horn and Summit Public Schools’ Diane Tavenner in conversation with educators, school leaders, students and other members of school communities as they investigate the challenges facing the education system amid this pandemic — and where we should go from here. Find every episode by bookmarking our Class Disrupted page or subscribing on Apple Podcasts, Google Play or Stitcher (new episodes every other Tuesday).

In this episode of Class Disrupted, Diane and Michael break down the latest frenzy around artificial intelligence and education. They explore if and how ChatGPT could be an innovation in education, what would make it innovative (hint: it’s not the technology!), and how it could be a useful tool in creating better learning experiences.

Listen to the episode below. A full transcript follows.

Diane Tavenner: Hey, Michael.

Michael Horn: Hey, Diane.

Tavenner: Happy new year, Michael. It’s good to be in 2023. Our families, we’ve learned over the last two years, share the annual tradition of reflecting and some version of goal-setting for the new year. And as part of that process, we always read our previous thoughts and comments on past years. And I was fascinated to revisit just how challenging 2020 was, thinking about the start of the pandemic. And that was followed by the even more difficult 2021 and 2022 was an improvement for sure, but I think there’s still room to grow in ’23, Michael.

Horn: Well, I will say 2022 has certainly gone out like a lion for us, Diane. But despite that, looking back over the year, it showed us just how much we all grew and how much fun we’ve had with a variety of experiences opening up literally across the world, which I think is right. It leaves us where you are, which is optimistic for 2023.

Tavenner: Indeed. And I’m sorry that your year end has been dotted with a round of illness and dashed vacation plans. Sadly, I think that’s part of the quote, “new normal,” of whatever that means. And that the arc of our podcast has really followed that new normal, created by the pandemic arc. And because of who we are, Michael, that means we are really hopeful for the second half of our fourth season because things can only get better. And this year we are finally getting to return to innovation after, quite frankly, a tough number of years where we were hoping the pandemic would be cause for new ways of doing things, but it just wasn’t in many cases. And I think we are so much happier in the space of thinking about, talking about, and most importantly, doing things that modernize our education system and help it meet the needs of children, communities and our country. And so Michael, I’m pumping this up a bit because today I really want to go after it.

Horn: OK, so that sets up my alarm bells. I want to know, “What does that mean? What do you want to go after?”

Tavenner: Well, I really want to talk AI today, which might be a little bit baffling to you.

Horn: AI. Well, I mean AI I guess is, it’s everywhere I turn right now. Diane, in the newsletters I read, in the email threads with other education researchers and writers, in the round-ups of the most important developments of 2022, in text messages with friends, it’s everywhere. So I’m curious, what’s the angle on your mind?

Tavenner: As you know Michael, I had college kids home for the holidays, and so not only were we talking AI, but we were playing with the ChatGPT. And as you know, there’s a ton of speculation about the potential implications and we haven’t had a chance to unpack it all yet. I’m super curious about your take.

Horn: Well, I’m happy to go there, Diane, but let’s start with a basic explanation of what we’re actually talking about, because I suspect not everyone is following the latest in AI developments. And here’s the roundup. In December, a company called OpenAI released this program called ChatGPT, which you just referenced. Now ChatGPT, it’s free, at least for now. OpenAI basically wants to get user feedback and learn how people interact with it. But essentially it’s a product that interacts with people in a conversational way. So you can ask it questions and it will give you answers in whatever form you’ve asked it to. Want a song? It’s got you. Want a conversation, you got it. Want a college admissions essay? Well, it’ll do that as well. And what makes it unique is that it can answer follow-up questions, admit it’s made a mistake, challenge incorrect premises, and even reject inappropriate requests. And I will tell you, it’s a bit addictive to use, at least for me.

Tavenner: I experienced all of those things when using it, Michael, which, of course made me curious as to how it does what it does. And what I learned — and let me be clear, these are my words, not the official words — is that they essentially trained this program to play a game where it guesses the next word, and they trained it by having it read — well, that’s my word — read basically everything on the internet. And when you study everything written on the internet, you can, for example, guess the capital of France will be Paris, because it just appears that way over and over and over again. And once you figure that out and store that information, well, I guess in a vague sense that the capital of France is Paris, which feels a bit like intelligence when you’re engaging with it.

Horn: Yeah, I think that’s right. I mean, it’s essentially in the words of Stratechery’s Ben Thompson, a probabilistic model. It doesn’t actually quote, unquote “know,” if you will, the answers, but it’s able to make a quote, unquote “guess” based on statistical likelihood in essence. So it’s in the words that we love to talk about it’s correlative, it’s not causal in other words. It’s also largely historical looking at this point and hasn’t indexed more current information. And although it’s pretty freaking amazing, it could also make some big mistakes as a result. So for example, Ben Thompson asked it about Hobbes’s view on separation of powers, and it answered him back that Hobbes believed in separation of powers, which for all the teachers out there is completely wrong, as you probably know. So why did it possibly quote unquote “think” that? Well, presumably because Hobbs mentioned in lots of articles on the internet alongside John Locke who did believe in separation of powers. And it probably couldn’t sort that out because it’s not actually quote unquote, “thinking” as we would define it.

Tavenner: Well, I’ll be honest, that feels a little bit more human to me, Michael, because I must admit I’ve confused Hobbs and Locke before, but setting that aside, I do not get confused about where I earned university degrees, which is what happened to me when I asked it to write a bio for Diane Tavenner. And I’ll admit it turned out a pretty well written and pretty accurate account with one problem. I do not have a degree from UC Berkeley, but my family has something to tease me about now. All the while, all of this is kind of fun and interesting. The real reason I wanted to talk to you about it though, Michael, is because of all the hand-wringing and worry about what is this going to do to education, what’s the impact it’s going to have?

Horn: Yeah. And Diane, we’ve seen all manner of reactions on that infamous internet that’s at the basis of the AI to begin with. You’ve seen folks like Checker Finn argue on the continued importance of the process of writing as a way for humans to think through challenging problems. On the positive side of the ledger, you’ve seen things like Zeynep Tufekci, and I hope I’m pronouncing that correctly, of The New York Times arguing that ChatGPT will essentially be the equivalent of the calculator, but for writing. So it’s an aid to help you write things just like the calculator helps you do math. It’s a tool, in other words. Others have said that this basically creates the necessity for the flipped classroom because you better do the work in the classroom or else who knows if you’re really doing the work. 

Some like Michael Brickman at AEI have hailed, “this is the great leveler in education,” whereas others have said it will increase inequality. And then you’ve got the Robert Pondiscios of the world who’ve argued that this increases the importance of our humanity and understanding what knowledge is true and knowing how to apply it. And then you’ve got Peter Green saying, “Well, look, this isn’t going to kill the English essay. It’ll actually be a great way for teachers to test their prompts to make sure that they are asking better questions in the first place.”

There’s a lot more, let’s say, but there was also the apocalyptic, if you will, various reactions. And one of them, I think he falls under this category, was Daniel Herman in The Atlantic that got a lot of attention because he came out really early and really blazing when he argued that high school English is over now. And I just want to quote the last paragraph, Diane, because I’m sure he would object, “That’s the headline writers that wrote that.” But he wrote, “Everything is made up. It’s true. The essay as a literary form made up, grammatical rules as markers of intelligence writing itself as a technology made up. Starting now, open AI is forcing us to ask foundational questions about whether any of those things are worth keeping around.”

Tavenner: That ending is interesting to me, Michael, and it is this article you’re referencing from The Atlantic that got me pretty fired up. And one of the things that got me fired up was at the start of the article, he makes, I guess, what’s a more modest claim, but a claim that teachers will no longer be able to give take-home exams or homework because ChatGPT will be able to do the work for students. And that’s not even the thing that got me. I was thinking about this as a former English teacher and a school designer. I’ve got a lot of thoughts about that one. But before we go there, earlier in our podcast we talked about the different types of innovation, sustaining and disruptive, and I’m just super curious if this is even an innovation, I guess, but if it is, I thought it’d be fun to think through what kind as the initial discussion always seems to be kind of super hyperbolic with people making all these claims, as you said.

Horn: Yeah, it’s a good question. And let’s try to unpack that and we’ll go back to, gosh, it was several episodes at this point where we talked about disruptive versus sustaining innovation, but essentially disruptive innovation as a theory is basically a theory of competition. There are other frameworks in the world for innovation, but this basically helps you understand the dynamics of competing forces to predict the likelihood of what’s going to win out over the long run. And the theory posits that they are basically two kinds of innovation. On the one hand, you have what’s called sustaining innovation, and this is innovation that makes an existing product or service better, as better is defined. So, sharper TVs, faster internet, faster cars, more features on the cars, those sorts of things are sustaining innovations. They take the existing product or service, make it better.

Disruptive innovation, on the other hand, is the one everyone always thinks is sort of the breakthrough leap that changes everything, but it’s not. It’s actually something that is simpler and more primitive compared to those original sustaining innovations, if you will. But it brings to market something that’s more affordable, more convenient, more accessible, such that many more people can access it.

Now I’ll get into the debate of which one this is maybe a little bit later in the episode, Diane, because there’s a few other components to this, to figuring out, is something disruptive or sustaining. But to actually help us answer that question, I think it’s worth knowing something first, which is after reading people’s takes and having your own reactions to them, Diane, as that leader of schools, as that designer of school models, what do you think it might do in schools?

Tavenner: This is great. First of all, I just have to keep reminding myself that sustaining and disruptive innovations are different types. One is not better than the other. And do think that…

Horn: Yeah, that’s right.

Tavenner: …Humans often have, or at least people like me, feel like, “Oh, one’s got to be better than the other.” And so we try to convince ourselves, but I love this idea of, “Let’s talk about specifically how it’s going to happen in schools or what it will do in schools.” And so let’s start with the status quo. If you’re a teacher who is still assigning homework that asks students to answer basic questions about what they’ve read, and that’s a pretty standard English class practice. Michael, I think you probably had it. I had it. Lots of people still have it. ChatGPT will absolutely be good at completing those types of assignments. There’s no denying that it writes pretty well. The language is clear, it’s grammatically correct, and in my experience, it was just thorough, if not verbose in some cases.

And so despite its challenges with Hobbs and my bio, it’s pretty good at knowing the types of things teachers are asking, at least middle and high school students about what they’ve read in a textbook or listened to in a lecture. And so even today, things like short answer questions on homework assignments, basics, essays will be handled pretty well by this new technology.

And Michael, that’s nothing new. Some people will see this as the newest technology used to cheat and others will view it as a tool students can use to create better work and get better grades. And there’ll be some debates around the margins about if it’s plagiarism or whatever, but ChatGPT — I’m going to identify the faulty premise here, which is the assignment and this type of teaching that’s actually the faulty premise. The Atlantic article lamented that the essay is the way we teach children how to research, think and write, which could not be further from the truth.

Horn: OK, so I want you to say more in just a moment on that “could not be further from the truth” line, so we can unpack that. But before you do, one thing that I just want to comment on because it’s something you said and I think it’s spot-on, which is that some people are going to argue that this is a great tool for people to do more and others are going to take the negative view of this and all the ways that you referenced, we’re going to catch kids cheating and all this stuff. We actually had an episode of My Future You podcast on a different topic. It was virtual reality where the guest, Sean Michael Morris talked about just how backward it is that we have a pedagogy and a school system that villainize students in this way and asks, “Are they cheating or not?” And tries to catch them.

Instead of asking how we might build learning environments that plays into the idea of each and every student achieving success. And it’s just a different frame of mind. And I just wanted to pause on it because it’s a shift from that zero sum positive sum thinking that we talked way back in the first season of Class Disrupted. That can really change one’s vantage point in all this. But to go back to your statement, Diane, that the essay, I think what you’re saying is that it’s not the only way to teach people to research, think and write, but I’d love you to unpack it more.

Tavenner: I’m happy to, and in a refreshing turn, I don’t even think I have to get that technical or nerdy today because if we all just pause for a moment and sort of decompose the statement, the teachers use essays to teach students how to research, think and write. The words “research,” “think” and “write,” Michael, they’re not describing a single skill or even a small group of skills. They’re describing a vast set of skills that are all bundled under this big headline. I mean, think about “think,” thinking is a huge multi-skill. There’s so much going on when a person is thinking, and this terminology may not matter in day-to-day life, but when someone takes on the responsibility of teaching, thinking, or writing or researching, you take on the responsibility to begin by breaking down those big categories into, I’m just going to call them micro-skills so they can be learned and then ultimately mastered.

I mean, that’s our job as teachers is to help every child learn these things. And so for example, at the heart of an essay is a claim or a thesis statement or a main idea. Those are somewhat interchangeable terms for at the heart of an essay you’ve got to have a central idea. And so students, young children can learn about main ideas, Michael, they can learn to identify them and what they’re hearing or reading, notice that’s not writing them down yet. They can also learn to come up with them on their own and then write them. Notice that just in that example, there are a number of skills and ways that you can express. You can express them in writing verbally, graphically, all of those things are main ideas, and that’s all part of the learning process. And it only gets more complex from there.

Main ideas become more sophisticated and turn into, for example, an argumentative claim. The content that the idea’s about becomes more complex. And the language used to express the idea’s more nuanced and detailed. The point being here that the same set of skills that a second-grader learns are scaled all the way up to professional writers publishing essays in The Atlantic. Maybe we could have done a little bit more work there. The way people learn these skills and get better at them is through practice. Yes, so, writing, but just writing things is not going to get there. The way that you get better is by getting feedback on your writing or your thinking or what you’re seeing, reflecting on it and incorporating it. That’s the learning process. And I guess what I’m saying, Michael, is let’s not be confused, writing an essay and getting a letter grade on it has nothing to do with learning, as you noted. And I would love for us to be obsessed with what we’re doing in schools with the tools we have, including this new AI tool to help children learn. That’s what we should be obsessed with.

Horn: I like where you’re going with this, Diane, and bringing it back to learning, and not just the artifact I would call it, of learning, if you will, and development of skills into what might be called a skillset and a set of knowledge that allows an individual to do some pretty complex things. But let’s leave our audience with something practical. And I’m very curious on this one, which is how would you, Diane Tavenner, use such a tool?

Tavenner: Wow. OK. This one’s fun. All right, let me think for a minute. So you’re asking me to put my money where my mouth is, which is a fair ask. Well, let’s start with one of the first things that comes to my mind is how we might use the answers that this chatbot generates to create what we would call exemplars that learners could dissect. And so what do I mean by that? Well, one of the tools we use in teaching is to create models or examples of essays or paragraphs or pieces of writing. We want students to produce themselves, and we use them as examples. We call them exemplars. And the idea is that you need to see a model of what you’re aiming for so that you know what you’re going after, what you’re trying to create. And so what if a group of students and their teacher were to ask the chatbot to create an exemplar of an essay or a paragraph that the class is going to write about.

And if I were the teacher, we would talk about and know that the chatbot is far from perfect, and there are likely mistakes and misconceptions or issues when it creates things. And then as a group, we could dissect the product that it would produce, and maybe we’d even ask it different ways to see if it produced different products. And we’d have a set of them, and we would ask ourselves, “What’s good about this? What could be improved? What did it get right? What did it get wrong?” And I would have the students use our writing rubric to score the essay and tag the claim and to be critically thinking about the quality and caliber of what was written. And just start to imagine a classroom where students are working in small groups to really dissect this writing. I think it would be buzzing, Michael. There would be so many different skills being practiced and learned in that session.

Students are doing the type of metacognitive, the thinking about thinking work that we want them to be doing. And you can see it in me, I’m getting pretty excited right now even thinking about this because I can imagine designing a lesson that would be captivating, that would be able to meet each of the students where they were in that moment. They’d be working in small groups. They would sort of have this common goal that they were going after. And I think it would be A, really fun, and B, as I put my teacher hat back on, I used to spend an extraordinary amount of time creating exemplars. The idea that this thing could create them for us and we could work with them is really awesome. All of this that I’m describing, Michael, can happen in a classroom in collaboration with others, and without a need for the teacher to take a big stack of papers home and mark it up with a red pen.

And all the while the teacher can be circulating, giving their own feedback, modeling what that looks like. And quite frankly, assessing on the spot where students are in the learning process on a whole variety of skills. I mean, I could go on and on, but hopefully you get the flavor for what would be possible, which honestly is sparking 20 more ideas in me right now. So I might have to go write some lesson plans after this, but wow, fun. Thanks for asking the question, Michael. And I must admit, I didn’t think this was an innovation when I’ve been reading about it until we’re talking now, but honestly, I’m starting to wonder, am I describing a sustaining innovation?

Horn: Oh, OK. I love how you brought it back to before we did the dive-in, Diane, and I love the ideas, right? I think it’s starting to show the productive uses in very tangible ways for folks. And I think it’s showing something else, which is the reality is that a technology by itself, it’s actually not inherently disruptive or sustaining. This was actually an improvement to the theory of disruptive innovation itself that Clay Christensen made after his first book came out. When, back then, when The Innovator’s Dilemma came out, and I suspect a bunch of people saw the movie that featured The Innovator’s Dilemma book in it over the break, but it called them disruptive technologies back then. And what Clay learned over time is it should be called “disruptive innovations” because it’s not the technology per se, it’s how the technology was used and the model that surrounded it, that determined what it was.

So, quick example for this particular technology ChatGPT, and the AI and its impact on, say, search. So the industry of internet search, on the one hand, AI is obviously a sustaining innovation, like Google is using AI all the time to improve the way it gives search results. But AI could also be disruptive to Google. Just think about the tool we’ve been talking about. And if you use that to do your search, and you could immediately see in your mind’s eye how it would undermine Google’s paid ads and the like. Yeah, it would not be as good as Google with its mistakes and occasionally, frankly, very offensive answers. But it would be a lot more affordable to run, it’d be simpler to use and so forth.

So over time, you can imagine this disrupting search. But what’s interesting, Diane, is the audience listening right now actually did not need me to explain this because in this case, I fed the question into the open AI chat tool, and it told me the same thing. It basically said it needed more context to answer the question because it depends on how the technology is used. Now that answer, I’m quite sure, is not nearly as elegant as hearing me express it in my words. But that’s where this all… 

Tavenner: No chance.

Horn: Right, thank you. But that’s where this all connects, which is what you described as not a traditional classroom. It’s a redesigned classroom. It’s the kind that you and I talk about and we advocate for all the time, which is perhaps why this technology doesn’t have to be a threat, but it can be a potential advantage. But the thing I want to leave people with is, it’s all about how the technology is used, and it’s not just the technology itself, but the model is far, far, far more important.

Tavenner: Exactly. And honestly, this is where you and I have always come together around what we want and what we see and how we view what’s happening in education. This is always our key point. Technology is a tool. It’s a super powerful tool. A tool that we would be wise to use more liberally and effectively in education. But that path is through redesign models of schools and learning environments. It’s not a silver bullet.

And Michael, that feels like a good place to leave things today. But before we go, what have you been reading, watching, and listening to with so much down/sick time over the holidays?

Horn: Oh boy. Good question. Well, so I’ve done a few things, Diane, I’ve read a forthcoming book on critical thinking for a publisher. I’ve been reading a lot of student papers, interestingly enough for today’s topic and grading them. But I’ve also been trying to unplug, and I binged over the holidays on the just-released third season of Jack Ryan on Amazon Prime Video. And it plays to the types of storylines and entertainments that my wife and I love getting lost in together, which has been fun. And I will say the writers did a heck of a job making the content feel awfully and sometimes terrifyingly current. So I’ll leave it there. But what have you been looking at?

Tavenner: Fascinating. Well, some more for my list, as always. Two fun things from me this week. Over the holidays, we watched Glass Onion, a Knives Out Mystery, which is honestly, in many ways just plain fun. And if you want it to be just that, go for it. But it honestly was also a really good chance to reflect on everything from perceptions of folks in Silicon Valley that hit a little close to home, as well as race and gender dynamics and a host of other modern issues too, so to your point, very, very current. And I spent a lot of time buried in Dishoom: From Bombay with Love, which is described by the authors as “a cookery book and highly subjective guide to Bombay with map.” That’s literally how they described it. It’s the experience, the love of these authors for their city has me so excited to visit. And of course, the incredible recipes. As you know, I love to cook. So what a fun experience that was.
Horn: Oh, those are both speaking right to you. And for those who caught my reference earlier, Glass Onion is where the Innovator’s Dilemma makes a cameo at about an hour 23, if I’m not mistaken. So with that, we will leave this conversation there and say a happy new year from our Class Disrupted family to yours, and we’ll see you next time.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter