MARY MELTON: Today I’m talking to Ethan Mollick, an entrepreneurship and innovation professor at the Wharton School of Business, who embraces the power of AI to further the education of his students—and of his own education. In January, Ethan mandated the use of AI in his curriculum. In today’s episode, he shares what he’s learned from that experience and how he sees AI positively transforming not just the future of education, but of entrepreneurship and the workplace. He talks about how business leaders can use the technology to help them in decision making, and he walks us through some specific cases of AI in action in the workplace. When he isn’t researching or teaching, he’s advising start-ups and organizations. And now, my conversation with Ethan.
MARY MELTON: Hello, Ethan Mollick, and welcome to WorkLab. Thank you for joining us.
ETHAN MOLLICK: Thanks for having me. I’m really pleased to be here.
MARY MELTON: What are the topics of expertise that you dabble in, and how does AI fit into all of those?
ETHAN MOLLICK: So I’m sort of an accidental expert in AI. I’ve been AI-adjacent my whole career. So I worked with the MIT Media Lab and Marvin Minsky’s AI lab back in the day, but I’ve never been a computer scientist. What I have been fascinated by is both entrepreneurship—so, I teach a lot on research entrepreneurship, especially team success and innovation. I’m also super interested in, how do we teach in new ways. So I run something at Wharton called Wharton Interactive, which is their internal game studio, where we build teaching games to teach business skills at scale. And that’s sort of where I’ve been encountering AI the most is, how do we use this as a teaching tool? So I’ve been playing with this a lot since before ChatGPT came out. When Chat was released, it happened to overlap very much with what I was already studying and interested in. So I sort of took a deep dive into that area.
MARY MELTON: When did you start Wharton Interactive?
ETHAN MOLLICK: It’s been around in one form or another since about 2014, officially kind of kicked off in 2018. So I’ve been building games for teaching for a while. I wrote a book on the topic back over a decade ago, so it’s been a topic of a lot of interest. How do we teach the most people real work skills that are useful at scale? Because it turns out, even minor amounts of business knowledge can transform people’s lives. So it’s a really important thing to be able to do.
MARY MELTON: So you’ve got a wealth of knowledge on the topic of AI. Can you reflect for a moment on what you make of this moment? And were you surprised at how fast we got here?
ETHAN MOLLICK: Oh, absolutely. I mean, AI has always been almost here, right? So before Chat came out in November of last year, I was experimenting with GPT-3, the previous version. It was kind of miraculous, it wrote as well as a fifth grader. Like, that was so cool. But we’ve been expecting AI to be the thing forever, and it hasn’t ever taken off, right? There’s been these AI winters. I think I was less surprised than a lot of people, because once I saw Chat, I was like, oh my god, this is the moment. It’s all going to happen here. Maybe took other people a month or two to catch up, but that’s a pretty fast adoption curve for any technology.
MARY MELTON: Where were you when you first realized that this had taken off, like, that this was going to become actually the topic du jour and move from something that was in the background to something that everyone is talking about?
ETHAN MOLLICK: Actually, I introduced it to my entrepreneurship class three days, four days after ChatGPT came out, and by the end of that first class, one of the students had already created a working software prototype using GPT-3.5, using Chat, to demonstrate the product they were developing for the class. And I posted it on Twitter that night. By the next day, we see scouts had already contacted them about potential funding opportunities. By the Thursday, a couple of days after that, 60–70 percent of my students already used Chat to do things anywhere from working on better messages for their clubs to explaining why they got problems wrong on tests to helping them brainstorm ideas for outlines, all sorts of uses.
MARY MELTON: So on one hand, you sound very positive, but on the other hand, I’ve also seen that you’ve written in your newsletter, which is called One Useful Thing, that we’re living in something that you’ve described as an “AI-haunted era.” How does that measure up with the positivity part of it?
ETHAN MOLLICK: AI is a general purpose technology. It is going to affect everything we do. General purpose technologies are these rare events like steam power, the computer, or electrification, or maybe the internet, where a new technology comes along that touches everything. And so AI is doing that, right, and that means its results are going to be very different to different places. Some industries will be unaffected—not that many, but some. Some industries will be hugely affected. Some jobs it will have a huge impact, some will not. It’s hard to know in advance. So when I say “AI-haunted era,” I mean AI is kind of a background to everything we’re doing, sort of like the internet is today. And that is going to be both good and bad. I think that trying to lump this together as it’s one set of risks or dangers, you know, versus one set of big wins is hard. It’s going to be that way on a very micro basis. The level of jobs, organizations, companies, industries, countries, societies is a big picture.
MARY MELTON: So looking at it, you see it as that much of a game changer in the way that steam engines and the internet changed the way we live our lives.
ETHAN MOLLICK: I want to make it clear, like when people talk about technologies in the future, they often talk about them—like, I’m sure you had blockchain conversations on this. Blockchain was like five years out, and the proponents were like, It’s going to change everything in five years from now. Like, that’s not the case with this. And I think it’s hard for people to wrap their head around the fact that, like, this is here now. If every letter banning AI goes through and we don’t produce any more AI after today’s, it’s still going to have a profound effect on how we work, on how we learn, because it’s an incredibly capable system already. I don’t feel like I’m going out on a limb here to say that it’s going to be transformational, because you don’t need to wait four years or five years to see if it’s transformational. You can see it right now in the fact that 14 percent of Americans have already tried this technology, which is a really new technology. And of those, you know, over a third of them consider it incredibly useful and a third find it useful. And very few people find it not useful at all. And that’s without any training or knowledge. So I think this is the beginning of something big.
MARY MELTON: Fourteen percent is a huge number.
ETHAN MOLLICK: This is the fastest technology we know of to 100 million users.
MARY MELTON: Wow.
ETHAN MOLLICK: It’s a big deal, right, ChatGPT. So 14 percent penetration of the US in a short period of time for a new tech is quite big.
MARY MELTON: One thing you’ve said is that we should think of AI as a person, not a software. Tell me more about what you think about that.
ETHAN MOLLICK: Let’s start two steps back. Let’s just talk about what AI is, because it means a lot of different things, right? People think about the Terminator robot or about HAL or about Jarvis in Iron Man, or they think about self-driving cars, the kind of AI that business analytics at Microsoft offers. That was sort of what we talked about with AI before November, which is the idea of machine learning, of predictive analytics, the idea that you could take a whole bunch of data, throw it at the AI, and it would tell you a pattern in that data. And, pretty good at predicting patterns, it was pretty bad at human-sounding interactions. So in 2017, a famous paper called “Attention Is All You Need,” and it proposes the idea of a large language model and a few tools that created it. Large language models are also predictive. They’re predicting the future, but they’re predicting what word or part of a word, called a token, would come next in a conversation—so, fancy autocompletes, essentially. So they sucked out every piece of the information on the internet and created very complex associations between various words and phrases to complete sentences. Now, the weird thing that happened is when those models got large enough, when they reached the size of billions of parameters the way ChatGPT did, then they started to exhibit a large amount of the illusion of reasoning and creativity. I mean, they actually act creative, right? We don’t quite know the reasons why the size of the model made such a difference. It’s not that these systems are sentient, but as a result, they act in a way very different than other kinds of software, they act more like people than like software. And by that I don’t mean they’re alive, they’re not sentient, but that they are good at humanlike tasks, like writing and coding. They’re bad at machine-like tasks like math, and they make mistakes and sort of fool themselves like humans do. So when I say “work with them like humans,” I don’t mean they’re people, but I do mean that it’s a useful way to think about what they’re good at rather than thinking about them like software.
MARY MELTON: Well, let’s talk a little bit about what AI can and cannot do. You wrote a practical guide about this and the six capabilities that you stated. It can write stuff. It can make images. It can come up with ideas. It can make videos. It can code and it can learn stuff. Which one of those do you think are going to be most useful to business leaders?
ETHAN MOLLICK: So we missed a few things there, right? Like, it could do analysis. It is capable of doing original work as well. I mean, look, the biggest use and the thing all business leaders are going to need to grapple with is AI being integrated into office applications, writing performance reviews, writing a marketing research document, writing marketing material, where the AI can do that stuff faster. So I think this is a huge opportunity to think about, what do we do with a giant productivity gain? How do we get people to do more meaningful work and that they’re aimed in the right directions? There’s a lot of open questions to think about there.
MARY MELTON: So what are the best ways to write a prompt or engage with a tool like Bing Chat? And also, what are some common mistakes that people are making?
ETHAN MOLLICK: So on the common mistakes side, the first three things everyone does with AI are always kind of the same. It doesn’t work like traditional search, right, it’ll get some things wrong, it integrates information. That’s the first thing people do. The second thing people do is try and interact with it, like having a fun conversation, usually ask it about the future of AI. The AI is not magical. It doesn’t know the future, and it doesn’t have a personality really. So people get frustrated. Third thing they do is maybe they ask it stuff about themselves and they run again into hallucinations. The idea that when you ask the AI to know something it doesn’t know, it makes up the information. That’s a very common consequence, and then people get pretty annoyed and walk away. The problem is that that’s not really showcasing what makes AI powerful. It is actually quite good at search in the right kind of way. Think about it like an intern you’re delegating tasks to: Write me a draft of something. Actually, paragraph two is pretty good. Make paragraph three better. Add a different example in paragraph four. Can you make it sound more formal? That kind of interaction is much more powerful, so it’s less of us starting with the perfect prompt, but it’s much more about interacting with the system the way you would with a person.
MARY MELTON: Yeah. What have you learned from how you’ve approached bringing AI into the classroom that may be helpful for managers and leaders, in terms of creating that psychological safe space to create an environment where you’re talking about this and you’re sharing best practices and what you’re learning. I think, based on my understanding, that you’ve made it actually mandatory for students to use AI.
ETHAN MOLLICK: Yes, we are seeing 30 to 70 percent performance improvements across different studies. Nobody knows the real answer yet, but that’s huge. Put that in context: steam power was 18 to 22 percent when it was put into a factory in the early 1800s. These are numbers we’ve never seen before, right? Companies will do a massive installation of software to get a 3 or 4 percent performance improvement. These are huge numbers. This is the biggest thing that’s happened to white collar work—you know, at least since the computer, maybe even, you know, before. It’s hard to know. And it’s happening all at once. So I think every organization should have every alarm bell ringing about what’s going on now. Both about how their employees are using it, how they might want to use it, how they could gain an advantage, how competitors might gain an advantage, how everybody all over the world who didn’t used to speak English fluently can now speak English fluently. And that’s a big change to happen overnight. So I think there’s two things. One, making it mandatory, making people use it. I don’t think you’d be remiss as a leader of a large scale Fortune 1000 company to take, you know, the top 20 percent most creative people in your company, require they all use AI for a week, and give a million-dollar prize to whoever comes up with the best way to automate parts of their job while promising you’re not going to fire anyone as a result of this. Like, I don’t think it’s an overreaction. I think a lot of people are viewing this as, is it an IT problem or a legal problem or a grand strategy problem. It’s not. This is a very down and dirty situation that has to be dealt with. So what I’ve learned from class is, people have to use it a lot. You need like 10 hours of time on ChatGPT or Bing or whatever before you start to actually get use out of it and really get it. And then you also need some training. It helps to understand, the training is not like you have to pay a consultant tons of money. It’s just a very basic sense of like, okay, you interact with this like you do a person, but a little bit of training does help.
MARY MELTON: You’ve said that in the future, AI in classrooms will be undetectable, ubiquitous, and transformative, and that the quality of the work for your students has improved since doing this. Is that right?
ETHAN MOLLICK: I mean, I have really smart students, but the quality of ideas has definitely gone up because now people can bounce ideas off more people. Certainly I’ve demanded a lot more material for my class. So they used to have to put an outline together. Now the outline actually has to be critiqued by three famous entrepreneurs, have 10 possible things that go wrong, require one almost impossible task they do, and it has to have some visions of the future, all generated by AI to go along with the outline that they write.
MARY MELTON: Now, have you required them to do more work because you know they’re going to have more time to do more work because of the assistance of the AI?
ETHAN MOLLICK: Yeah, a lot of the things that we used to have to spend time on, we don’t have to and we can generate a lot more material. It changes the way you relate to work, right? You’re working in hybrid with the AI; you’re not just working on your own anymore.
MARY MELTON: Tell me, what is the transfer problem?
ETHAN MOLLICK: So there’s a general problem in education. We could teach you stuff in a classroom pretty well, but people have trouble applying that to other situations other than exactly what they learn in class. So that’s transfer. If I teach you how to solve a math problem, will you see that math problem in the real world and know how to solve it? AI has a lot of really positive things it could do for education. One of the sets of stuff is about, you know, helping teachers. One way to transfer ideas is to actually teach someone else who could teach the AI, correct it when it’s wrong about topics. We’ve also been using AIs to create simulations so students can have a simulated partner to negotiate with, or discuss things with—another really powerful approach to solving transfer with AI.
MARY MELTON: Those are all very exciting possibilities. Is there any one in particular that excites you the most regarding the future of AI, and the impacts it could have on either education and/or entrepreneurship?
ETHAN MOLLICK: Both of them have the same kind of answer, which is that one of the problems we have in the world is the hidden Einstein problem, right? Which is that talent is much more evenly distributed than opportunity. Just to give you some examples. The start-up world is full of broken opportunities. So people in Philadelphia raised more money for venture capital last year than everyone in Japan put together. Actually, Penn grads raised more money than everyone of France and Germany put together. Women make up 38 percent of business owners in the United States—they only get 2 percent of venture capital. These are not even numbers. And that’s just in the US where you have access to things. There’s lots of parts of the world where there’s very smart people who don’t have access to the kinds of tools or abilities that we do here, and that includes an opportunity to learn. We’ve known for a long time, or at least strongly suspected, that the most transformative kind of teaching you can get is essentially one-on-one adaptive tutoring. And it’s really hard to do that at scale. It’s very hard to do in much of the world where there’s not a lot of money in teaching and people have lots of opportunity costs when they’re do teach. You can actually do some really impressive tutoring at scale. So the idea of having a tool that is universally applicable, that works for everybody around the world—Bing’s in like 169 countries, I think. I mean, that’s an incredible tool. So to me it’s the democratization of opportunity. Think about all the innovations and things, the ideas that were lost that can now be taken advantage of.
MARY MELTON: You’ve talked about how getting AI ready requires rethinking systems rather than job roles. Can you say more about that?
ETHAN MOLLICK: So there’s actually, jobs is the wrong unit of analysis for thinking about change. When we talk about jobs in academic literature, we actually think about jobs as bundles of tasks. And some of those tasks, AI is going to be very good at helping you with. Some, it’s going to be able to take things off your plate, some it’s not going to be good at at all. So change is going to happen at the task level, not the job level. Change is also going to happen at the system level. The way we run companies today is the same way we ran companies roughly in 1920 or even 1853, right? Large multinational corporations, lots of layers of middle management. Those are dependent on the technologies and capabilities we have today. So that’s about to change. We have different capabilities now. Are you still going to do sprints as the way of organizing work? When AI can let some people work much harder than they did before. You don’t need to wait for people to catch up. Do you still want to have all the stand-up meetings you did? Like, we have to change the systems of work, and that’s going to be a very big change.
MARY MELTON: That’s a huge change. You teach entrepreneurship and you work with start-ups. You have said that AI is an amazing co-founder.
ETHAN MOLLICK: So, a third of Americans had an idea for a start-up and haven’t done anything about it. And part of it is there’s lots of barriers. It’s hard to do research. It’s hard to write a business plan. Guess what? You can ask the AI, Give me 20 ideas for how to launch a business. You know, tell me details, step by step, how to do it. Write me the letter that I need to send. Help me fill out this form. Help me create code for this. How should I test this idea? You have a co-founder you got for free that can help you with lots of tasks. That’s incredible power.
MARY MELTON: And you’ve experimented with this yourself and with your students. And have you found the answers that you get when you propose something like, Give me 20 ideas for how to write a business plan are pretty on target?
ETHAN MOLLICK: Yes. I mean, they’re not right. I mean, but most ideas are wrong. When I ask it for business advice, it’s good, right? I would say, you know, a lot of the common tasks out there, AI hits around the 80th percentile of ability. Like, I’d like to think I teach a better class than the AI would, but it’s not terrible, right? It makes mistakes too. But so do humans. I find it to be very useful to use this as an adjunct to the kind of work you’re doing otherwise. It’s good enough to kind of get you over the starting line. Not as good as the best human, but pretty good.
MARY MELTON: And it sounds like it gives you great jumping-off points to think about ways to phrase questions to yourself or for something that’s like a larger business plan.
ETHAN MOLLICK: Even more than that, there is a bunch of research that is coming out showing that AI is reasonable as a proxy to talk to also for market research. So you can interview the AI and you’ll get reasonable answers. They’re not going to be accurate as much as talking to people, but it can help you practice talking to people to get some interesting ideas. When you survey AI about willingness to pay, it gives reasonably accurate survey results. So it’s not just about, you know, having a companion to punch ideas off of. It’s not just a tool to create content. It’s also about this other piece.
MARY MELTON: That’s incredible. What is some advice you could give to a business leader who hasn’t yet dived deeply into this and may be feeling nervous about what they should be doing?
ETHAN MOLLICK: I really strongly believe the only way out is through here, and you have to just start using it. So the question is, who’s using it in your organization? Do they feel safe talking to you about how they’re using it? Are you using it? The idea that somebody is, like, too busy to play with AI, I can tell you this is a COVID moment. This is as big a deal as anything your organization has ever encountered. And you need to be spending your time right now dealing with this. So just putting things on the back burner doesn’t make sense either. I see people handing things off to IT departments. This is not a really good IT solution. It’s something very different. So you can’t just have IT handling it. This has to be a whole-of-organization approach to solving and addressing a very, very, very big burning issue.
MARY MELTON: It’s not too late to jump in, obviously. You’re right at the start of it, but at the same time it can be too late pretty fast if you don’t start.
ETHAN MOLLICK: If you really—if scenarios 2 or 3 are right and there’s either exponential growth or continued fast linear growth, right, in some way, then you need to get used to this now, because only then will you start to get a sense of what’s happening next. I just can’t emphasize it enough: It’s not too late. But, you know, this is the second-best time to start using AI. The first best time was a couple of months ago.
MARY MELTON: Well, thank you so much, Ethan Mollick, for joining us and getting us inspired to get in there and not be scared and start working on it.
ETHAN MOLLICK: Oh, well, thank you for having me.
MARY MELTON: Thank you so much.
MARY MELTON: Thanks again to Ethan Mollick for that insightful and really fascinating conversation about the future of work and AI. If you’ve got a question you’d like us to pose to leaders, drop us an email at worklab@microsoft.com. And check out the WorkLab digital publication, where you’ll find transcripts of all of our episodes, along with thoughtful stories that explore the ways we work today. You can find all of it at Microsoft.com/WorkLab. As for this podcast, rate us, review us, and follow us wherever you listen, please. It helps us out a lot. The WorkLab podcast is a place for experts to share their insights and opinions. As students of the future of work, Microsoft values inputs from a diverse set of voices. That said, the opinions and findings of our guests are their own, and they may not necessarily reflect Microsoft’s own research or positions. WorkLab is produced by Microsoft with Godfrey Dadich Partners and Reasonable Volume. I’m your host, Mary Melton, and my co-host is Elise Hu. Sharon Kallander and Matthew Duncan produce this podcast. Jessica Voelker is the WorkLab editor. Thanks for listening.
