Companies like Amazon and Facebook have systems in place to continuously respond to how users interact with their apps to facilitate the user experience. What if educators could use the same “adaptive experimentation” strategy to periodically improve their teaching materials?
This is the question posed by a group of researchers who developed a free tool called the Adaptive Experimentation Accelerator. This AI-powered system recently earned him first place in the annual XPrize Digital Learning Challenge. The challenge splits his $1 million among the winners.
“At Amazon and Facebook, in order to quickly get a better understanding of what small changes would be more effective, we were able to quickly adjust the situation, change what the audience sees, and make that change more. audience,” said director Norman Beer. He is a member of the Open Learning Initiative at Carnegie Mellon University and worked on the project. “Thinking of this in the context of education really opens up the opportunity to give more students the kind of things that support learning more effectively.”
Bia and others working on the project said they are testing the approach in a variety of educational settings, including public and private K-12 schools, community colleges and four-year colleges.
Edsurge interviewed Beer and another researcher on the project, Ph.D. We heard more about their thoughts on the challenges and opportunities to take advantage of it. AI in the classroom.
The discussion took place in front of a live audience at the recent ISTE Live conference in Philadelphia. (EdSurge is an independent newsroom that shares a parent organization with ISTE. Learn more about EdSurge’s ethics and policies here. Supporters here.)
Listen to the episode where you can get podcasts, such as Apple Podcasts, Overcast, Spotify, or use the player on this page. Or read the partial transcript below, lightly edited for clarity.
EdSurge: The app you developed helps teachers test learning materials to see if they work. What’s new in your approach?
Norman Beer: If you think about a standard A/B test, [for testing webpages], they usually work on averages. Trying to average everything out creates a student population where interventions that are good for everyone are bad for the individual. One of the real benefits of adaptive experimentation is that we can start to identify, “Who are the subgroups of students?”, “What specifically would be a better intervention for them?” being able to provide them. We will continue to provide real-time interventions that are better for them. Therefore, we believe there is a real opportunity to serve students better and approach the concept of experimentation more equitably.
One aspect of this, I understand, is called “learner sourcing”. what is that?
Stephen Moore: The concept of learner sourcing is similar to crowdsourcing with large numbers of people participating. Remember the game show Who Wants to Be a Millionaire? When contestants vote for the audience. They ask the audience, “Here he has four options.” I don’t know which one, which one should I choose? ‘ And the audience says, ‘Oh, please choose option A.’ This is an example of crowdsourcing and the wisdom of crowds. All these brilliant minds band together to find a solution.
So learner sourcing is an extension of this where you actually get all the data from the students in your courses, you collect the data from the students in these massive open online courses, and you actually get them to do something. , you can put it back into the system. course.
One particular example is asking a student taking an online chemistry course to create a multiple-choice question. So if you have a course with 5,000 students and they all choose to create multiple-choice questions, that chemistry course will have her 5,000 new multiple-choice questions created. increase.
But what about their quality? Honestly, it can vary greatly. But with the whole wave of ChatGPT and all these massive language models and natural language processing, we’re now able to process and improve those 5,000 questions to find the best questions that can actually be used in our courses. I was. Instead of just blindly returning to the course.
Beer: We are asking students to write these questions not because we are looking for free labor, but because we believe they will actually help them develop their knowledge. Their questions and feedback help us improve our course materials. We feel from a lot of research that a beginner’s perspective is actually very important, especially in lower level courses. And a rather implicit idea of this approach is that it takes advantage of the novice perspective that students are bringing with them, the idea that everyone loses as they gain expertise.
What role does AI play in your approach?
Moore: In our XPrize effort, we had some algorithms powering the backend that took all the student data and basically did the analysis “Should student X be given this intervention?” So AI was definitely a big part of that.
What are the scenarios for how classroom teachers use your tool?
Beer: The Open Learning Initiative has a statistics course. This is an adaptive course. Think of it as an interactive tech textbook. Thousands of students in Georgia colleges use this statistics course as an alternative to textbooks. Students read books, watch videos, but more importantly, get involved immediately to answer questions and get targeted feedback. So we can introduce these learner sourcing questions into this environment and some approaches to motivate students to write their own questions.
Moore: There is a good example from one of the pilot tests for this project. We wanted to know how we could engage students in any activity. This OLI system has all these great activities and I would like my students to do additional statistical problems and such, but no one really wants to do it. So we’d like to say, “Hey, if we can provide a motivational message or something, hey, go ahead, do 5 more problems, and you’ll learn more.” You can do better on these exams and tests.” ” How can these motivational messages be tailored to engage students in these optional activities, whether they are providing learner information or simply answering multiple-choice questions? do you want?
And there were some motivational phrases in this XPRIZE contest pilot test. But one of them was about memes. I thought maybe some undergraduates taking this particular course would like memes. So I inserted a picture of a capybara that resembles a large hamster or guinea pig. You’re sitting in front of your computer with your headphones on, your glasses on, and no text. We’re like, ‘Let’s throw this in and see if the students do it. And in about five different conditions, just a picture of a capybara with headphones on and facing a computer increased student engagement in subsequent activities. Perhaps it made them laugh, but no one knows exactly why. But compared to all these motivational messages, it worked best for that particular class.
Capybara with reading glasses looking at a laptop computer pic.twitter.com/xCYVlDI17d
— React (@reactjpg) April 8, 2021
There is a lot of excitement and concern about ChatGPT and the latest generative AI tools in education. Where are you two on that continuum?
Moore: I definitely play both sides. I see a lot of great progress happening there, but you absolutely have to be very hesitant. I think the output from the generative AI you’re using always needs a human eye. Always keep a human eye in mind instead of just blindly trusting what you have been given.
I’d also like to throw away that ChatGPT’s plagiarism detection capabilities are currently terrible. don’t use them.they are unfair [because of false positives].
Beer: This human up-to-date concept is a hallmark of our work at CMU, and we’ve been thinking strategically about how to keep that human up-to-date. And that kind of contradicts some of the current hype. “What we really need is to build a magical tutor who can directly access and ask questions of every student,” some quip. It has many problems. We all know this technology has a tendency to hallucinate, but that tendency is further complicated by the fact that many learning studies show that we like things that confirm our misconceptions. will be Our students are least likely to challenge this bot if it tells them what they already believe.
So we’ve been thinking about what are the deeper applications of this, and how we can use those applications while keeping a constant eye on humans. And there are many things we can do. The development of content such as adaptive systems has aspects that humans are good at but dislike. As a courseware author, my faculty authors hate writing questions with good feedback. That’s not what they want to spend their time on. So it’s exciting for us to provide a way for these tools to start delivering the first drafts that are still under review.
listen to the entire conversation EdSurge Podcast of the Week.
