Why universities are turning to oral exams to fight AI

This assignment does not involve laptops, chatbots, or any kind of technology. In fact, there is no pen or paper.

Instead, students in Chris Schaefer’s biomedical engineering class at Cornell University are required to speak directly to their instructors in what Schaefer calls an “oral defense.”

This is a testing method as old as Socrates, and it’s making a comeback in the age of AI. To address the crisis in higher education, a growing number of university professors are turning to oral exams and combining old-fashioned and cutting-edge techniques.

“With oral exams, you can’t leverage AI as much as you want,” says Schaefer, who introduced oral defense last semester.

Educators are no longer simply wondering if their students will use generative AI to do their homework for them. The big question now is how to determine what students are actually learning.

University instructors across the country are noticing a troubling new trend as generative artificial intelligence becomes more sophisticated. The essays and other written assignments I brought back are coming back perfect. However, when students are asked to explain their work, they cannot. The long-term impact of using AI on critical thinking remains to be seen, but educators are concerned that students are beginning to see the hard work of thinking as optional.

Some universities have moved to face-to-face exams

Emily Hammer, associate professor of Middle Eastern languages ​​and cultures at the University of Pennsylvania, currently combines oral and written exams in her seminar classes.

“It looks like they’re trying to deter fraud,” Hammer said. “That’s not why we’re doing this. We’re doing this because students are actually losing skills, they’re losing cognitive abilities, they’re losing creativity.”

Ms. Hammer prohibits the use of AI in all writing assignments, but she tells her class that she knows she can’t force it. But if you haven’t written the paper yourself, defending the material face-to-face is likely to be a “very stressful situation.”

Bruce Renthall, executive director of the school’s Center for Teaching and Learning, said Hammer’s class is part of Penn’s “large-scale transition to in-person assessment” of written and oral forms. The Ivy League school is one of a small but growing number of universities that has begun holding workshops for faculty on oral exams.

Unlike some European universities, oral exams have not traditionally been part of the modern American undergraduate system. For example, in the UK’s Oxbridge tutoring system, students meet weekly with their teachers for discussions. Interest in ChatGPT has increased since its launch in 2022, with some U.S. universities moving to oral exams to address concerns about online cheating during the COVID-19 pandemic.

Amid the pandemic, engineering professor Huihui Qi began a three-year study of how to scale up oral exams at the University of California, San Diego. Since then, several universities have invited her to hold workshops for faculty or discuss her research.

Combat “fire with fire” with AI

At New York University, several types of oral evaluations are on the rise. Increasingly, faculty are requesting office hours, assigning presentations, and calling students in class. Instructors “need to look students in the eye and ask, ‘Do you know this material?'” said Clay Sharkey, vice provost for AI and educational technology.

A New York University professor has put a modern twist on the traditional oral exam.

Last semester, Professor Panos Ipeirotis of New York University’s Stern School of Business announced an AI-powered oral exam for the final exam of his AI product management class. He calls it “putting out fire with fire.”

Students can log in from home whenever it suits their schedule. A cloned voice from a business school professor greets them.

A voice saying “Hello” appears on the screen. After asking for the student’s name and school ID number, the system displays “Ready to administer the exam today.”

The chatbot starts with questions about the final group project and drills down into details based on each student’s answers. If a student stumbles, the AI ​​agent provides hints along with criticism and positive feedback. Ipeirotis scores exams independently, also with the help of AI.

“What we wanted to ask was: Did you know what your team did? Were you a free rider? Did you outsource everything to AI?” says Ipeirotis, who designed the tool in collaboration with Eleven Labs, a company that develops generative AI voice agents to conduct job interviews.

Students in this semester’s class have redesigned the AI ​​agent to iron out some glitches, and Ipeirotis plans to use the agent in all future classes.

“I now want to do oral exams everywhere. I want to combine them with all written assignments,” Ipeirotis says. “I no longer believe that written assignments are the result of actual thinking.”

Feedback from students last semester was mixed.

Business major Andrea Rui found the chatbot’s voice to be surprisingly human-like, but the conversation felt oddly choppy and choppy. I got confused because multiple questions came up at once. And it was uncomfortable to hear voices but not see people.

“It felt kind of awkward talking to a mostly blank screen,” says Louis, 21.

But she agreed with concerned educators: “There’s no perfect world where AI exists and kids don’t abuse it.”

Schools believe shy students can benefit too.

Across STEM fields, including the humanities and computer science, educators are concerned that students who skip the brain games needed to solve problems will not develop the skills they need to advance to advanced classes and careers.

That’s why Professor Schaefer at Cornell University introduced oral defenses to his biomedical engineering classes. He asks students to submit written problem sets, which are given several times each semester, and then sign up for a 20-minute session of Socratic questioning.

In a class of 70 students, Schaefer shares work with a teaching assistant. They no longer grade written question sets, only oral arguments. He calls this “motivating” students to do the work, or at least understand it well enough to explain it.

Schaefer’s class is featured in a new Oral Assessment Workshop offered by Cornell University’s Center for Educational Innovation.

In another example at Cornell University, a religious studies professor began holding a 30-minute “final conversation” with students in lieu of a final exam. In another engineering course, a professor conducts a four-minute mock interview with each student in the 180-person class.

Carolyn Aslan, who heads Cornell University’s oral exam training, says skeptics say oral exams can be unnerving for shy students or those with severe anxiety, but clarifying the format in advance and starting with softball questions can help.

“Sometimes it’s actually good to have one-on-one conversations with quiet students, so you can finally hear their point of view, and that can be a game-changer,” says Aslan.

Some of Schaefer’s students said they were nervous at first, but ended up liking the oral exams better.

“Honestly, I loved it,” says Olivia Picerchia, a third-year biomedical engineering major at Cornell University. At first, I felt nervous about oral defense, but I have come to value the one-on-one time with the instructor. This has helped her avoid getting lost in large classes and has given her the skills to articulate the specialized knowledge she needs on the job.

“Having a live check-in creates accountability,” Piserchia says. “It’s much harder to look someone in the eye and say out loud, ‘I don’t know this,’ and then realize, ‘I should study this.'”

___

Associated Press education coverage receives funding from private foundations. AP is solely responsible for all content. Learn about AP’s standards for working with philanthropy, a list of supporters, and funded areas at AP.org.