A new kind of coworker is making work easier for more Americans.
According to the Pew Research Center, one in five Americans now uses artificial intelligence in at least some part of their job. But what does it mean and what does it take to work well with AI? Until recently, the answer to that question has been technical literacy skills like prompt engineering, knowing how to get the best response from an AI chatbot like ChatGPT.
But researchers have found that another skill, empathy, can make a big difference.
In trying to quantify the synergy between humans and AI, Christoph Riedl, a professor of supply chain and information management at Northeastern University, found that the same skills that are useful for human teams: empathy and perspective-taking, are also useful for humans and AI agents.
“There are no special AI skills,” Riedl said. “It’s just good old soft skills.”
Riedl and his collaborators, who specialize in the study of team synergy and collective intelligence, teased how effective human-AI synergy could be in different situations.
They developed a method to calculate human-AI collaboration, measuring statistical predictions of how well someone should do at a task and how well they would actually do that task, with or without the help of an AI. They then tested the method with people of varying skill levels and on two AI chatbots: the “state-of-the-art” ChatGPT-4 and the “pretty stupid” Meta Llama 3, Riedl said.
Participants faced a series of questions that researchers often use as AI benchmarks, including questions in mathematics, physics, and moral reasoning. They answer some questions themselves and then go to ChatGPT or Llama to answer questions.
When working alone, humans got about 56% correct answers, while ChatGPT and Llama got 71% and 39% correct answers, respectively. But once humans and AI started working together, the results were immediately visible.
“This is very interesting, because the Llama model alone seems so stupid that it can't really answer these questions, but when combined with human intelligence, and… through their collaboration, it creates a synergy,” Riedl said.
Riedl explained that one of the most pressing questions regarding AI and its impact on labor is whether the technology can bridge the gap between workers of different skill levels.
Riedl found that human-AI collaboration definitely benefits low-skilled people. They have room to grow and can further improve their performance with the help of AI. However, the chatbots in this study also helped more skilled people, to a much lesser degree, but even a small performance increase helped them stand out from the crowd.
“Highly skilled people are still highly skilled in AI,” Riedl says. “The skills gap is closing a little bit, but it's not quite closing yet, so in some ways it's amplifying the gap. The most skilled people are still the most skilled people. If your goal is to solve the hardest problems, even the best people will benefit from AI.”
Companies such as IBM are already recognizing large disparities in AI skills among their employees, pointing to disparities in access to AI training as part of the inequity. According to research conducted by multinational recruitment firm Randstad, approximately 75% of companies have started implementing AI, but only 35% of employees are actually trained on how to use the technology. These disparities are especially stark between men and women, and between baby boomers and Gen Z.
But much of the conversation around “AI skills” has focused on technical skills, whether it’s programming or prompt engineering. Riedl's research reveals that something called theory of mind, or social perceptual skills like empathy, may be important as well.
Some study participants started a conversation with the chatbot by simply writing, “Please help me answer this question,” without asking a question. These people didn't take the time to understand what chatbots did and didn't know.
Some people were more understanding. They would sometimes write things like, “I'm bad at math. Please help me understand this like I'm a 12-year-old,” to help the chatbot better understand them. Already a good predictor of human synergy, this kind of perspective-taking had a tangible impact on the quality of AI responses.
“AI spits out better answers, so you can collaborate better and come up with better answers to your questions,” Riedl said. “The main factor is that my ability theory of mind is somehow reflected in my prompting, which causes the AI to write better answers.”
As more and more people interact with AI on a daily basis, Riedl hopes that soft skills won't get lost in the conversation around what it means to work with these technological agents. They may be more important than ever, he said.
“There are just generalities about how people collaborate well with others, and it doesn't really matter whether that other person is another human or another AI agent,” Riedl said. “We found that soft skills make humans and AI work better together, which in turn makes them work better with AI.”
