A featured artificial intelligence (AI) tool, ChatGPT, was shown to be more creative than 99% of college students who took the same prestigious creativity test, researchers report.
Previous tests using early versions of commonly known AI have not been as creative as human AI. But these new results with GPT-4 show that AI may be closing the creativity gap sooner than expected.
Fear that AI will replace human workers generally did not affect creative thinkers
Ever since the announcement of AI tools like ChatGPT showed the world how advanced some of these systems have come, fears have risen that AI will replace human workers in many fields. It is rising. Some reports indicate that such things are already happening in areas such as advertising and coding, but even the most dire predictions of job losses due to AI are likely to continue, at least in the foreseeable future. has been speculated that the realm of human creativity is likely the safest.
According to the researchers behind this latest discovery, that was also the case a year ago, with early versions of ChatGPT failing to compete with many humans in creative thinking.
“We learned of a previous study on GPT-3 that was done a year ago,” said Dr. Eric Guzik, clinical assistant professor and lead researcher at the University of Montana’s School of Business. “At the time, ChatGPT did not score as well as humans on tasks involving independent thinking.”
Still, Guzik and his colleagues Christian Gilde of UM Western and Christian Byrge of Vilnius University, after trying ChatGPT for a year, believe that the latest version may have enhanced creativity enough to stimulate human thinking. He said he was fascinated. their money.
“We were all investigating with ChatGPT and realized it was doing some interesting things that we didn’t expect,” Guzik said. “Some of the responses were novel and surprising. That’s when we decided to test how creative it really was.”
Standard test that measures human creative thinking yields surprising results
To see if the latest AI tools are really getting more creative, Guzik and his research team decided to test it. literally.
First, they used a tool known as the Torrance Test for Creative Thinking (TTCT). The TTCT, which has been used for decades to measure human creativity, prompts subjects with a series of questions designed to determine how creative they are.
“Let’s say it’s basketball,” Guzik explains about the kinds of prompts the TTCT might give subjects. “Think of as many uses for basketball as possible.”
Obvious responses to this question might include throwing it into a basketball hoop or playing catch with a friend. But creative thinking humans may provide responses that computers do not.
“If you are forced to think of a new use for it, it might be a good idea to cut it and use it as a planter,” Gujik explains. “You can use bricks to build things or use them as paperweights.
Again, researchers point out that, until recently, AI tools have not performed as well as humans at giving unconventional and creative options for these queries.
After asking ChatGPT a series of eight questions designed to elicit creative solutions on the TTCT test, Guzik administered the same test to 24 human entrepreneurship students. The researchers also collected her 2,700 responses from students on the 2016 national test to significantly increase the comparative dataset.
Guzik and his colleagues then submitted the responses to the Scholastic Testing Service (STS) for scoring. Notably, STS officials were unaware that AI was involved in the trial and treated all subjects as if they were humans. Also importantly, the researchers point out that because TTCT is a protected and proprietary material, ChatGPT cannot cheat by searching online for creative answers to questions.
After STS personnel finished scoring the responses submitted by Guzik and his colleagues, the results were even better than expected.
“AI applications were in the top percentiles for fluency (the ability to generate a large number of ideas) and originality (the ability to come up with new ideas),” the researchers report.
“For ChatGPT and GPT-4, we showed for the first time that they performed in the top 1% for originality,” Guzik said. “It was fresh.”
Notably, the AI scored “only” at the 97th percentile of flexibility. Flexibility is defined as the ability to generate different types and categories of ideas in testing. Still, Guzyk and his team say the numbers are staggering.
AI says humans may need better tools and understanding to assess creativity
One of the interesting results of their research came when Guzik asked ChatGPT what it means to get a good score on the TTCT.
“ChatGPT said we may not fully understand human creativity, and I believe they are right,” Guzik said. “It also suggested that we may need more sophisticated evaluation tools that can distinguish between human ideas and AI-generated ideas.”
Of course, this study did not ask ChatGPT to write a symphony or a multi-volume fantasy-fiction series, but only the foundations of creativity. As a result, this kind of effort may still be unattainable with even the most sophisticated AI tools. Still, the results the team gleaned from their analysis are generally unexpected, and may indicate that AI systems are learning to lose weight like humans, faster than anyone expected. I have.
“I think we know that in the future there will be some form of AI,” Guzik said. “We need to be careful how we use it and consider the necessary rules and regulations. But companies are already using it for many creative tasks. And this is a big game changer.”
Christopher Plain is a science fiction and fantasy novelist and head science writer at The Debrief.follow him and connect twitterfor his book, please visit: plainfiction.comor email him directly christopher@thedebrief.org.
