By Carrie Simek, UM News Service
Missoula – New research from the University of Montana and its partners suggests that artificial intelligence can rival the top 1% of human thinkers on standardized tests of creativity.
The study was directed by Dr. Eric Guzik, Clinical Assistant Professor at UM’s College of Business. He and his partner used the Torrance Creative Thinking Test, a well-known tool that has been used for decades to assess human creativity.
The researchers submitted eight responses generated by ChatGPT, an application powered by the GPT-4 artificial intelligence engine. They also submitted responses from a control group of 24 of her UM students taking Guzik’s Entrepreneurship and Personal Finance classes. These scores were compared to 2,700 college students nationwide who took her TTCT in 2016. All submissions were scored by the Scholastic Testing Service, which was unaware of AI involvement.
As a result, ChatGPT was ranked among the elite companies in terms of creativity. AI applications entered the top percentile for fluency (the ability to generate a large number of ideas) and originality (the ability to come up with new ideas). AI has moved down a bit to the 97th percentile to gain flexibility, the ability to generate different types and categories of ideas.
“For ChatGPT and GPT-4, we showed for the first time that they performed in the top 1% for originality,” Guzik said. “It was fresh.”
He was happy that some of UM’s students were in the top 1%. However, ChatGTP outperformed the majority of college students nationally.
Guzik tested the AI and his students in the spring semester. His work was supported by Christian Gilde of UM Western and his Christian Byrge of Vilnius University. The researchers presented their findings at the Southern Oregon University Creativity Conference in May.
“We were very careful not to interpret the data too much at the meeting,” Guzyk said. “We have only published our results. But we have shared strong evidence that AI appears to be developing creative capabilities that match or exceed those of humans.” .”
Guzik said he asked what ChatGPT would indicate if it worked well with TTCT. AI came up with a powerful answer and shared it at the conference.
“ChatGPT said we may not fully understand human creativity, and I believe they are right,” he said. “It also suggested that we may need more sophisticated evaluation tools that can distinguish between human ideas and AI-generated ideas.”
He said that because TTCT is a protected proprietary material, ChatGPT cannot access information about the test on the internet or public databases to “cheating”.
Guzik has long been interested in creativity. As a seventh grader growing up in the small town of Palmer, Massachusetts, he was in a program for gifted students. That experience brought him to the future problem-solving process developed by pioneering psychologist Ellis Paul Torrance, who was also the creator of his TTCT. Guzyk said that was when he became obsessed with brainstorming and how it taps into human imagination. He is still actively involved in the Future Problem Solving Organization and has met his wife at the conference.
Guzik and his team decided to put ChatGPT’s creativity to the test after experimenting for the past year.
“We were all investigating with ChatGPT and found it doing some interesting things that we didn’t expect,” he said. “Some of the responses were novel and surprising. That’s when we decided to test how creative it really was.”
Guzik said the TTCT test uses prompts that mimic real-world creative work. For example, can you come up with new uses for the product or improve this product?
“Let’s say it’s basketball,” he said. “Think of as many uses for the basketball as you can. You can shoot it into a hoop and use it for display. Or you could use it as a paperweight, but maybe you could pulverize it and make it into something completely new.”
Guzik had some hopes that ChatGPT would be good at generating a lot of ideas (fluency). Because that’s what generative AI does. And they excelled at responding to prompts with many ideas that were relevant, useful, and valuable in the evaluator’s eyes.
He was even more amazed that it worked so well in generating original ideas, a hallmark of the human imagination. The test evaluator is given a list of common responses to the prompt, the responses most likely to be expected to be submitted. However, the AI ranked in the top percentile for coming up with fresh responses.
“At the conference, we learned about previous work on GPT-3 that was done a year ago,” Guzik said. “At the time, ChatGPT did not score as well as humans on tasks that required independent thinking. It contains.”
As advances in AI accelerate, he expects AI to become an important tool for the future business world and a significant new driver of regional and national innovation.
“For me, creativity is doing things differently,” Guzyk said. “One of my favorite definitions of entrepreneurship is that being an entrepreneur is about thinking differently. So AI is transforming the world of creative thinking into business and innovation. process, which is very interesting to me.”
He said the UM College of Business is open to teaching AI and incorporating it into its courses.
“I think we know that in the future there will be some form of AI,” Guzik said. “We need to be careful how we use it and consider the necessary rules and regulations. But companies are already using it for many creative tasks. And this is a big game changer.”
###
contactBy: Erik Guzik, Clinical Assistant Professor of Management and Entrepreneurship, UM College of Business, Phone 406-243-6053, erik.guzik@mso.umt.edu.
