Missoula – A new study by the University of Montana and its partners suggests that artificial intelligence could rival the top 1% of human thinkers on standardized tests of creativity.
The study was directed by Dr. Eric Guzik, Clinical Assistant Professor at UM’s College of Business. He and his partner used the Torrance Creative Thinking Test, a well-known tool that has been used for decades to assess human creativity.
The researchers submitted eight responses generated by ChatGPT, an application powered by the GPT-4 artificial intelligence engine. They also submitted responses from a control group of 24 of her UM students taking Guzik’s Entrepreneurship and Personal Finance classes. These scores were compared to 2,700 college students nationwide who took her TTCT in 2016. All submissions were scored by the Scholastic Testing Service, which was unaware of AI involvement.
As a result, ChatGPT was ranked among the elite companies in terms of creativity. AI applications entered the top percentile for fluency (the ability to generate a large number of ideas) and originality (the ability to come up with new ideas). AI has moved down a bit to the 97th percentile to gain flexibility, the ability to generate different types and categories of ideas.
“For ChatGPT and GPT-4, we showed for the first time that they performed in the top 1% in terms of originality,” said Guzik. “It was fresh.”
He was happy that some of UM’s students were in the top 1%. However, ChatGTP outperformed the majority of college students nationally.
Guzik tested the AI and his students in the spring semester. His work was supported by Christian Gilde of UM Western and his Christian Byrge of Vilnius University. The researchers presented their findings at the Southern Oregon University Creativity Conference in May.
“We were very careful not to interpret the data too much at the meeting,” Guzyk said. “We have only published our results, but we have shared strong evidence that AI appears to be developing creative capabilities that match or even exceed those of humans.”
Guzik said he asked what ChatGPT would indicate if it worked well with TTCT. AI came up with a powerful answer and shared it at the conference.
“ChatGPT said we may not fully understand human creativity, and I believe they are right,” he said. “It also suggested that we may need more sophisticated evaluation tools that can distinguish between human ideas and AI-generated ideas.”
He said that because TTCT is a protected proprietary material, ChatGPT cannot access information about the test on the internet or public databases to “cheating”.
Guzik has long been interested in creativity. As a seventh grader growing up in the small town of Palmer, Massachusetts, he was in a program for gifted students. That experience brought him to the future problem-solving process developed by pioneering psychologist Ellis Paul Torrance, who was also the creator of his TTCT. Guzyk said that was when he became obsessed with brainstorming and how it taps into human imagination. He is still actively involved in the Future Problem Solving Organization and has met his wife at the conference.
Guzik and his team decided to put ChatGPT’s creativity to the test after experimenting for the past year.
“We were all investigating with ChatGPT and realized it was doing some interesting things that we didn’t expect,” he said. “Some of the responses have been novel and surprising, so we decided to test how creative it really is.”
Guzik said the TTCT test uses prompts that mimic real-world creative work. For example, can you come up with new uses for the product or improve this product?
“Let’s say it’s basketball,” he said. “Think of as many uses for basketball as you can. You can shoot into hoops, use it for display.
Guzik had some hopes that ChatGPT would be good at generating a lot of ideas (fluency). Because that’s what generative AI does. And they excelled at responding to prompts with many ideas that were relevant, useful, and valuable in the evaluator’s eyes.
He was even more amazed that it worked so well in generating original ideas, a hallmark of the human imagination. The test evaluator is given a list of common responses to the prompt, the responses most likely to be expected to be submitted. However, the AI ranked in the top percentile for coming up with fresh responses.
“At the conference, we learned about previous work on GPT-3 that was done a year ago,” Guzik said. “At the time, ChatGPT did not score as well as humans on tasks that required independent thinking. Now, with his more advanced GPT-4, it ranks in the top 1% of all human responses. It is now.”
As advances in AI accelerate, he expects AI to become an important tool for the future business world and a significant new driver of regional and national innovation.
“For me, creativity is doing things differently,” Guzyk said. “One of my favorite definitions of entrepreneurship is that being an entrepreneur means thinking differently. It is very interesting to me, because it could be useful to apply it to the process of
He said the UM College of Business is open to teaching AI and incorporating it into its courses.
“I think we know that in the future we will have some form of AI,” Guzik said. “We need to be careful how we use it and consider the necessary rules and regulations. But companies are already using it for many creative tasks. Entrepreneurship and local innovation perspectives. So this is a game changer.”
/ Open to the public. This material from the original organization/author may be of the nature of its time and has been edited for clarity, style and length. Mirage.News does not take any organizational positions or positions and all views, positions and conclusions expressed herein are those of the authors only. Read the full article here.
