Whether you search for cooking tips on Reddit, troubleshoot technical issues on community forums, or ask questions on platforms like Quora, you've benefited from the online help community. These digital spaces rely on people all over the world to provide free knowledge, and have become essential tools for solving problems and learning new skills.
New research reveals that generative artificial intelligence tools like ChatGPT can create double edge effects for users in these communities, and at the same time overwhelm them to reduce responses while making them even more useful.
“On the positive side, AI helps users learn to write more organized and easier to read answers, and the number of responses increases significantly,” he explained. Liangfei Qiu, Ph.D. , Researcher at the University of Florida Warrington University College of Business and Professor Pricewaterhousecoopers. “However, if users rely heavily on AI, the mental effort required to process and improve the output of AI can actually reduce participation. This means that AI can empower and burden contributors.
This study investigated Stack Overflow, one of the world's largest Q&A platforms for computer programmers, to investigate the impact of generated AI on both the quality and quantity of user contributions. Qiu and his co-author Guohou Shan of D'Amore-Mckim School of Business at Northeastern University measured the impact of AI on user count per day, response length and readability.
Specifically, we found that users who used AI tools to generate responses provided almost 17% more responses per day compared to users who did not use AI. Both AI-generated responses were about 23% shorter, making them easier to read.
However, participation has decreased as people relied heavily on AI tools. Qiu and Shan noted that the additional cognitive burden associated with the use of heavier AI negatively affected the impact on user response quality.
For the online help community working on AI policies, this study provides valuable insight into how these policies can be updated in the current AI environment. While some communities like Stack Overflow have banned AI tools, this study suggests that a more nuanced approach could be a better solution.
Instead of banning AI entirely, researchers suggest that they balance it with allowing AI to use it while encouraging responsible use. They argue that this approach does not undermine high-quality content and user perception, but can benefit users from efficiency and learning opportunities.
“For platform leaders, takeout is clear. AI can encourage participation when thoughtfully integrated, but its cognitive demands must be managed to maintain long-term user contributions,” Qiu said.
