In a recent study published in International Journal of Educational Integrity, Chinese researchers conducted the first comparison of the accuracy of artificial intelligence (AI)-based content detectors and human reviewers in detecting both original and paraphrased AI-generated rehabilitation-related articles. The researchers found that among the tools offered, Originality.ai detected 100% of the AI-generated texts, professor reviewers correctly identified at least 96% of the AI-paraphrased articles, and student reviewers correctly identified 76% of the AI-paraphrased articles, highlighting the effectiveness of AI detectors and experienced reviewers.
Research: The Great Detective: Humans vs. AI Detectors in Detecting Medical Documents Generated with Large-Scale Language Models. Image Credit: ImageFlow / Shutterstock
background
ChatGPT (short for “Chat Generative Pretrained Transformer”) is a large-scale language model (LLM) chatbot that has been widely used in various fields. In medicine and digital health, this AI tool can be used to perform tasks such as generating discharge summaries, assisting in diagnosis, and providing health information. Despite its usefulness, scientists are opposed to authorship in academic publishing due to concerns about accountability and reliability. AI-generated content can be misleading, so robust detection methods are needed. Existing AI detectors such as Turnitin and Originality.ai show promise, but struggle with paraphrased text and often misclassify human-written articles. Human reviewers have also shown moderate accuracy in detecting AI-generated content. Continued efforts to improve AI detection and develop discipline-specific guidelines are essential to maintain academic integrity. To fill this gap, researchers in this study aimed to examine the accuracy of generic AI content detectors in identifying LLM-generated academic papers and compare them with human reviewers with different levels of research training.
About research
In this study, we selected 50 peer-reviewed papers related to rehabilitation from high-impact journals. Then, artificial research papers were created (they were asked to imitate academic writers) using specific prompts from ChatGPT version 3.5. The resulting articles were rewritten using Wordtune to increase their authenticity. In addition, six AI-based content detectors were used to distinguish between original papers, ChatGPT-generated papers, and AI-rephrased papers. The included tools were available for free (GPTZero, ZeroGPT, Content at Scale, GPT-2 Output Detector) or paid (Originality.ai and Turnitin's AI Writing Detection). Importantly, the detectors did not analyze the methods and results sections of the papers. For analysis and comparison, AI, perplexity, and plagiarism scores were determined. Statistical analysis included the use of Shapiro-Wilk tests, Levene tests, analysis of variance, and paired analysis. t-test.
In addition, four blinded human reviewers, including two undergraduate reviewers and two professor reviewers with physical therapy backgrounds and varying levels of research training, reviewed the original paper and the AI-paraphrased results. I was given the task of reviewing and identifying papers. Reviewers were also surveyed to understand the reasons behind article classification.
Results and discussion
The accuracy of AI content detectors in identifying AI-generated articles was found to vary. Originality.ai showed 100% accuracy in identifying both ChatGPT-generated and AI-paraphrased articles, while ZeroGPT achieved 96% accuracy in identifying ChatGPT-generated articles, with a sensitivity of 98% and specificity of 92%. Additionally, GPT-2 Output Detector and Turnitin showed 96% and 94% accuracy, respectively, for ChatGPT-generated articles, while Turnitin's accuracy for AI-paraphrased articles dropped to 30%. GPTZero and Content at Scale showed lower accuracy in identifying ChatGPT-generated papers, with Content at Scale misclassifying 28% of the original articles. Interestingly, Originality.ai was the only tool that did not assign a lower AI score to paraphrased articles compared to ChatGPT-generated articles.
a The frequency with which each reviewer identified the main reason for an article to be rephrased by artificial intelligence (AI). B Relative frequency of each reason why articles were identified as paraphrased by AI (based on the top three reasons given by four reviewers)
In human reviewer analysis, the median time it took four reviewers to distinguish between the original article and the article paraphrased by AI was 5 minutes and 45 seconds. He observed high correct answer rates of 96% and 100% on insightful AI paraphrased articles by two professor reviewers, who found that his 12% of human-written papers were AI paraphrased. I classified it incorrectly. On the other hand, a student reviewer said he only achieved 76% accuracy in identifying articles that had been paraphrased by the AI. The main reasons why articles are judged to have been paraphrased by AI are: lack of consistency (34.36%), grammatical errors (20.26%), and insufficient evidence-based claims (16.5%). Vocabulary diversity, misuse of abbreviations, and creativity follow. , writing style, ambiguous expressions, and contradictory data. Agreement between professors was observed, with almost perfect agreement on binary answers and fair agreement on identifying primary and secondary reasons.
Furthermore, Turnitin showed that papers generated by ChatGPT and paraphrased with AI had significantly lower plagiarism scores compared to the original papers. We found no significant differences in scores or reviewer ratings for original papers published before and after the release of GPT-3.5-Turbo.
This study is the first to provide valuable and timely insights into the ability of a novel AI detector and human reviewers to identify AI-generated scientific documents (both original and paraphrased). However, our findings are limited by the use of ChatGPT-3.5 (an older version), the possible inclusion of AI-assisted original papers, and the small number of reviewers. Further research is needed to address these limitations and improve generalizability across disciplines.
Conclusion
In conclusion, this study validates the effectiveness of a peer review system to reduce the risk of publishing AI-generated medical content and suggests Originality.ai and ZeroGPT as useful initial screening tools. It highlights the limitations of ChatGPT, calls for continued improvement in AI detection, and emphasizes the need to regulate the use of AI in medical documentation to maintain scientific integrity.
