Systematic literature reviews (SLRs) are resource-intensive and time-consuming, but a necessary part of evidence generation. SLR presents an opportunity for artificial intelligence (AI) to deliver significant resource savings, and several AI tools have been developed to ease the burden on literature review teams.
Here, we examine whether the AI-powered SLR platform EasySLR™ can replace human reviewers to improve accuracy and whether this translates into time and cost savings in the real world.
Replace human reviewers with SLR camera AI
As a basis for comparison, we used economical SLRs in two very similar breast cancer populations. We tested the “AI as Reviewer” feature, which replaces a single human reviewer in the SLR process. The platform’s transparency meant that the AI’s decisions and speed could be examined and compared to those of human reviewers in several areas, including abstract and full-text screening, data extraction, and the development of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). To measure accuracy, we used the human reviewer’s final decision as the “correct” decision against which the AI’s decision was compared. Finally, we evaluated the time and cost investment by comparing the time required and associated costs for projects with and without EasySLR™.
AI-powered analysis shows human-like accuracy and saves up to 40% time on SLR tasks
The AI platform outperformed humans in speed on all tasks (Figure 1). In terms of accuracy, EasySLR™ produced similar results to humans in PRISMA generation and abstract screening. For full-text screening and data extraction, platform accuracy was up to 10% lower than human accuracy (Figure 1). This is because AI relies heavily on identifying precise wording from population, intervention, comparison, and outcome (PICO) criteria and has struggled to extract complex data such as assumptions and costs in some economic models (Figure 1).
Figure 1. Accuracy and speed comparison between human reviewers and the AI-powered platform EasySLR™ at each step of the SLR process.
Real-world cost savings of up to 30% are possible, but there are caveats
The use of an AI platform saves significant time at each step, potentially leading to cost savings of up to ~30% compared to a fully manual process across PRISMA development, screening, and extraction. AI is especially cost-effective when you need to interrogate subpopulations or tweak PICO and rerun searches, which would otherwise be prohibitively time-consuming for SLR projects.
However, we found that time and cost savings can vary depending on the type of project (systematic vs. targeted review), the complexity of the SLR questions, and the human reviewers’ familiarity with the AI prompts.
Some of this complexity could be alleviated by using AI tools to improve prompts, optimize protocols, and train AI in human decision-making. Such optimization tools can further enhance the time savings provided by AI SLR, which could lead to further cost savings as the understanding of AI improves.
conclusion
In conclusion, the AI-powered platform EasySLR™ can replace human reviewers and save up to 40% of the time and 30% of the cost required to conduct SLR. Although leveraging AI in SLR cameras can significantly reduce the required human workload, limitations still exist, such as reduced AI accuracy. To ensure that the time savings achieved translate into cost savings and to ensure that these improvements do not plateau, all AI SLR tools need new capabilities that improve AI understanding.
For more information, please contact Manca Povsic at manca.povsic@amiculum.biz or visit our Insights page for more content.
