This research serves as a “stepping stone” for future applications of AI in vascular surgery

Applications of AI



LR: Quang Le and Michael Amendola

A new study on chat generative pre-trained transformer (GPT) technology and its success rate in the Vascular Education and Self-Assessment Program (VESAP) offers insight into the future of artificial intelligence (AI) in vascular surgery training and practice, say researchers Michael Amendola (Richmond, USA) and Quan Le (Charlottesville, USA). Vascular News.

Lu, a medical student, University of Virginia School of Medicine First author The research director said that this project It started with a petition to the Vasu Association.Ophthalmic Surgery (SVS) Self-Evaluation CommitteeThe committee is the fourth The fourth edition of VESAP (VESAP4) will be held in April 2023.

After that, VESAP4 material (name385 non-image questions Ten areas of knowledge in vascular surgery:GPT-3.5-Turbo (GPT 3.5) Large-scale language models. Two independent The judges reviewed the AI-generated answers.We questioned them for accuracy and content and compared them with the key answers provided. Programming Interface (API) requests are Three replicates were performed to assess consistency.

Research is distributed as a modPoster presentation at this year's Southern Conference Society for Vascular Surgery (SAVS) Annual Meeting (January 24-27, Scottsdale, USA) GPT 3.5 is 49.4% of the questions were answered correctly. 77.8% of the correct answers were similar. Across all three queries.

Le reported that GPT 3.5 performed best. For questions about radiation safety, The correct answer rate was 54.4%, the worst result. Answering questions about dialysis access Only 39% of the answers were correct.

Among the questions answered incorrectly, Le said the most common cause is The inaccuracy was the acquisition of incorrect information. or inability to obtain important facts.

The team will conduct further research and present a poster in 2024. Vascular Annual Meeting (VAM; June 19-22, Chicago, USA) GPT 3.5 had an accuracy of about 48%. The corresponding figure in the later IteraModel GPT 4 had an accuracy of about 63%. But the researchers also Limited consistency, GPT 3.5 only 55% consistency across 3 cuesGPT4 was consistent across 90% of trials. The answer.

“Unfortunately, the industry The updates had mixed effects,” Le He added that the accuracy The time between the reThe leases for these two models (June 2023 and November 2023) are consistent. It was 65% in GPT 3.5, but dropped to 79% in GPT. 4 during this period.

Mr. Amendola, professor of surgery at Ville HospitalZinnia Commonwealth University Head of Medical and Vascular DepartmentHe underwent eye surgery at the Central Virginia VA Healthcare System in Richmond, Virginia. As the study's lead author, Key lessons learned from this project:The notable finding here is [ChatGPT] It wasn't as good as I thought it would be.”

assignment

This comment,The debate about the future of AI — umBrera terminology varies From large-scale language models to technology Machine learning algorithms like ChatGPTRhythm in Vascular Surgery. Amendola And Le said, Needs to be addressed at this early stage Developing before wider adoption With training and practice you will be successful.

“Much of this technology is experimental. At the moment,” Le noted. The application is currentlyIt varies by institution and country.

Both Le and Amendola emphasized the rules.One of the constraints is the integration of tools such as AI. Large-scale language models continue to Major legislative challenges “There's a lot of consideration as well as integration into existing medical systems and medical record systems,” Le said, with Amendola adding: Privacy and Health Insurance Portability Accountability Law (HIPPA) Concerns “It limits the penetration of these models into large health systems.”

But perhaps the biggest drawback right now, according to Le, is The aforementioned research produces “hallucinations.” He explained: “Our large-scale language models can sometimes produce information in ways that could be harmful if used in a clinical setting.”

Amendola also said: Early AI on Vascular Training and StressEducational institutionsDiscuss ways to handle usage effectively There is growing interest in AI among students. Your own AI-based position or persona“What statement do you put on your application?” he asked. It highlights an important question at the heart of this conundrum.

Data is also an issue, Le said. Large language models are affected The training data contains “hidden He also pointed out that the prediction Machine learning modeling requires large amounts of clean data, which is often “Unfortunately,” he emphasized, “it's not available.” “Data collection during clinical care is The quality is low for various reasons. Much information is missing or poorly documented, which reduces the strength of such predictive models.”

opportunity

Overall, though, both researchers expressed cautious optimism. The potential of AI technologyIn the field of vascular surgery On the future: “Our research willThese new large Language models are just a stepping stone Towards these innovative applications “Tools,” Le said, and his and Amendola's Contextual exploration.

Machine learning offers great potentialLe noted that there is room to individualize care. He believes that AI in general Vascular Practice and ED EfficiencyCiting rapid drafting of education and health care billscal documentation and distillation Take historical medical data as an example. The language model is Simulation of realistic clinical situations for training purposes.

“We are currently using this technology Many other parts of us Live and ultimately Become part of What we see at the bedside and in our practice,” Amendola asserted.

But in addition to this possibility, Amendola Emphasised the need For safety measures, please see ” There is promise and opportunity, A lot of policies are needed, and we Put up guardrails, These models can be seen as follows:

Finally, Mr. Le commented on the introduction of AI at the doctor level:Modern and user-friendly Large language models will become more prevalent and the learning curve will shorten over time. “Overall, these technologiesAs organizations develop, barriers to entry and The learning curve will continue to decrease.” He commented.

“That will help,” Amendola said in closing.This statement is cited as one of his key messages. The fact that it's not a question of “if” but “when” With the introduction of AI, Please embrace this new skill set. The best quote I've heard about AI is AI will not replace doctors and surgeons, but Do not use surgeons or doctors AI will be replaced.”



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *