Researchers explore foundational model for generalist medical artificial intelligence

Foundation models can be applied to various downstream tasks after being trained on large and diverse datasets. From textual questions that respond to visual descriptions and game play, individual models are now able to achieve state-of-the-art performance. Growing data sets, larger models, and improved model architecture are creating new possibilities for underlying models.

Due to the complexity of healthcare, the difficulty of collecting large and diverse medical information, and the novelty of this discovery, these models have yet to penetrate medical AI. Most medical AI models use task-specific model building techniques. To train a model to analyze chest x-rays and detect pneumonia, the photos must be manually labeled. If this algorithm detects pneumonia, a human should produce a radiation report. This highly focused, label-driven methodology produces a rigid model that can only perform tasks in the training dataset. Such models may need to be retrained on new datasets to adapt to new tasks and data distributions for the same goal.

Developments such as multimodal architectures, self-supervised learning techniques, and in-context learning capabilities have enabled a new class of sophisticated medical-infrastructure models called GMAI. Their “generalist” label suggests replacing more specialized models for specific medical tasks.

Researchers at Stanford University, Harvard University, University of Toronto, Yale University School of Medicine, and the Scripps Research Translational Institute have identified three essential properties that distinguish GMAI models from traditional medical AI models.

1. The GMAI model can be easily adapted to new tasks by simply writing the work in English (or another language). A model can address new challenges after it has been introduced into the model (dynamic task specification) and before it needs retraining.
2. GMAI models can ingest data from a variety of sources and produce results in a variety of formats. GMAI models explicitly reflect medical knowledge, allow reasoning through novel challenges, and communicate results in language that medical professionals can understand. Compared to existing medical AI models, GMAI models may be able to tackle a wide variety of tasks with fewer or no labels. His two distinctive features of GMAI (support for different combinations of data modalities and the ability to perform dynamically set tasks) allow GMAI models to engage with users in a variety of ways.
3. A GMAI model should explicitly represent knowledge in the medical domain and use it for advanced medical reasoning.

By allowing users to interact with models via bespoke queries, GMAI offers greater adaptability across jobs and situations, making AI insights accessible to a wider range of consumers. A user may use a custom his query to generate a query such as “describe the mass appearing in this head MRI scan”. Is it likely a tumor or an abscess?”

Two important features, dynamic task specification and multimodal input and output, are enabled by user-defined queries.

1. Dynamic task specification: Artificial intelligence models can be retrained on-the-fly with custom queries to learn how to deal with new challenges. When asked, “What is the thickness of the gallbladder wall obtained by this ultrasound?” GMAI can provide unprecedented answers. Thanks to in-context learning, GMAI could be trained on new concepts in just a few cases.
2. Multimodal Inputs and Outputs: Custom queries allow arbitrary combinations of modalities for complex medical problems. When seeking a diagnosis, doctors can attach multiple photos and lab reports to the query. If a customer requests a text response and accompanying visualization, the GMAI model can easily accommodate both requests.

Here are some examples of how GMAI can be used.

• Radiological Discovery You Can Trust: GMAI paves the way for a new class of flexible digital radiology assistants that assist radiologists at every stage of the process, significantly reducing their workload. Radiology reports, including both abnormal and appropriate normal results and taking into account the patient’s medical history, can be automatically generated by the GMAI model. Combined with text reports, interactive visualizations from these models can be very helpful for physicians, for example by highlighting areas specified in each phrase.
• Enhanced Surgical Methods: With the GMAI model, surgical teams are expected to perform treatments more easily. GMAI models may perform visualization tasks such as annotating live video feeds of operations. When a surgeon discovers an abnormal anatomic event, he or she may communicate the information verbally by sounding an alarm or reading the relevant literature aloud.
• Helps you make tough calls at your bedside. A GMAI-enabled bedside clinical decision support tool that builds on existing AI-based early warning systems to enable more detailed explanations and recommendations for future care.
• Text-to-protein creation: GMAI synthesized protein amino acid sequences and three-dimensional structures from text input. This model may be contingent on producing protein sequences with desirable functional characteristics, such as those found in existing generative models.
• Joint note-taking. The GMAI model automatically drafts documents such as electronic memos and hospital discharge reports. Doctors only need to inspect, update, and approve them.
• medical chatbot. A new patient assistance app will be powered by GMAI to help deliver high-quality care outside of the clinical setting.

check out paper and Reference articledon’t forget to join Our 19k+ ML SubReddit, cacophony channeland email newsletterWe share the latest AI research news, cool AI projects, and more. If you have any questions about the article above or missed something, feel free to email me. Asif@marktechpost.com

🚀 Check out 100 AI Tools in the AI ​​Tools Club