Microsoft is revealing details of artificial intelligence systems that perform better than human doctors in complex health checks, creating a “path to medical closeness.”
The company's AI unit, led by British engineer Mustafa Suleyman, has developed a system that mimics a panel of specialist doctors working on “diagnostically complex and intellectually demanding” cases.
When paired with Openai's advanced O3 AI model, Microsoft said that its approach “solved” more than eight of the 10 case studies specifically selected for the diagnostic challenge. When these case studies were attempted for practice physicians who had no access to co-workers, textbooks, or chatbots, the accuracy rate was 2 out of 10.
Microsoft said it was also a cheaper option than using a human doctor because it was more efficient to order tests.
Despite highlighting potential cost savings from the study, Microsoft reduced the impact of the work by stating that it believes AI will complement the role of doctors rather than replace them.
“Their clinical roles are much broader than simply making a diagnosis. They need to navigate ambiguity in a way that doesn't have anything to set up for AI to do and build trust with patients and their families,” the company wrote in a blog post publishing the research being submitted for peer review.
However, using slogans, “The Road to Overmed Medical” brings the prospect of fundamental change in the healthcare market. Artificial general information (AGI) refers to systems that match human cognitive abilities in a particular task, while superintelligence is an equally theoretical term that refers to systems that exceed the overall human intellectual performance.
Explaining the rationale behind the study, Microsoft questioned the ability of AI to score very well in the US medical licensing exam, an important test for obtaining a medical license in the US. The multiple choice test supports memorizing answers surrounding subjects' deep understanding, saying it could help “exaggerate” the capabilities of AI models.
Microsoft said it is developing a system that, like real-world clinicians, can take step-by-step steps to reach the final diagnosis, such as asking specific questions or requesting diagnostic tests. For example, patients with symptoms of cough or fever may require blood tests and chest x-rays before a doctor arrives to diagnose pneumonia.
The new Microsoft approach uses complex case studies from the New England Journal of Medicine (NEJM).
Suleyman's team converted over 300 of these studies into “interactive case challenges” and used it to test the approach. Microsoft's approach used existing AI models, including those produced by ChatGpt developers Openai, Mark Zuckerberg's Meta, Anthropic, Elon Musk's Grok and Google's Gemini.
Microsoft then used a specific model of what the tests and diagnostics were to order, using AI systems like bespoke agents called “diagnostic orchestrators.” Orchestrators effectively mimic the doctor's panel, which causes the diagnosis.
Microsoft said that when combined with Openai's advanced O3 model, more than eight of the 10 NEJM case studies have been “solved.”
Microsoft said that the approach could span multiple medical disciplines, allowing it to exercise “width and depth” beyond individual physicians.
“Expanding this level of reasoning could potentially restructure healthcare. AI can self-manage patients with routine aspects of care and provide clinicians with sophisticated decision support for complex cases.”
Microsoft has admitted that the work is not ready for clinical use. For example, further testing is required with an “Orchestrator” to assess performance in more common symptoms.
