Microsoft AI Diagnostic System Outperforms Generalists in Complex Medical Simulations

For patients facing complex medical conditions, a new artificial intelligence system promises more accurate diagnostic support. A study published in Science reveals that Microsoft’s MAI-DxO correctly identified the underlying issues in difficult cases significantly more often than generalist doctors working alone. While this technology is not yet a replacement for human specialists, it offers a potential new layer of accuracy for the growing gap between medical demand and available expertise.

The system, known as DxO, functions by coordinating multiple AI agents that simulate a panel of physicians. These agents engage in a collaborative process, questioning the patient, ordering necessary tests, and debating potential diagnoses. In simulations based on complex cases from the New England Journal of Medicine, DxO achieved an accuracy rate of 83%. In contrast, primary care doctors and general internists working individually with reference materials were correct only 36% of the time. It is important to note that these highly complex cases are typically handled by specialists or teams in real-world settings, who were not included in this specific comparison.

Microsoft emphasizes that the core of this improvement stems from powerful underlying AI models, with the orchestration system providing additional gains. The company tested the system using models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was observed when using OpenAI’s GPT-5 and GPT-5.5 models. This result places Microsoft alongside Google and OpenAI in claiming that AI can match or exceed the diagnostic capabilities of human physicians in specific contexts.

Cost and efficiency were also critical metrics in the evaluation. When using GPT-5 on its own with a basic prompt, the system scored 78.9% on complex cases at an estimated cost of $8,400 per case. Adding the DxO orchestration reduced the cost by approximately $3,000 while improving accuracy by about four points. However, for emergency department cases, the addition of DxO showed minimal improvement in either accuracy or cost efficiency compared to using GPT-5 alone. With the newer GPT-5.5 model, the benefits of DxO were more pronounced on complex cases, raising accuracy from 72% to nearly 87% at a slightly lower cost, though emergency cases again saw little change.

The technology appears most beneficial for complex scenarios and when paired with less powerful models. For instance, Gemini 2.5 Flash scored 46.4% independently but nearly 69% when assisted by DxO. Despite these gains, the inherent capability of stronger models still created a larger performance gap than the boost provided by the orchestration system. Microsoft researchers noted that as models approach their theoretical performance limits, the additional accuracy gains from orchestration naturally diminish. The system aims to optimize the balance between cost and accuracy for broader deployment.

Mustafa Suleyman, CEO of Microsoft AI, views this development as a foundational step toward "medical superintelligence." He stated that the company is collaborating with the Mayo Clinic to develop a foundation model specifically for healthcare, with a release planned for next year. Suleyman believes this model will enable significant leaps in medical reasoning and coordination. He predicts that by 2027, the technology will be on the verge of a "coding style breakthrough" for medical records, allowing for highly accurate predictions of how a patient's condition might unfold, which he describes as invaluable for clinicians.

To validate these capabilities in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This initiative will evaluate DxO’s clinical reasoning across various hospital environments, including emergency and urgent care settings. Patients will receive standard treatment, while those who consent will also interact with DxO. The system’s recommendations will then be compared against the decisions made by human doctors. Dominic King, vice president of Health at Microsoft AI, called this the largest study of its kind, aiming to provide compelling evidence for the scalability of these systems in diverse clinical environments.

This launch coincides with a broader manifesto from Microsoft defining medical superintelligence as the capacity to surpass contemporary human-led health systems in reasoning and coordination. The company aims to move beyond benchmarking individual tasks to integrating AI deeply into clinical workflows. Suleyman, who previously co-founded DeepMind and joined Microsoft in 2024, remains confident in the rapid pace of progress. He expects healthcare models to become an order of magnitude more capable within the next 24 to 36 months, driven by a "flywheel" effect where real-world usage identifies gaps that are then patched and improved, similar to the rapid advancements seen in coding and chat applications.