Microsoft's AI Panel Outperforms Generalists on Complex Diagnoses

For patients with complex medical conditions, a new diagnostic system from Microsoft suggests that artificial intelligence is beginning to close the gap with human expertise in specific scenarios. A study published in Science on Thursday reports that Microsoft’s multi-agent AI system, MAI-DxO, correctly diagnosed difficult cases in a simulation more than twice as often as generalist physicians. While this does not replace doctors immediately, it signals a shift in how medical institutions might leverage AI to manage the growing strain on clinical resources.

The core of this development is the architecture of MAI-DxO, which functions as a coordinated panel of AI agents rather than a single model. These agents simulate a team of specialists, questioning the patient, ordering tests, and debating potential diagnoses. In simulations using complex cases from the New England Journal of Medicine, the system achieved an 83% accuracy rate. By comparison, primary care doctors and general internists working individually with reference materials were correct only 36% of the time. It is important to note that these complex cases are typically handled by specialists or teams in reality, so the comparison highlights the system's ability to aggregate diverse medical reasoning.

The performance gains are driven largely by the underlying AI models, with Microsoft’s orchestration adding incremental improvements. Testing revealed that the system performed best with OpenAI’s GPT-5 and GPT-5.5 models. For complex cases, GPT-5.5 saw its accuracy jump from 72% to nearly 87% when assisted by the DxO framework, while also reducing costs. However, the benefits were less pronounced for emergency department cases, where the AI’s performance remained similar whether or not the orchestration layer was active. This suggests the technology is currently best suited for intricate diagnostic puzzles rather than acute, high-speed emergency triage.

Microsoft is moving quickly to validate these findings in real-world settings. Through a partnership with Beth Israel Deaconess Medical Center in Boston, the company will evaluate DxO’s clinical reasoning across various hospital environments. Patients will receive standard care while consenting participants are also assessed by the AI, allowing researchers to compare recommendations directly. This trial aims to provide compelling evidence on the system's scalability, mirroring similar recent efforts by Google to test diagnostic AI in urgent care settings.

Mustafa Suleyman, CEO of Microsoft AI, frames this work as a step toward “medical superintelligence,” defined as the capacity to surpass contemporary human-led health systems in reasoning and coordination. He expects healthcare models to become an order of magnitude more capable within the next 24 to 36 months, driven by a “flywheel” of real-world usage and iteration. With a foundation model for healthcare planned for release next year in collaboration with the Mayo Clinic, Microsoft aims to integrate AI deeply into clinical workflows, moving beyond simple task automation to comprehensive decision support.