Microsoft tests AI MAI Diagnostic Orchestrator tool that outperforms doctors in complex medical diagnoses

Microsoft’s MAI Diagnostic Orchestrator significantly outperforms physicians in diagnostic accuracy and cost efficiency across 300+ clinical case studies, though experts urge caution ahead of real-world deployment.

Microsoft has developed a diagnostic system called MAI-DxO (Microsoft AI Diagnostic Orchestrator) that significantly outperforms human doctors in diagnosing complex clinical cases. The tool combines responses from several leading large language models — GPT-4, Gemini, Claude, Llama, Grok, and DeepSeek — to simulate a multi-doctor consultative approach.

The system was evaluated on 304 cases from the New England Journal of Medicine, a set of scenarios typically used for medical education due to their complexity. A panel of 21 experienced physicians from the US and UK was asked to diagnose the same cases, step by step, without access to external tools.

MAI-DxO achieved an 85.5 percent accuracy rate, compared with 20 percent for the physicians. It also reduced diagnostic costs by approximately 20 percent, primarily by selecting fewer and less expensive tests.

Dominic King, vice president at Microsoft, says the system, “performs incredibly well, both getting to the diagnosis and getting to that diagnosis very cost effectively.”

Mustafa Suleyman, CEO of Microsoft AI, describes the system’s design as a breakthrough, arguing that, “this orchestration mechanism—multiple agents that work together in this chain-of-debate style—that’s what’s going to drive us closer to medical superintelligence.”

The approach involves having multiple AI agents reason together through each case, mimicking the behavior of a team of physicians debating diagnoses in real time.

Optimism tempered by caution

The findings have sparked discussion across the medical and AI sectors. Conor Grennan, Chief AI Architect at NYU Stern School of Business, shares his reaction on LinkedIn, calling the results “bonkers” and writing: “Microsoft just announced a breakthrough in medical AI with its new Microsoft AI Diagnostic Orchestrator—outperforming experienced human doctors by a wide margin in diagnosing complex medical cases.”

Grennan highlights the design of the system, noting: “Their new tool, called MAI-DxO, just let the models talk to each other and work together, like a group of doctors would.” He adds that the AI “ordered fewer and less expensive tests, reducing overall diagnostic costs by about 20% compared to the doctors.”

Clinical availability remains distant

MAI-DxO is not yet approved for clinical deployment and remains in a research phase. Microsoft has not confirmed whether it will commercialize the system, though integration into products like Bing or clinical decision-support tools remains under review.

Suleyman says Microsoft plans further trials: “What you’ll see over the next couple of years is us doing more and more work proving these systems out in the real world.”

For now, the system marks a significant development in AI-assisted diagnostics. Whether it can deliver the same results in a hospital setting, where variables like patient communication, ethics, and institutional logistics come into play, remains to be seen.

RTIH AI in Retail Awards

Our sister title, RTIH, organiser of the industry leading RTIH Innovation Awards, proudly brings you the first edition of the RTIH AI in Retail Awards, which is now open for entries. 

As we witness a digital transformation revolution across all channels, AI tools are reshaping the omnichannel game, from personalising customer experiences to optimising inventory, uncovering insights into consumer behaviour, and enhancing the human element of retailers' businesses.

With 2025 set to be the year when AI and especially gen AI shake off the ‘heavily hyped’ tag and become embedded in retail business processes, our newly launched awards celebrate global technology innovation in a fast moving omnichannel world and the resulting benefits for retailers, shoppers and employees.

Our 2025 winners will be those companies who not only recognise the potential of AI, but also make it usable in everyday work - resulting in more efficiency and innovation in all areas.