Microsoft AI claims its multi-agent diagnostic system outperforms doctors on historical cases, pending clinical validation

Mustafa Suleyman, CEO of Microsoft AI, told The Verge that the company has developed an AI tool called Diagnostic Orchestrator that can diagnose conditions up to four times more accurately than human doctors in simulated tests. The findings were published on October 22, 2026, in the prestigious journal Science.

Diagnostic Orchestrator is a multi-agent system designed to mimic the process of a panel of medical experts: multiple AI agents reason through symptoms and medical histories, decide which tests to order, and arrive at a diagnosis. Microsoft compiled these cases into its own benchmark, SDBench, which it plans to open-source.

According to the paper, on cases drawn from hundreds of reports published in The New England Journal of Medicine (NEJM), the system achieved a diagnostic accuracy of around 83 percent, compared with roughly 36 percent for generalist physicians using references and 22 percent for those working without them. In emergency department cases from a large database of patient admissions at Beth Israel Deaconess Medical Center in Boston, the AI scored 80 percent accuracy versus an average of 47 percent for human physicians. The system also tended to reach diagnoses at a lower estimated cost.

Despite the significant data, Microsoft explicitly noted current limitations. Suleyman acknowledged that the tests were based only on past medical cases and the system has not yet run in real clinical environments. “It needs clinical validation,” Suleyman said. “That is a leap, but this is a major milestone along that journey.” Microsoft plans to begin testing the system in hospital settings with Beth Israel Deaconess Medical Center in the coming months.