Microsoft claims to have built an AI tool called Diagnostic Orchestrator that can diagnose medical conditions up to four times more accurately than human physicians. In an interview with Robert Hart at The Verge, Microsoft AI CEO Mustafa Suleyman said that after years of industry promises about healthcare AI, “it is finally here.”
The company’s findings, published on October 22 in the journal Science, evaluate the AI system against human doctors across scenarios pulled from emergency department records and medical case reports. Microsoft gathered those cases into an internal benchmark named SDBench, which the company plans to make open-source.
How Diagnostic Orchestrator works and performs
Built to operate like a panel of medical specialists, the Diagnostic Orchestrator relies on multiple AI agents working together to mimic how physicians investigate cases. The agents analyze medical histories and symptoms, choose which diagnostic tests to request, and settle on a final diagnosis.
In evaluations using hundreds of case reports from The New England Journal of Medicine, Microsoft’s tool recorded roughly 83 percent diagnostic accuracy. By comparison, generalist doctors scored around 36 percent when allowed to consult reference materials, and 22 percent when working without them.
The tool also beat human doctors on emergency room cases taken from a large patient admissions database at Boston’s Beth Israel Deaconess Medical Center, reaching 80 percent accuracy against the physicians’ 47 percent average. According to the research, the AI system also tended to arrive at its diagnoses at a lower estimated cost than the human doctors.
Powered by OpenAI’s GPT-5, the multi-agent system also beat standalone frontier AI models, according to the researchers. However, the peer-reviewed study does not test newer models such as OpenAI’s GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon, or Anthropic’s Opus 5.5.
Microsoft originally built the Diagnostic Orchestrator when GPT-4.1 was the leading model available. The research team states that follow-up testing indicates that “improvements in model capability do not necessarily diminish the benefits” of using multiple collaborative agents.
Clinical limitations and upcoming hospital trials
Microsoft notes that significant work remains before the system can see broad deployment. Although the benchmark scenarios reflect real diagnostic workflows - asking questions, weighing possible explanations, and ordering tests before diagnosing - the tool has only been evaluated on historical medical cases, leaving its performance in live clinical settings unproven.
It needs clinical validation. That is a leap, but this is a major milestone along that journey.
Mustafa Suleyman, AI CEO, Microsoft
Study coauthor Dr. Dominic King, who leads AI health at Microsoft, told The Verge that the company plans to start testing the tool inside hospital environments with the Beth Israel Deaconess Medical Center to evaluate its real-world clinical performance. While neither Suleyman nor King gave a firm schedule, King noted the trials will likely “get underway in the next couple of months.”
Microsoft has not announced how or when it might commercialize the technology, though it already maintains a large footprint in US healthcare through its digital tools, and health queries remain one of the top uses for its consumer Copilot chatbot. King stressed that any rollout must be handled responsibly given the strict compliance and regulatory standards in medicine.
Microsoft’s push for medical superintelligence
The study fits into Microsoft’s wider goal of building what it calls “medical superintelligence” - AI capable of outperforming the best human-run healthcare systems. As The Verge notes, Microsoft used the term “superintelligence” before Donald Trump and his administration adopted it as a general label for AI.
According to King, diagnosis represents only one piece of the company’s healthcare roadmap. In an October 22 blog post and an upcoming paper in the journal BMJ Digital Health & AI, Microsoft details five additional areas it believes AI must master, including coordinating clinical care, making treatment decisions, and managing the human side of medicine.
Suleyman predicted that “next year we’re going to see extremely impressive deployments of intelligent systems in production,” adding that he believes it will significantly change how care is delivered. Even while forecasting “something that is indistinguishable from human performance on medical superintelligence in the next couple of years,” Suleyman maintained that human doctors should and will always keep a central role in caring for patients.
While acknowledging that AI could end up giving physicians more work, Suleyman said he hopes the technology will remove some of “the drudgery” from medical practice so clinicians can spend more time with patients.
This article was written by AI from reporting by theverge.com.
