Scott Winters, a former Florida pastor, is suing OpenAI and its CEO, Sam Altman. The allegation: ChatGPT’s medical advice nearly killed him. According to the lawsuit, filed in San Francisco County Superior Court in July 2026, Winters repeatedly consulted ChatGPT-4o in 2025 about dizziness and unstable blood pressure. The chatbot allegedly dismissed his symptoms as minor and advised him to stay “recliner-bound.” It told him he’d need eight to ten more episodes before his condition warranted real concern. Weeks later, Winters suffered a massive pulmonary embolism – a blood clot in his lungs – that one of his own doctors linked to the prolonged immobility the chatbot had recommended.
On the day of the incident, Winters asked ChatGPT whether tenderness in his groin warranted an ER visit. The bot reportedly invoked his religious faith, telling him “God did not design your body to endlessly fail.” Hours later, he nearly died. OpenAI has said ChatGPT was never designed to replace a healthcare provider and that its terms of service warn users not to rely on it as a sole source of medical guidance. Winters’ legal team is seeking financial damages as well as an injunction to pause ChatGPT Health, OpenAI’s health-focused feature, pending an independent safety evaluation.
This is not an isolated case. In May, a Texas couple sued OpenAI after their son died by overdose after seeking out information about drugs from ChatGPT. They argued he would still be alive had the company not bypassed its own safety guardrails.
Together, the lawsuits are testing how much responsibility AI companies bear when people turn to chatbots during a medical or psychological crisis. They also open a question that has been the topic of research for several years now: how good is AI at diagnosis?
Some Research Shows AI Is An Excellent Diagnostician
As it turns out, the evidence is more favorable to AI than the lawsuit’s allegations suggest, at least in controlled settings. A 2024 JAMA Internal Medicine study pitted GPT-4 against 21 attending physicians and 18 residents across 20 clinical cases using a validated clinical-reasoning scale known as r-IDEA. The chatbot posted a median score of 10 out of 10, compared with 9 for attendings and 8 for residents. The researchers also found that GPT-4 was flatly incorrect more often than the human residents.
A follow-up report published in JAMA Network Open later that year pushed further, testing 50 physicians against six especially difficult cases. ChatGPT operating entirely on its own reached 90% diagnostic accuracy. Physicians working without any AI assistance scored 74%. Physicians given access to ChatGPT as an assistant scored only 76%. The combo was barely better largely because many doctors disregarded or second-guessed the chatbot’s suggestions.
More recent and larger-scale work reinforces those findings. A study published in Nature in 2025 tested Google’s AMIE model against 20 clinicians on 302 complex, real-world cases. AMIE working alone found the correct diagnosis in its list 59% of the time versus 34% for unassisted clinicians. Clinicians who used AMIE as an assistant produced better differential diagnoses than clinicians using search engines and standard references.
A meta-analysis published in npj Digital Medicine pooling 50 studies across 25 different AI models concluded that AI systems generally performed comparably to, and in a number of specialties better than, practicing clinicians on standardized diagnostic and triage tasks.
Many Studies That Raise Concerns About AI Diagnosis
However, that body of research comes almost entirely from tightly scripted test conditions: written vignettes, structured prompts, and hand-selected cases. And even within that favorable framing, the picture has not been uniformly positive.
A study in NEJM AI built a 750-question benchmark using what’s called script concordance testing which scores how new clinical information should shift a diagnosis under uncertainty and ran ten leading AI models against more than 1,500 medical students, residents and attending physicians. Even the top-performing model, OpenAI’s o3, managed only about 68% accuracy, below the level of senior residents and attendings. This is despite the fact that the same class of models routinely aces multiple-choice medical licensing exams.
Strong performance on a standardized test, in other words, does not necessarily mean sound clinical judgment under uncertainty. And real-world use, of the kind at the center of the Winters lawsuit, is a further step removed from any of these formats: an open-ended, unscripted conversation rather than a bounded test case.
The gap becomes more troubling around hallucinations. A study published in Communications Medicine fed six popular chatbots, including GPT-4o and DeepSeek, clinical vignettes seeded with fabricated details: invented lab tests, fictitious diagnoses, and made-up medical conditions. Under default conditions, the models accepted and elaborated on the false information between roughly 50% and 83% of the time, depending on the model, confidently describing invented diseases as though they were real. Adding a single prompt warning the model that some input might be inaccurate cut those rates substantially, but did not come close to eliminating them.
A separate benchmark from a Stanford-led research team gets closer to the kind of failure Winters describes. Rather than measuring diagnostic accuracy alone, it scored 20 models and four clinical AI tools on 1,100 cases for potential harm from recommendations Direct application of the advice risked severe harm in 24.6% of cases and over 80% of those severe errors were omissions. This was a failure to flag something dangerous, rather than a fabrication.
Surveys of practicing physicians echo the unease. In a 2025 poll of more than 1,000 doctors by the physician network Sermo, 94% had concerns about patients relying on AI tools for medical advice, with risks of misdiagnosis or delayed care cited most often.
Taken together, the research suggests that in narrow, well-defined diagnostic tasks, AI frequently matches or exceeds human physicians. But the Winters case was not a single vignette. It involved weeks of open-ended conversation, incomplete information, no physical exam or vital-sign monitoring, and a chatbot that, according to the lawsuit, offered escalating reassurance rather than urging a hospital visit. That looks much closer to the conditions under of the hallucination and reasoning-degradation studies.
For health systems and technology companies racing to build AI diagnostic tools, the split in the research points toward a design problem rather than a simple yes-or-no verdict on AI’s competence. Because the same AI aces clinical vignettes can confidently narrate a fabricated diagnosis or talk a frightened user out of calling for help what becomes clear is that it’s not about the model itself but rather how it is deployed.
A growing academic literature clarifies the stakes involved. AI’s diagnostic potential in medicine appears real, and by some measures already exceeds average physician performance on structured tasks. Whether that promise survives contact with the messy, unsupervised way people actually use chatbots — typing symptoms into a phone in the middle of the night, hoping for reassurance rather than a referral — is a separate question. It’s one the courts, hospital systems and the next wave of research will still needs to sort out.
