Study Finds ~20% High-Risk Rate in Real ChatGPT Health Advice, Rising to ~35% with User Context
A first-of-its-kind audit of real ChatGPT health conversations reveals that roughly 20% of responses are rated high-risk, increasing to nearly 35% when user-specific context is considered.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
A large-scale audit of real-world ChatGPT health consultations found that approximately one in five responses received a high-risk rating. When the evaluation incorporated the user's own accumulated context, this share rose by 13 to 17 percentage points, indicating that nearly one in four responses deemed safe in isolation were actually unsafe for the specific individual asking.
Previous safety evaluations of LLM health advice relied almost entirely on clinician-authored vignettes or synthetic prompts, leaving a gap in understanding real-world usage. The study by Shreyasi Roy Chowdhury and Kiran Garimella analyzed 15,555 health conversations from 1,252 users in India, Nigeria, Brazil, and Pakistan between December 2022 and February 2026.
Identified failure modes included unsafe medication or dosage guidance, harmful self-diagnosis, missing safety warnings, and a failure to refer users to professionals. The researchers conclude that safety evaluations built on synthetic cases likely miss these harms.