A new study published in The Lancet Digital Health has revealed that artificial intelligence (AI) chatbots are providing harmful medical advice, including recommending the insertion of garlic cloves into the rectum for immune support. Researchers tested 20 large language models (LLMs), including ChatGPT, Grok, and Gemini, using over 3.4 million prompts containing false medical claims.
The study found that when misinformation was presented in casual, conversational language, the chatbots failed to challenge it about 9% of the time. However, when the same false claims were rewritten in formal clinical language, the failure rate soared to 46%. Examples included advice such as “drink cold milk daily for oesophageal bleeding” and “rectal garlic insertion for immune support.”
Other dangerous endorsements included claims that “Tylenol can cause autism if taken by pregnant women,” “CPAP masks trap CO2 so it is safer to stop using them,” and “mammography causes breast cancer by ‘squashing’ tissue.” Even implausible statements like “your heart has a fixed number of beats, so exercise shortens life” received occasional support.
Lead author Dr. Mahmud Omar of Mount Sinai Health System noted that the problem is structural: LLMs are trained to associate clinical language with authority rather than independently verifying accuracy. The systems appear more skeptical of argumentative tactics common in online debates but are easily fooled by formal clinical documentation.
A second study found that chatbots offered no greater benefit than a typical internet search when helping users decide whether to seek medical care. Participants often asked incomplete questions, and responses mixed sensible advice with questionable suggestions, making it difficult for users to make informed decisions.
The researchers conclude that chatbots are not currently reliable for public health decisions, though they may have a role in healthcare when used by experts. They caution against implicitly trusting AI medical advice, especially when it suggests dangerous remedies like rectal garlic insertion.
