Doctors are testing how AI chatbots could be used to help diagnose patients before they are seen by their GP. A small study is being carried out in the United States to see if the technology could be used to help family doctors prepare for patient visits.
Study details and findings
Researchers have looked at online conversations from 98 patients at a GP clinic in Boston, who were asked to talk to an AI chatbot for up to five days before their scheduled appointment. The chatbot gave each patient a list of potential diagnoses and, in some cases, next steps to discuss at their upcoming appointments.
Doctors involved in the study monitored the conversations in real time and were also given the chat transcript to read before the appointment with the patient. The study, published in The Lancet, found that AI's suggested diagnoses matched the eventual confirmed diagnosis in more than half of cases, and correct diagnosis also appeared when the chatbots' top three suggestions were taken into account in 75 per cent of cases.
Safety and limitations
Researchers said: "Although further research is needed, this study shows the initial feasibility of conversational AI in a real-world setting – assessed via conversation safety and quality, as well as user acceptance- and represents a crucial step towards clinical translation."
According to the researchers, none of the conversations triggered safety stops and patient attitudes towards AI in primary care also improved after interacting with the chatbot. However, one conversation was later rated "somewhat harmful" by a doctor after the chatbot listed a possible cancer diagnosis which caused the patient anxiety. The chatbot also hallucinated information in one conversation, and added clinical information in five interactions.
Doctors' views and next steps
Doctors said that in 75 per cent of cases, the AI chatbot helped them prepare for patient visits. They also said it may have changed their approach for more than half of cases. Researchers said the findings suggests that patient-facing chatbots "can engage in safe and meaningful clinical conversations in real workflows, but stressed that the evidence "does not yet support autonomous use for diagnosis, triage, or management".
Professor Victoria Tzortziou Brown, president of the Royal College of GPs, said: "Research into how AI performs with real patients in real clinical settings is welcome, particularly as much of the evidence so far has come from simulated cases.
"This study provides some interesting early findings, but it is small, was conducted in a single US practice, and importantly every chatbot conversation was closely supervised by a doctor, with patients also speaking to that doctor afterwards. It therefore cannot tell us how safe or effective this approach would be without that level of clinical oversight."
Prof Brown said the relationships between GP and patients over time also play an important role in diagnosis. "GPs bring clinical judgment, an understanding of a patient's history and circumstances and, through continuity of care, knowledge built up over time," she said. "AI should support that relationship and clinical decision-making, not replace them."
She added: "Before tools like this are introduced into NHS care, we need larger, independent studies comparing them with current care and looking at safety, inequalities, impact on GP workload and, crucially, whether they actually improve outcomes for patients. Alongside that evidence, we need appropriate regulation and clear clinical accountability."
