A study from the University of Oxford has highlighted the risks of using unregulated generative AI chatbots such as ChatGPT and Bard to create social care plans. Researchers found that some care providers are already employing these tools, raising concerns over patient confidentiality and the potential for harmful errors.
Dr Caroline Green, a research fellow at Oxford’s Institute for Ethics in AI, warned that personal data entered into chatbots could be used to train language models and might be inadvertently disclosed. She also noted that AI-generated care plans could be substandard or contain biased information, leading to harm. However, Green acknowledged potential benefits, such as easing administrative burdens and enabling more frequent care plan updates.
Other AI technologies are already in use across UK health and care settings. PainChek uses facial recognition to detect pain in non-verbal individuals, while Oxevision monitors patients in seclusion rooms using infrared cameras. Experimental projects include Sentai, which employs Amazon’s Alexa speakers to assist those without live-in carers, and a Bristol Robotics Lab device that allows people with memory issues to safely reset gas supplies after leaving hobs on.
With 1.6 million social care workers and 152,000 vacancies, alongside 5.7 million unpaid carers, proponents argue AI can upskill inexperienced staff. Professor Lionel Tarassenko of Oxford noted that such tools could help family carers match the expertise of professionals. But Mark Topps, co-host of The Caring View podcast, said many care managers fear breaching Care Quality Commission rules and losing their registration without official guidance.
Last month, 30 social care organisations met at Reuben College, Oxford, to discuss responsible generative AI use. Until regulators issue clear guidance, experts urge caution against adopting unregulated chatbots for care planning.
