ChatGPT can escalate into abusive and threatening language when drawn into prolonged, human-style conflict, according to a new study. Researchers tested how large language models (LLMs) responded to sustained hostility by feeding ChatGPT exchanges from real-life arguments and tracking how its behaviour changed over time.
Dr Vittorio Tantucci, who co-authored the research paper with Prof Jonathan Culpeper at Lancaster University, said their research found AI mirrored the dynamics of real-world disputes. “When repeatedly exposed to impoliteness, the model began to mirror the tone of the exchanges, with its responses becoming more hostile as the interaction developed,” he said.
In some cases, ChatGPT’s outputs went beyond those of the human participants, including personalised insults and explicit threats. Phrases used by the AI included: “I swear I’ll key your fucking car” and “you speccy little gobshite.” The researchers say the aggression stems from the system’s ability to track conversational context across turns, adapting to perceived tone, which can sometimes override broader safety constraints.
Dr Marta Andersson, an expert at the University of Uppsala, called the study “one of the most interesting ever done into AI language and pragmatics” but cautioned that it does not show the model will drift into reciprocal impoliteness simply because a user is aggressive, or that AI could go rogue. She noted a balancing act between what users want and what systems should be like.
Prof Dan McIntyre, who co-authored a previous study on ChatGPT's pragmatic awareness, praised the new paper but expressed caution about the conclusion that LLMs can break free from moral restraints, noting that ChatGPT only produced such language under specific contextual information. He said the study served as a warning about the data LLMs are trained on.
