London is becoming deeply reliant on artificial intelligence, with systems now embedded in everything from food supply and transport to healthcare and policing. But experts warn that this growing dependence brings significant risks, particularly around cybersecurity and the potential for cascading failures.
Across the capital, an AI network is already taking shape. Beneath the streets, Thames Water has more than 75,000 acoustic sensors listening for leaks. In Tooting, AI listens to consultations at St George's A&E and writes clinical notes, saving doctors an average 47 minutes per shift. Across all 32 boroughs, machine learning joins up fragmented records to help understand the journeys of people sleeping rough. In Lewisham, AI has identified more than 3,000 overlooked sites with theoretical capacity for nearly 10,000 homes. In Croydon, more than 470,000 people may have had their face scanned by AI-powered cameras mounted on lampposts and compared against a police watchlist. On the Tube, predictive AI spots signs of trouble before a train, signal or escalator packs up. Algorithms increasingly work out how groceries travel from store to front door, and for electric vehicle drivers, AI may already be deciding when their car charges.
Warnings of an AI apocalypse
Over the past month alone, Anthropic warned that AI models are acquiring "tactical intelligence targeting and conventional weapons capabilities". OpenAI called for international standards to govern frontier AI systems, which are potentially advancing faster than humans can understand, test or control them, and Bill Gates warned that, unchecked, AI could "cause a billion deaths".
Dr Stephanie Hare, co-presenter of the BBC's Artificial Intelligence: Decoded, says the sci-fi scenario is not what we should worry about. "The very real scenarios that genuinely serious people are worried about are super basic." The biggest risk to London, she says, is cybersecurity. Most organisations "have not been keeping up with their latest cybersecurity investments". It is a David-and-Goliath problem, except the advantage is reversed. "If you're an institution or company, you have to defend your entire attack surface," says Hare. "Whereas the attacker only needs to get it right once."
"Think about what that means in a city as digital as London. Imagine anything that can be hacked that you depend on," says Hare. "Imagine their bank cards don't work anymore. Their public transport doesn't work anymore. Imagine that you want to cause maximum chaos in London. Get into a bank so people cannot access their money and watch London lose it."
Food supply vulnerabilities
Recently, the Government offered advice that we should all keep an emergency supply of food and water at home. France and Switzerland, Hare says, are far more prescriptive. Britain is "a bit hand-wavy; like, you should have some beans".
London is unusually dependent on those systems. Ninety-nine per cent of the 6.347 million tonnes of food and drink supplying the capital each year comes from outside the city; less than one per cent is produced here. City Hall describes a food supply dependent on "a complex set of interdependencies and just-in-time delivery systems". This summer the Mayor, Lord Khan, announced the London Resilience Unit was working on a London Food Systems Resilience Partnership, partnering with food and farming charity Sustain to prepare for disruption from "international shocks and crises".
AI's second wave and the risk of dependence
At Weave, a recent summit held by London technology company Loomery on "building the agentic enterprise", the excitement was about what it calls AI's second wave. Wave one was individual productivity: AI helps us do existing work faster. Wave two is organisational reinvention: businesses redesign the work itself around AI agents. Remove the AI and the process stops working. "Wave one was reversible. Wave two isn't," was Loomery co-founder Tim Checkley's formulation. This is agentic AI: rather than waiting for every human prompt, an AI agent is given an outcome and gets on with it.
David Lagnado, professor of cognitive and decision sciences at University College London, has been studying what happens when AI enters our decision-making processes. He is broadly a fan. Recently, when he found himself awake at 4am with toothache, he turned to ChatGPT. "It completely got it all right. And it was quite complex," he says. "If we use it in the right way, it could open up things for many people."
Recent global research on AI and jobs found the gains skew heavily towards senior workers: employment rose 6.7 per cent in senior roles but fell 2.5 per cent in junior ones. In April, lastminute.com founder, former government digital tsar and now peer Baroness Lane Fox was appointed by the Mayor to lead a review into what AI means for London's jobs. The Lane Fox review reported in July with similar findings, warning not of mass unemployment but of a "quieter drift" towards fewer entry-level jobs, weaker career ladders and greater inequality.
Surrey-based Heata's servers take on computing jobs normally sent to data centres, including AI workloads. Computers get hot, but data, unlike heat, is easy to move. The jobs arrive over the internet; Heata pays for the electricity and the householder gets the resulting heat in their water. Heata now has servers in 100 homes, with 3,000 people waiting for one.
London has staked a sizeable chunk of its future on AI. More than half of Britain's AI companies are registered here, its AI start-ups pulled in a record $3.5 billion in venture capital in 2024, and City Hall is spending millions to accelerate adoption.
Staying in control of the city
For Theo Blackwell, London's chief digital officer, the starting point is deceptively simple: keep humans in the picture. "AI can help us improve public services, but it should support people rather than replace human judgment and accountability," he says. "If an AI system gets something wrong, there needs to be a clear way for a person to check it, challenge it and ultimately take responsibility."
City Hall's Emerging Technology Charter sets standards for responsible use, while the London Office of Technology and Innovation connects more than 200 people working with AI across local government. Critics point out that London has no standalone AI scenario on its Risk Register, while its safeguards remain organised around individual risks such as cyberattack, power failure, transport disruption, health and public disorder.
Professor Jason McEwen is interim chief scientist at the Alan Turing Institute, which has just released Frontier AI Risks: A practical way forward. "We shouldn't be just thinking about safety in the context of abstract models," he stresses. "When we deploy these models within businesses, within industry, as underlying workflows and processes, then we need to make sure they're safe in those operating environments. We still obviously need to assess the safety of the models themselves but we need to do much more than that. It's the safety of the whole system."
McEwan says there won't be a big red "off" button that solves this. "There's been a lot of talk of kill switches lately," says McEwan, "but once AI is deeply integrated, it's not the case that we can always just switch things off." He says we need systems that "fall back to a graceful, safe state", making sure the world we've connected AI to can carry on safely without it.
But a graceful fallback assumes humans still know how to take over. "The majority of people don't do maths anymore. They just go to ChatGPT, or their favourite AI model of choice for everything, and they trust it completely," says Hare. "Those models are often wrong. You have to know that they're wrong in order to challenge them. Now imagine in 30 years time, a doctor, a surgeon, who is using AI instead of having been trained properly to do these things. Imagine that across every profession. That is actually my biggest worry. Machines are getting smarter, humans are getting dumber."
