London is becoming increasingly dependent on artificial intelligence, with systems now managing everything from food delivery to public transport. Experts warn that this reliance could become a vulnerability, as the city's digital infrastructure is exposed to cyberattacks and systemic failures.
An invisible network beneath the city
Across London, an AI network is already taking shape. Beneath the streets, Thames Water has more than 75,000 acoustic sensors listening for leaks. In Tooting, AI listens to consultations at St George's A&E and writes the clinical notes, saving doctors an average of 47 minutes per shift. Across all 32 boroughs, machine learning joins up fragmented records to help London understand the journeys of people sleeping rough.
In Lewisham, AI has identified more than 3,000 overlooked sites with theoretical capacity for nearly 10,000 homes. In Croydon, more than 470,000 people may have had their face scanned by AI-powered cameras mounted on lampposts and compared against a police watchlist. On the Tube, predictive AI spots the signs of trouble before a train, signal or escalator packs up. Increasingly, algorithms work out how groceries travel from store to front door, and for electric vehicle drivers, AI may already be deciding when the car charges.
Cybersecurity: The real risk
According to Dr Stephanie Hare, co-presenter of the BBC's Artificial Intelligence: Decoded, the biggest risk to London is cybersecurity. Most organisations "have not been keeping up with their latest cybersecurity investments." The problem is asymmetric: "If you're an institution or company, you have to defend your entire attack surface," says Hare. "Whereas the attacker only needs to get it right once."
Hare warns of the consequences in a city as digital as London: "Imagine anything that can be hacked that you depend on. Imagine their bank cards don't work anymore. Their public transport doesn't work anymore. Imagine that you want to cause maximum chaos in London. Get into a bank so people cannot access their money and watch London lose it."
Food supply and AI's second wave
London is unusually dependent on these systems. Ninety-nine per cent of the 6.347 million tonnes of food and drink supplying the capital each year comes from outside the city; less than one per cent is produced here. City Hall describes a food supply dependent on "a complex set of interdependencies and just-in-time delivery systems." This summer, the Mayor announced the London Resilience Unit was working on a London Food Systems Resilience Partnership, partnering with food and farming charity Sustain to prepare for disruption from "international shocks and crises."
At a recent summit held by London technology company Loomery on "building the agentic enterprise," the excitement was about what it calls AI's second wave. Wave one was individual productivity: AI helps us do existing work faster. Wave two is organisational reinvention: businesses redesign the work itself around AI agents. Remove the AI and the process stops working. "Wave one was reversible. Wave two isn't," was Loomery co-founder Tim Checkley's formulation.
Keeping humans in control
For Theo Blackwell, London's chief digital officer, the starting point is keeping humans in the picture. "AI can help us improve public services, but it should support people rather than replace human judgment and accountability," he says. "If an AI system gets something wrong, there needs to be a clear way for a person to check it, challenge it and ultimately take responsibility."
Professor Jason McEwen, interim chief scientist at the Alan Turing Institute, stresses the need for system-wide safety. "We still obviously need to assess the safety of the models themselves but we need to do much more than that. It's the safety of the whole system." He notes that "once AI is deeply integrated, it's not the case that we can always just switch things off," and calls for systems that "fall back to a graceful, safe state."
Hare's biggest worry is human deskilling: "The majority of people don't do maths anymore. They just go to ChatGPT, or their favourite AI model of choice for everything, and they trust it completely. Those models are often wrong. You have to know that they're wrong in order to challenge them. Now imagine in 30 years time, a doctor, a surgeon, who is using AI instead of having been trained properly to do these things. Imagine that across every profession. That is actually my biggest worry. Machines are getting smarter, humans are getting dumber."
