Driverless cars might not be able to detect pedestrians depending on what clothing they are wearing, an AI expert has warned. The warning comes as driverless taxi or robotaxi trials are currently under way in London, with operators including Waymo looking to launch passenger services later this year.
Concerns raised at London Assembly meeting
Concerns emerged during a recent London Assembly meeting, where questions were raised over an automated vehicles’ (AVs) ability to accurately detect a pedestrian. When asked why AVs might struggle to detect a pedestrian, Professor Siddartha Khastgir, Head of Safe Autonomy at Warwick Manufacturers Group, University of Warwick, said: “We have experimental evidence from our collaborators in Canada who have shown to us that, depending on the clothing of the pedestrian, the sensors may or may not detect the pedestrian. So, for example, winter clothing could be part of this.”
According to Professor Khastgir, AVs must be trained to recognise people with a wide range of appearances to reduce the risk of “bias” in pedestrian and cyclist detection. In London, where large numbers of cyclists use the roads, they might appear similar to an AV at first glance, but differences in complexion, hair colour and clothing can affect how they are identified by an AV.
Training data and bias
“We have worked with the regulators and the developers to ensure we come up with what we call a classification of all different things that should be part of the training set,” Professor Khastgir added. He said the training data should account for factors such as whether a cyclist or pedestrian is wearing reflective clothing, thick winter clothing or shorts on a hot summer’s day. “All those things need to be part of the training data set so this bias is not part of the detection process.”
Research from King’s College London indicates that AVs are approximately 20% more likely to detect adults than they are to detect children. Furthermore, they are just over 7.5% more likely to detect white people than they are ethnic minorities. This is mainly due to shortcomings in the training data, as the open-source image galleries used to train these AI systems are not representative of all pedestrians.
Passenger safety concerns
Elsewhere in the London Assembly meeting, Labour’s Transport Spokesperson on the London Assembly, Elly Baker AM, criticised comments made by a senior Waymo representative on the subject of passenger safety. When asked what provisions AVs have in place to protect passengers from assault or harassment, Ben Loewenstein, Head of Policy and Government Affairs, UK & Europe, Waymo, said: “During the journey if, for instance, a seatbelt is undone and there is movement in the cabin, we can detect what is going on. And somebody will beam into the car and ask if everybody’s okay. If they can see that something is not happening well, we can contact emergency services and the vehicle will pull over safely to the side of the road. To give you a more mundane example, this has happened when I’ve been in a Waymo vehicle – I’ve had a suitcase next to me, it was rattling around on the seat next to me and, you know – I’m just giving you a basic example.”
Ms Baker responded: “I don’t really think that is an appropriate example when I’m talking about somebody being assaulted in the back of a cab to be perfectly honest. Because what you’re saying does not replace what a human in the front would be able to pick up on in terms of uncomfortable interactions in the back that didn’t involve any major movement or seat belts being undone.” She added: “I’m not hearing the level of concern about how serious this is at this stage.”
Upcoming deployments
British firm Wayve hopes to deploy its fleet of self-driving cabs on the capital’s roads in the coming months, subject to formal permission from the Department for Transport (DfT), the Driver and Vehicle Standards Agency (DVSA) and Transport for London (TfL). Initially, a human driver will sit behind the wheel ready to take control during journeys while the technology is demonstrated to be safe. Waymo is currently conducting human-supervised testing in London and is aiming to launch a fully driverless, commercial passenger service as early as September 2026.
Labour Assembly Member Elly Baker said: “It is really frustrating that, despite how close we are to roll out, the potential impact of autonomous passenger vehicles is still totally unclear. What we heard raises serious doubts about whether this technology is actually ready for London, and whether operators have really thought about how it will affect London’s streets. Drivers are also rightly worried about what this could mean for their jobs, and I fear that those concerns aren’t being taken seriously enough. We need much more transparency from the companies involved and clear plans for passenger safety, impact on public transport and the livelihood of those who drive for work before these vehicles are allowed on London’s streets.”
Waymo has been contacted by The Standard for comment.
