Military surgeon Dr Darren Mann has warned that autonomous drones already kill without distinction, and that the loss of control in weapons is a present danger, not a future one. Writing in response to recent articles, he argues that a drone told to clear a “kill box” does what it is told, killing whatever is inside, combatant or child.
A loss of control that kills today
Dr Mann notes that AI’s builders fear losing control of it, but this is not only a future superintelligence’s problem. It is already loaded into weapons, where a loss of control kills a civilian today, not humanity tomorrow.
The gap between a target and a lawful target is the principle of distinction, which no instruction closes, because the instruction was lawful. Dr Mann suggests that the check need not be another opaque mind but a deterministic one: its rulebook the Geneva conventions, its judgments auditable, returned to a human. “We need not trust it; we can check it,” he writes.
The fallacy of bloodless war
A further danger is peculiar to war. Autonomy is sold as combat without our own dead, and a bloodless war is easier to start. But that is a fallacy: you can take the human from the targeting loop, not the nation from the war. Ground must be won and held; the dead still come, in attack or defence. Armies no longer go to war; nations do.
Holding the line
Dr Mann concludes that if we cannot teach a drone to keep the laws of war, we will not teach a greater intelligence to keep faith with all we value. To abandon those laws for survival is the confusion of interests with values that our strategists warn against; here the two are one. “Machines that kill without distinction, answerable to no one, leave no one safe. Our interests do not merely permit us to hold the line; they demand it.”
