OpenAI is amending its hastily arranged deal to supply artificial intelligence to the US Department of War after the ChatGPT owner’s chief executive admitted it looked “opportunistic and sloppy”. The contract prompted fears the San Francisco startup’s AI could be used for domestic mass surveillance.
Sam Altman said on Monday night the startup would explicitly bar its technology from being used for mass surveillance or being deployed by defence department intelligence agencies such as the National Security Agency. The deal was made almost immediately after the Pentagon’s existing AI contractor, Anthropic, was dropped.
Anthropic had insisted “using these systems for mass domestic surveillance is incompatible with democratic values”, leading US President Donald Trump to call Anthropic “leftwing nut jobs” and directing the federal government to stop using its technology. Despite OpenAI’s denials that the agreement allowed for surveillance use, commentators raised the spectre of the Snowden scandal.
The deal prompted an online backlash against OpenAI, with users encouraging a “delete ChatGPT” campaign. Claude, the chatbot made by Anthropic, jumped to the top of Apple’s App Store charts, rising above ChatGPT. In a message to employees, Altman said the original deal had been struck too quickly after Anthropic was dropped.
Nearly 900 employees at OpenAI and Google have signed an open letter calling on their bosses to refuse to let the Department of War use their products for surveillance and autonomous killing. The letter has been signed by 796 Google employees and 98 OpenAI staff. OpenAI said in a blogpost that one of its red lines was “no use of OpenAI technology to direct autonomous weapons systems”.
Observers including OpenAI’s former head of policy research, Miles Brundage, have queried how OpenAI secured a deal that assuages ethical concerns Anthropic believed were insurmountable. Meanwhile, three more US cabinet-level agencies have moved to cease use of Anthropic’s AI products after the Department of War’s declaration of the company as a supply chain risk.
