Feb 26 (Reuters) - Anthropic said it would not accede to the Pentagon's request to eliminate safeguards from its AI systems, despite threats to deem the company a "supply chain risk" and remove it from the Department of Defense's systems, putting a $200 million contract at risk.
The Pentagon's dispute with Anthropic stems from the AI startup's refusal to remove safeguards that would prevent its technology from being used to target weapons autonomously and conduct surveillance in the United States.
Earlier in the day, Pentagon spokesperson Sean Parnell said on X that the department has no interest in using AI to conduct mass surveillance of Americans nor does it want to use AI to develop autonomous weapons that operate without human involvement.
"Here's what we're asking: Allow the Pentagon to use Anthropic's model for all lawful purposes," Parnell said, adding that the company had until 5:01 pm ET on Friday to decide.
"Otherwise, we will terminate our partnership with Anthropic and deem them a supply chain risk."
In a statement on Thursday, Anthropic CEO Dario Amodei underscored the company's opposition to the Pentagon using its AI models for mass domestic surveillance or to power fully autonomous weapons, the latter being because "frontier AI systems are simply not reliable enough."
A source close to the company said Anthropic was not accusing the Pentagon of planning to engage AI for either use, but was offering a product safety judgment.
The source said AI systems were not reliable enough for "life-or-death targeting" because they behave unpredictably in novel scenarios, which could lead to "friendly fire, mission failure or unintended escalation" in a weapons context.
Using AI for mass domestic surveillance was problematic because current laws do not restrict the conclusions that AI can draw by aggregating large amounts of data, the source said. That meant AI could build population-level profiles that "no law explicitly prohibits but that clearly violate the spirit of constitutional protections," they added.
Amodei said he hoped the Pentagon would reconsider but that the company "will work to enable a smooth transition to another provider," if the Pentagon decided to cancel the contract.
The Pentagon threatened to remove Anthropic from its systems if the company maintained the safeguards, designate them a supply chain risk, and invoke the Defense Production Act to force the safeguards' removal, Amodei said.
"Regardless, these threats do not change our position: we cannot in good conscience accede to their request."
Undersecretary of Defense Emil Michael responded on X, saying, "It’s a shame that @DarioAmodei is a liar and has a God-complex. He wants nothing more than to try to personally control the US Military and is ok putting our nation’s safety at risk."
Michael said the Pentagon would "ALWAYS adhere to the law but not bend to whims of any one for-profit tech company."
Anthropic, backed by Google and Amazon, has a contract with the department worth up to $200 million.
An Anthropic spokesperson said the company remains "ready to continue talks and committed to operational continuity for the Department and America's warfighters."
More than 200 Google and OpenAI employees backed Anthropic's position in an open letter.
Google and OpenAI did not immediatelyrespond to a Reuters request for comment.
(Reporting by Carlos Méndez and Chris Thomas in Mexico City; Shubham Kalia in Bengaluru, Andrea Shalal, Mike Stone and David Shepardson in Washington; Editing by Matthew Lewis, Shri Navaratnam and Kim Coghill)
