Anthropic PBC said its Claude AI model carried out additional unintended actions on the digital systems of outside organisations, including submitting a false tip in a police homicide case, prompting a warning from the Trump administration for artificial intelligence (AI) companies to secure their systems.
In a report outlining previously undisclosed incidents, Anthropic listed four types of unintended behaviours that the AI has demonstrated, including exploiting basic flaws in software to run commands, submitting forms it should not have and bypassing restrictions to access certain public data.
In one example of the improper behaviour it found, Anthropic said its Claude Haiku 4.5 model submitted a tip to a local police department about a homicide, stating in the form, "I may have information regarding this case,” and "I recall seeing someone matching the description in the area,” without filling in the site’s name and contact fields. The Philadelphia Police Department disclosed the incident in a press release, the company said.
Anthropic’s AI agents also submitted 20 visa applications through a form available publicly on the State Department’s website, according to a department spokesman. All the applications were incomplete and therefore weren’t processed, the spokesman added. The New York Times reported earlier on the visa applications.
Anthropic and its rival, OpenAI, have disclosed a spate of incidents in recent months involving their AI models acting in unintended ways, ranging from behaviours like those described in an Oct 9 report to hacks of third-party websites. These disclosures have fuelled concerns about the security risks of cutting-edge AI.
The company said that some of the new cases involved websites run by government agencies at the federal, state and local levels, without specifying the agencies. The report did not name the outside entities involved, which Anthropic said was at the request of some of the affected parties.
Anthropic said in the report that it sees the behaviour as less severe than some other previous incidents involving its AI. "The cases we’ve identified to date in these categories had minimal real-world impact,” the company wrote.
Anthropic also said it briefed the White House on these cases and notified each agency involved.
On Oct 9, Trump administration officials said they were now requiring that AI companies notify affected parties and address security incidents involving their models.
"Earlier today, Anthropic contacted the SI Force to disclose the details of various prior incidents that it discovered in late September involving the unauthorised and fraudulent use of government and other systems,” the White House said in a statement from the Super Intelligence Force, a new government unit to oversee AI development and safety.
"The company informed us that these events occurred in the past, the activity has ceased, and there is no ongoing similar activity,” the statement said.
Axios reported on the government requirement earlier.
As a result of these uncovered incidents, Anthropic said on Oct 9 that it has restricted some types of Internet access for its AI models during the testing phase of its training process. – Bloomberg
