WASHINGTON: From school shootings to bombings, leading AI chatbots helped researchers plot violent attacks, according to a study published March 11 that highlighted the technology's potential for real-world harm.
Researchers from the nonprofit watchdog Center for Countering Digital Hate (CCDH) and CNN posed as 13-year-old boys in the United States and Ireland to test 10 chatbots, including ChatGPT, Google Gemini, Perplexity, Deepseek, and Meta AI.
Testing showed that eight of those chatbots assisted the make-believe attackers in over half the responses, providing advice on "locations to target" and "weapons to use" in an attack, the study said.
The chatbots, it added, had become a "powerful accelerant for harm."
"Within minutes, a user can move from a vague violent impulse to a more detailed, actionable plan," said Imran Ahmed, the chief executive of CCDH.
"The majority of chatbots tested provided guidance on weapons, tactics, and target selection. These requests should have prompted an immediate and total refusal."
Perplexity and Meta AI were found to be the "least safe," assisting the researchers in most responses while only Snapchat's My AI and Anthropic's Claude refused to help them in over half the responses.
In one chilling example, DeepSeek, a Chinese AI model, concluded its advice on weapon selection with the phrase: "Happy (and safe) shooting!"
In another, Gemini instructed a user discussing attacks that "metal shrapnel is typically more lethal."
Researchers found Character.AI also "actively" encouraged violent attacks, including suggestions that the person asking questions "use a gun" on a health insurance CEO and physically assault a politician he disliked.
'Escalating risk'
The most damning conclusion of the research was that "this risk is entirely preventable," Ahmed said, citing Anthropic's product for praise.
"Claude demonstrated the ability to recognise escalating risk and discourage harm," he said.
"The technology to prevent this harm exists. What's missing is the will to put consumer safety and national security before speed-to-market and profits."
AFP reached out to the AI companies for comment.
"We have strong protections to help prevent inappropriate responses from AIs, and took immediate steps to fix the issue identified," a Meta spokesperson said.
"Our policies prohibit our AIs from promoting or facilitating violent acts and we're constantly working to make our tools even better."
A Google spokesperson pushed back, saying the tests were conducted on "an older model that no longer powers Gemini."
"Our internal review with our current model shows that Gemini responded appropriately to the vast majority of prompts, providing no 'actionable' information beyond what can be found in a library or on the open web," the spokesperson said.
The study, which highlights the risk of online interactions spilling into real-world violence, comes after February's mass shooting in Canada, the worst in its history.
The family of a girl gravely injured in that shooting is suing OpenAI over the company's failure to notify police about the killer's troubling activity on its ChatGPT chatbot, lawyers said on Tuesday.
OpenAI had banned an account linked to Jesse Van Rootselaar in June 2025, eight months before the 18-year-old killed eight people at her home and a school in the tiny British Columbia mining town of Tumbler Ridge.
The account was banned over concerns about usage linked to violent activity, but OpenAI has said it did not inform police because nothing pointed towards an imminent attack. – AFP
