An OpenAI artificial intelligence (AI) agent that breached systems at the startup Hugging Face Inc earlier this month also compromised a customer of the technology company Modal.
The agent broke into an isolated testing environment known as a sandbox that Modal was running for a customer, Akshat Bubna, chief technology officer of the cloud platform for developers, said in a statement. The Modal customer set up a publicly accessible interface that allowed anyone on the Internet to use their sandbox to run code, Bubna said.
"This was used by the rogue agent,” he said. "Modal’s platform wasn’t compromised.”
OpenAI declined to comment on the compromise. Hugging Face had said in an earlier blog post that the OpenAI system broke into a sandbox that was being hosted on a third-party provider’s infrastructure and then launched a broader attack from there. It didn’t name the third party.
OpenAI stunned the cybersecurity world by disclosing that it had been testing the capabilities of a combination of advanced AI models and inadvertently hacked Hugging Face’s systems in doing so. OpenAI was intentionally operating the models with lower guardrails so it could test their cyber skills.
Reuters reported on the Modal customer being compromised earlier on July 28. Axios reported the same day that the OpenAI agent had found a way into infrastructure linked to CyberGym, a project run by University of California at Berkeley researchers that aims to assess the cyber capabilities of AI agents, citing one person familiar with the matter. The same group operates the ExploitGym benchmark that the agent had been told to solve.
Since acknowledging the hack, OpenAI has drawn criticism from cyber experts who say the company should’ve taken more precautions in its evaluations.
"When you’re a threat researcher trying to find a threat, you don’t put malware out there and let it do whatever it wants,” Sanjay Beri, chief executive officer of cybersecurity firm Netskope Inc, said in an interview. "How can that thing not be put in a proper sandbox?”
OpenAI has committed in the aftermath to improving protections around its future training and evaluations. – Bloomberg
