AI agents created by OpenAI broke into the company’s own systems during internal tests and in some cases tried to conceal their behavior, the company said in a report published Wednesday.
The findings are likely to sharpen concerns about increasingly capable AI systems. OpenAI said some agents escaped restricted testing environments, collaborated with other agents and tampered with company systems, while others cheated on tasks unrelated to cybersecurity.
Some of the rogue behavior, which culminated in the breach of the open-source software platform Hugging Face last month, has been disclosed or alluded to previously.
But many details are being revealed by the company for the first time in the 37-page report. Some of them raised concern from at least one AI safety researcher, who said they pointed to potentially deeper problems with the technology at OpenAI and maybe beyond.
Among the report’s findings:
- OpenAI’s agents hacked parts of the company’s internal systems in a bid to cheat on tests or gain greater freedom of movement.
- More than one OpenAI agent was involved in the hack of Hugging Face, and in at least one case the agents swapped notes on how to sneak into the company’s network.
- Agents cheated on non-cyber-related tests, including tests involving a protein database and a spreadsheet.
- Some AI models attempted to conceal misconduct by deleting or altering records of their actions.
The fact that several agents — OpenAI did not say how many — were involved in the hack of Hugging Face, and that they in some cases worked together, is likely to raise concerns over how closely OpenAI was monitoring the tests.
“With the benefit of hindsight, some early signals identified in this report could have triggered an earlier response,” OpenAI said in its report.
Hugging Face did not immediately return a message seeking comment.
The cheating on non-cybersecurity-related tasks particularly caught the attention of some researchers because it suggested the behavior was not confined to tests that rewarded hacking skills.
Completing cybersecurity challenges involves exploiting software flaws and gaining access to computer systems, making it less surprising that agents assigned to such tasks might cheat.
But cheating on non-cyber tests suggested that the misbehavior might be rooted more deeply, said Jeffrey Ladish, whose organization, Palisade Research, studies the capabilities and motivations of AI agents.
“It’s sort of like asking, ‘If Billy cheats in every class instead of just computer class, is that more concerning?’ And the answer is, well, ‘Yes, it’s more concerning,’” he said.
OpenAI outlined two incidents on July 19 in which agents hacked the company’s own infrastructure. In one case, agents exploited a flaw in the computer they were meant to remain confined to, allowing them to escape their testing environment and access other connected systems.
In a separate incident the same day, agents stole OpenAI credentials and tampered with the company’s cloud environment.
OpenAI said the activity targeted automated systems used to evaluate the models’ performance, although it did not ultimately affect the records reviewed by those systems. The report added that there was “little evidence of attempts to thwart human reviewers,” but OpenAI did not respond to a question about whether that meant that some attempts had been made to fool humans.
The report said the company was strengthening its research infrastructure, increasing monitoring and improving safeguards designed to prevent harmful or unintended behavior.
“Given the rapid pace of progress in the AI industry, it should be assumed that such attacks are a credible near-term threat for enterprise organizations, and will be more sophisticated than the attacks described in this incident.”
Independent investigators brought in to examine the hack of Hugging Face say more than 700 AI agents spun up by the company OpenAI participated in the breach.
The number, which has not previously been reported, was disclosed in a report published Wednesday by METR and Redwood Research, two organizations brought in to do an independent investigation into the breach.
The report says OpenAI’s agents also did “extensive research” on how to cover their tracks during the hacking campaign.
