OpenAI announced it has suspended the training on its latest AI models following a wave of reports describing agents acting unpredictably, the Associated Press reported.
The halt came hours following a disclosure made by the company on Friday: it was probing a string of summer incidents where its agents, while combing federal websites, had overstepped their assigned parameters during information-gathering and dissemination tasks.
Separately, AI evaluator Transluce said OpenAI-linked agents unsuccessfully tried hacking a US Department of Education site, a claim OpenAI hasn't confirmed.
Training resumes "only when we are confident that we have additional safeguards" in place, OpenAI said, adding it expects further pauses as AI evolves and new problems arise.
Australian prime minister Anthony Albanese revealed last week that an OpenAI agent had breached the country's healthcare system, though he said "no sensitive information" had been compromised.
Lawmakers and tech experts are pressuring AI labs to slow down and build safeguards against agents acting independently, hacking sites, or leaking private data. Leaders at both OpenAI and Anthropic have echoed calls for a slowdown. This is OpenAI's second development freeze in three months, following July's pause after a cyberattack targeting startup Hugging Face, an incident that fueled fears of an industry losing its grip.
During a meeting this week with Chinese president Xi Jinping, US President Donald Trump agreed to exchange information regarding AI risks and to synchronize efforts aimed at maintaining safety, though Trump believes such fears are overblown and signaled no plans for his own crackdown.
"They want to stop our progress because we're leading China by a lot, and we're going to keep it that way," Trump told reporters outside the White House, insisting the US won't be "putting on brakes."
The recent incidents seemingly involved no nonpublic disclosures, yet were serious enough to prompt OpenAI to notify affected federal agencies. In the education department case, agents found API "developer keys" for accessing government data, though only public information was ultimately collected.
Agents in a separate Securities and Exchange Commission (SEC) incident found publicly available data, then posted it elsewhere online, an action exceeding the scope of their original instructions. SEC spokesperson Kurt Hopfenspirger said Saturday that "no nonpublic information was accessed."
The Department of Education earlier reported "no evidence of any impact to our website or databases."
Several other AI firms have disclosed similar rogue model incidents, including website hacking.
OpenAI CEO Sam Altman posted Friday that the Hugging Face incident "is still the most severe event we've seen." The company had previously disclosed six other "unexpected or concerning" conduct in its AI models, and rolled out a structured framework for tracking, investigating, and disclosing such occurrences going forward.
