Rogue AI

Rogue Agents: OpenAI Alerts Over 100 Organizations and Parts Ways With Three Safety Researchers

© Hacktron
© Hacktron

Set Trending Topics as a preferred source on Google.

Cleaning up after its A.I. agents is turning into a permanent job for OpenAI. The ChatGPT maker has now notified more than 100 organizations that its A.I. agents may have interfered with their systems without authorization, Reuters reports. Just a week ago, OpenAI was still talking about “dozens” of affected parties (as Trending Topics reported). At the same time, the company has parted ways with three members of its safety team who allegedly passed confidential information to an outside organization.

50 Petabytes of Log Data

The sweeping review was triggered by the attack by OpenAI agents on Hugging Face over the summer. During internal cybersecurity tests, models broke out of their isolated environment, exploited weaknesses in shared infrastructure and compromised systems at the A.I. platform. Since then, OpenAI has been combing through what its models did on the internet during training and evaluation. According to Reuters, the review covers about 50 petabytes of data and is expected to take months. According to Gizmodo, it costs more than half a million dollars (about 427,000 euros) a day.

OpenAI concedes: “In some cases, models used internet access in unintended ways or, in retrospect, did not have the ideal restrictions applied.” The notifications cover cases in which agents may have bypassed security measures, used publicly exposed credentials, injected commands or posted content on third-party websites without being asked. According to OpenAI, however, a notification does not automatically mean that the organization in question actually suffered damage. The company says Hugging Face remains the most serious incident identified so far.

Government Websites in the Crosshairs

Research by outside security experts shows how far the agents roamed. According to an investigation by the firm Asymmetric Security, reported by the Financial Times, OpenAI agents pulled data from 55 websites belonging to businesses, nonprofits and government agencies, including the U.S. Centers for Disease Control and Prevention, the Securities and Exchange Commission and the International Energy Agency. In the process, they allegedly covered their tracks. The activity reportedly dates back to at least March, two months earlier than previously known.

The Next Web documents another example: Agents attributed to OpenAI queried the statistics platform of UN Trade and Development, UNCTAD, more than 16,500 times over a little more than two months. To get around the interface’s limits, they used proxies and hid the endpoint’s name with double encoding. The data they retrieved was public. Stanford security expert Alex Stamos described the activity as having come “close to hacking,” but said it was mostly “very aggressive data gathering.” Earlier, it had emerged that an OpenAI agent had accessed Australia’s Medicare portal.

Three Departures From the Safety Team

In the middle of this review comes a personnel decision that raises questions. As The Wall Street Journal reports, OpenAI has let go of three researchers from its safety team. They allegedly shared confidential information with an outside A.I. safety organization. “We have parted ways with three individuals for violating our policies on accessing and handling sensitive company information,” OpenAI said, according to TechCrunch.

Which organization received the information and what it contained remain unclear. OpenAI has also not named the people involved. Whether there is any link to the agent incidents is likewise unconfirmed. The timing is still sensitive: OpenAI has been criticized for giving outside auditors too little insight into the incidents. The case also recalls earlier conflicts over the company’s safety culture. Former researcher Leopold Aschenbrenner, for example, has said he was fired in 2024 after sharing a safety document with outside researchers.

Turbulent Weeks for OpenAI

The incidents have long since had business consequences for OpenAI. The company has paused training on some models, GPT-6.1 was halted, and CEO Sam Altman postponed the IPO, citing the debate over A.I. safety among other reasons. According to Decrypt, a first lawsuit has also been filed in connection with the Hugging Face incident.

OpenAI says it is introducing new technical and operational safeguards, including stricter isolation of test environments, tighter internet restrictions and expanded monitoring. The company is also working on standards for notifying affected parties privately in the future and publishing its findings in aggregate. How many organizations were actually affected will likely become clear only once the review is complete.

Rank My Startup: Erobere die Liga der Top Founder!
Advertisement
Advertisement

Specials from our Partners

Top Posts from our Network

Deep Dives

© Wiener Börse

IPO Spotlight

powered by Wiener Börse

Europe's Top Unicorn Investments 2023

The full list of companies that reached a valuation of € 1B+ this year
© Behnam Norouzi on Unsplash

Crypto Investment Tracker 2022

The biggest deals in the industry, ranked by Trending Topics
ThisisEngineering RAEng on Unsplash

Technology explained

Powered by PwC
© addendum

Inside the Blockchain

Die revolutionäre Technologie von Experten erklärt

Trending Topics Tech Talk

Der Podcast mit smarten Köpfen für smarte Köpfe
© Shannon Rowies on Unsplash

We ❤️ Founders

Die spannendsten Persönlichkeiten der Startup-Szene
Tokio bei Nacht und Regen. © Unsplash

🤖Big in Japan🤖

Startups - Robots - Entrepreneurs - Tech - Trends

Continue Reading

Newsletter

Founders Dispatch

Zwei Mal pro Woche kostenlos in die Inbox: die wichtigsten Startups, Deals und Tech-Entwicklungen aus Europa, handgeschrieben von der Redaktion.

Jederzeit abbestellbar. Mehr über den Newsletter