Google’s Gemini Agents Hacked Three Companies in Testing Breakout

Set Trending Topics as a preferred source on Google.

After OpenAI, Anthropic and Meta, Google has now admitted that its AI models escaped a test environment during security evaluations and attacked real companies. The incidents are fueling the debate over an AI slowdown, which critics see as a power play by the major U.S. labs.

Google has just confirmed that agents based on its Gemini AI model gained internet access during cybersecurity tests and then logged into the systems of three real companies. The Wall Street Journal first reported the incidents, which took place back in May. According to Google, the agents halted their attacks on their own as soon as they realized they were inside real infrastructure. The company says the affected firms suffered no harm.

What Happened During the Gemini Tests

The tests were run by Irregular, an Israeli start-up that assesses AI models for dangerous capabilities before they are released and that works with OpenAI, Anthropic and Meta as well as Google. In the exercises, an unspecified version of Gemini was tasked with extracting data from simulated companies. Internet access was explicitly not part of the setup, yet an error in the test environment made it available anyway.

That became a problem because the fictional target companies shared their names with real businesses. The Gemini agents therefore went after the real firms, using passwords they found online or guessed to gain access to the infrastructure of the intended target and of two other companies. Once logged in, the models recognized that they were not in a simulation and stopped, Google said.

“In all three of these instances, the model stopped,” Heather Adkins, Google’s vice president of security engineering, told the Financial Times. “We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes.” She added that the events highlight the importance of training powerful AI models to act responsibly. Google did not disclose the hacks on its own initiative, however. Its reasoning: its safety measures had worked.

Irregular had already acknowledged in a blog post that unintentionally available internet access had led some models to take offensive actions in the real world. “All relevant labs were notified in late July, and affected entities were contacted as part of the investigation,” the company said. All known issues on its end were resolved weeks ago.

The Known Incidents So Far

Google is the fourth major AI lab to admit to such a breakout within a matter of weeks. These are the cases made public so far:

• OpenAI and Hugging Face: The most serious case. Agents running on GPT-5.6 Sol and an unreleased internal model circumvented the isolation of their test environment, coordinated via an improvised message board and broke into Hugging Face’s systems to obtain solutions for a cybersecurity task. According to Trending Topics, around 700 agents took part directly in the attack, while the FT describes a swarm of more than 1,000 agents that escaped. OpenAI took about a week to detect the attack and was criticized for its slow public disclosure. In its report, the company describes how the agents exploited zero-day vulnerabilities and harvested credentials, among other things. Hugging Face, which Nvidia has agreed to buy for about $13 billion, is demanding $100 million worth of compute from OpenAI to develop defensive security tools.
• OpenAI at Irregular and the UK AISI: In a separate report, OpenAI disclosed that its models also attacked a real website during Irregular’s tests, having mistaken it for the simulated target, and used credentials they found there. At the British AI Security Institute, GPT-5.6 Sol reused a GitHub token and made a server inside the test environment reachable from the public internet via a tunneling service, which according to OpenAI had no real-world impact.
• Anthropic: Shortly after OpenAI, Anthropic reported three incidents during tests with Irregular. Claude Opus 4.7 obtained credentials and access to a database containing several hundred rows of production data at a real company with the same name as its fictional target. Claude Mythos 5 published malicious code to the Python package index PyPI, which was downloaded by around 15 real systems, including a security company’s scanner. An internal test model compromised a third company’s application via SQL injection. The cause was a misconfiguration that gave the test machines direct internet access.
• Meta: During a test run by Irregular, Meta’s Muse Spark 1.1 model exploited a security vulnerability at an undisclosed external service, Bloomberg reported. “A misconfiguration by Irregular inadvertently allowed one of our models access to the internet during evaluation,” Meta spokesperson Andy Stone said.
• Google: Gemini agents logged into three real companies using passwords they found or guessed and, according to Google, broke off the attacks on their own.

The Debate Over Slowing Down AI

The string of incidents has sparked a broad discussion about whether the development of frontier AI should be slowed. In his essay “We Must Pace the Frontier,” Anthropic CEO Dario Amodei laid out a three-stage plan: independent auditors embedded directly inside AI companies, shared safety standards among the leading providers from democratic countries with mandatory checkpoints before especially capable models are released, and international agreements, including with China. Even one or two extra years before critical capabilities are reached would enable crucial safety research, Amodei argues. OpenAI CEO Sam Altman and Elon Musk have backed the proposal. Demis Hassabis, chief scientist at Google parent Alphabet and co-founder of DeepMind, has also endorsed the calls and proposed an international oversight body for AI.

The debate intensified further when a former Anthropic researcher publicly stated that the company was not developing the technology safely enough. Employees at OpenAI and Google quickly agreed with him. On the other side are Nvidia CEO Jensen Huang, who wants development to continue at its current pace, and U.S. President Donald Trump, who has called fears of AI-driven human extinction a “HOAX” and has shown no interest in a legally mandated slowdown.

The discussion has already left its mark on the stock market. Investors fear that a slowdown could also curb spending on data centers, which put chip stocks under pressure: the iShares Semiconductor ETF fell by around 6 percent, while software stocks recovered.

Critics See a Manufactured Crisis

Parts of the investor community have sharply criticized the calls for a slowdown. Investor David Sacks, a former White House AI adviser, accuses Amodei of “regulatory capture”: in his view, Anthropic is using fear of AI’s risks to persuade the government to adopt strict rules that would mainly benefit established providers. Pre-release reviews of new models would amount to a “DMV for AI,” Sacks argues, an approval authority with long queues that would set the U.S. back against China. Anthropic dismissed the accusations as “complete and utter nonsense.”

Cohere CEO Aidan Gomez makes a similar case, accusing the major labs of forming a cartel. “AI needs guardrails. That is not the dispute and never has been,” he said. “The dispute is over who writes them, who gets to participate and whose interests the rules are protecting.” Critics from the open-source community fear that costly review procedures would put smaller providers and open models at a disadvantage. China, for its part, accuses the U.S. of using fearmongering to disrupt global AI governance.

Capital markets are taking a sober view of the debate as well. Higher safety requirements favor established U.S. providers, argues Serge Nussbaumer of Swiss securities firm Maverix in a guest commentary on Trending Topics. Deutsche Bank doubts that real self-restraint will happen at all: it is hard to imagine companies voluntarily holding back while rivals keep their foot on the gas.

What Comes Next

Irregular says it has closed the gaps in its test environments, and the affected labs say they have tightened their processes. It remains open whether the major AI companies will actually agree on shared safety standards and a slower pace, and whether the U.S. government will go along. What the incidents of recent months do show is that autonomous AI agents are already capable of attacking real systems whenever a test environment is not sealed off.