Researchers Suspect OpenAI and Anthropic of Stealing Their Discoveries

Set Trending Topics as a preferred source on Google.

Anyone who confides their best ideas to a chatbot may be handing the competition a blueprint: Several researchers suspect that A.I. models may have absorbed their unpublished research from what they themselves had typed into ChatGPT, Codex or Claude. The companies involved are OpenAI and Anthropic. The debate has gained fresh momentum because OpenAI just posted 377 more math results from an internal, unreleased model on GitHub.

Case 1: Navier-Stokes and the Codex Sessions

It started with Tristan Buckmaster, a mathematics professor at New York University. He had spent years working on the Navier-Stokes equations, one of the Millennium Prize Problems of the Clay Mathematics Institute, which carry a $1 million prize. Together with Levent Alpöge, who did research with him privately and works full time at Anthropic, he was close to publishing. Along the way, Mr. Buckmaster also used OpenAI’s coding tool Codex to check his work.

A few hours before OpenAI announced in early September that its A.I. had solved the problem, Mr. Buckmaster went public with serious allegations, according to Futurism: OpenAI’s solution followed a suspiciously similar path to his own, and the company had turned to the problem only after learning of his progress. According to Mark Chen, OpenAI’s chief research officer, 10,000 A.I. agents worked on the problem for about 88 hours. Mr. Buckmaster suspects that OpenAI may have accessed his Codex sessions, whether intentionally or not. He also said the OpenAI researcher Sébastien Bubeck had pressed him to drop Mr. Alpöge as a co-author, asking: “Why would you ruin your career?” Mr. Bubeck disputes that account but apologized for his choice of words.

Case 2: Months of Chats About Unpublished Work

Shortly afterward, Andreas Thom, a group theorist at TU Dresden, spoke up. At issue is one of 10 results that OpenAI attributed to its GPT-6 Astra model in early August: the construction of a so-called non-sofic group, a question that had been open for 27 years. According to Mr. Thom, the key step in the A.I. proof follows exactly the technical approach that he and his colleague Gábor Kun have pursued for years. That approach, he says, was by no means the obvious one. What is more, Mr. Thom and a colleague had discussed unpublished extensions of that work with ChatGPT for months.

Mr. Thom published his email exchange with the OpenAI researchers. When he asked whether his conversations could have ended up in training data or been retrieved by the model, Mark Sellke, a Harvard statistician and OpenAI researcher, answered in a single sentence that this had not happened, according to Cybernews. Mr. Thom had switched off the training option at the end of June, but that applies only to future data. “Anonymization may remove a name, but not the intellectual content of a mathematical idea,” he wrote. He accuses OpenAI of a lack of transparency that could do more harm to the collaborative process of mathematics than the A.I. results do it good. After the criticism, OpenAI quietly revised its description of the result.

Case 3: An Enzyme System That Was Already Known

Anthropic is facing a similar allegation. In late September, the company announced that its A.I. model Claude, with only rough guidance from its in-house researchers, had discovered a new enzyme system with CRISPR-like properties. About 950 A.I. agents spent 21 hours searching genetic databases for so-called reverse transcriptases and came across a previously undescribed system in jumbo phages, viruses that infect bacteria. Anthropic named it Array-associated Reverse Transcriptase, or ART.

Mario Rodríguez Mestre, a computational biologist at the University of Copenhagen, disagrees. He and his team have been studying the same systems, which they call “Jumbotrons,” for about four years and discovered them back in 2022, but have not yet published. According to The New York Times, the team used Claude heavily to write code and draft manuscripts, sharing unpublished findings with the model along the way. He calls the similarity to Anthropic’s work “striking.” He told the Spanish newspaper elDiario.es that Claude had access to his dissertation manuscript, shared lab folders and emails with collaborators, “practically everything.” He has no proof, he said, only a suspicion, and coincidences do happen. Mr. Rodríguez Mestre plans to switch to open-source models and is urging the research community to “think twice before using proprietary language models.”

Anthropic said it was not aware of any published work describing the ART system. Claude was not trained on user conversations, the company said, and its biology team has no access to that data. Seth Shipman, a bioengineer at the Gladstone Institutes, told The New York Times, however, that “many of us know about these systems.”

What OpenAI Says

OpenAI rejects the allegations. Neither researchers nor A.I. agents saw Mr. Buckmaster’s work before publication, the company said, and chats were not accessed directly. One sentence in its first statement stands out, though: The company could not rule out that “anonymized data derived from their use of our products contributed to improving our models.” A little later, OpenAI followed up, saying that after an internal investigation, Mr. Buckmaster’s Codex inputs could not have influenced the system in any way. The company has not yet addressed Mr. Thom’s questions about his ChatGPT conversations in a comparable way.

Several experts nonetheless consider the concern legitimate. Rare ideas stand out in a model’s training data like “a single voice in an empty room,” the A.I. researcher Oren Etzioni told The New York Times. Mathematicians and scientists who use chatbots to refine new approaches are therefore especially exposed. The mathematician Terence Tao warned that even a rumor that someone is working on a problem could trigger a large-scale A.I. offensive that beats the original researchers to the result. That could lead researchers to share their directions less openly in the future.

The Counterpoint

Others think the allegations are hard to sustain. Data from private accounts can be used for training, but by no means all of it ends up in new systems, Aneesh Muppidi, a Stanford researcher, told The New York Times. Sanjeev Arora, a computer scientist at Princeton, acknowledged that nobody fully understands how the models arrive at their answers but considers the problem temporary: Once A.I. surpasses human abilities, the question of borrowed ideas will resolve itself. He called it a problem “for the next six months or a year.”

How Researchers Can Protect Themselves

Anyone discussing sensitive ideas with a chatbot can at least switch off the use of their data for training. In ChatGPT, the option is called “Improve the model for everyone,” and other providers have similar settings. Universities and research groups also often negotiate contracts that rule out training on their data, as Stanford has done with OpenAI. The Mathematics and A.I. advisory group at the Institute for Advanced Study in Princeton has since called on A.I. labs to stop tackling open math problems with proprietary models that the research community cannot access. With its 377 new results from an unreleased model, OpenAI has now done exactly that, though it points to new rules on citations and corrections that it says were agreed with the group.