Debate

Anthropic Doesn’t Want to Ban Open-Weights Models — it Wants a Chip Ban on China

Dario Amodei, Chief Executive Officer and Co-Founder, Anthropic. © World Economic Forum / Sandra Blaser
Dario Amodei, Chief Executive Officer and Co-Founder, Anthropic. © World Economic Forum / Sandra Blaser

Set Trending Topics as a preferred source on Google.

It is one of the sharpest conflicts to hit the US tech industry this year: ever since reports emerged that US officials are considering barring domestic companies from using Chinese open-weights models, the industry has been taking public positions. On July 24, an initial group of 25 companies and organisations published the letter “Open Weights and American AI Leadership,” which warns Washington against “premature restrictions” on freely downloadable AI models. Nvidia CEO Jensen Huang shared the paper in his first-ever post on X – an unusually personal intervention in tech policy for him.

The signatories include Nvidia, Microsoft, Meta, IBM, Dell, Palantir, Mistral, Hugging Face, Mozilla, Andreessen Horowitz and Y Combinator. Within days, the number of backers doubled to around 50, with OpenAI, Google, AMD, Cisco and GitHub among the late arrivals. Anthropic – alongside Amazon – stayed out. Among the leading model providers, Anthropic is now the only holdout, which has earned the company accusations that it wants to slow down open models out of commercial self-interest.

“Public good,” but no safety argument

On Monday, Amodei responded with a blog post on Anthropic’s website. His central point: Anthropic has never called for a ban on open-weights models. Open models without dangerous capabilities are, in his words, a public good that benefits businesses, developers and researchers and costs nothing beyond the compute needed to run them. “Anthropic has never advocated for a ban on open-weights models,” Amodei wrote.

By his own account, Amodei thinks little of protectionist bans – not out of any regulatory sympathy for open models, however, but because he considers them ineffective. A usage ban for US companies does not address the actual risks, he argues, because malicious actors are generally not legitimate US businesses. He concedes that such a ban would shield American AI firms from competition, but says that has never been his goal.

Two scenarios worry him, as he describes it. The first is authoritarian governments – above all the Chinese Communist Party – building more powerful models than the US and using them to achieve military superiority or far-reaching repression. Whether such models are released openly or kept closed is irrelevant here, he argues; the most dangerous model could be one trained in secret and handed directly to the military and the state security apparatus.

The second is the misuse of powerful models for cyber or biological attacks, along with alignment problems. Here he does see an elevated risk with open weights – regardless of country of origin – because safeguards are hard to enforce, usage is barely monitorable, and once weights have been released they cannot be withdrawn.

Three measures instead of a ban

Rather than a ban, Amodei names three levers that he says Anthropic has advocated for years:

Chip exports: No powerful chips and no chipmaking equipment for China, plus tougher action against smuggling and workarounds. Given its limited domestic production capacity, China cannot build stronger models than the US without American chips.

Distillation: Action against industrial-scale distillation, meaning the compute-efficient replication of model capabilities. This allows China to partially circumvent chip restrictions and close to within a few months of the US frontier. That many of the companies involved release open models is secondary, he argues.

Safety testing: Mandatory pre-release testing of all sufficiently capable models – open and closed alike – for cyber, bio and alignment risks. Smaller models from startups and academia would be exempt. Such a regime would only be effective globally, according to Amodei, meaning China would have to be on board.

Where he disagrees with the industry letter

Amodei agrees with large parts of the letter: open weights expand access to the AI economy, strengthen competition at least in some use cases, and give customers more control. But he rejects two central claims – that open models necessarily make it easier to develop safeguards, and that broad access to capabilities helps defenders more than attackers. The opposite is at least as likely, he argues. He cites biology as an example: a sufficiently capable model could quickly weaponise pathogens with pandemic potential using widely available materials, while countermeasures require years of operational work even in the best case, as Operation Warp Speed showed. Whether open models raise the risk should be settled empirically through testing, he says, rather than assumed in advance.

Commercial interests on both sides

Neither side is acting free of interest. The letter’s signatories include chip and infrastructure providers that benefit directly from the wide spread of open models, as well as labs that publish open weights themselves. Anthropic, in turn, earns its money exclusively from closed frontier models and is among the companies most directly affected commercially by distillation – it has recently made such accusations publicly. The current debate was triggered by the Chinese open-weights model Kimi K3, which approaches the performance of Western frontier models at significantly lower cost.

Politically, it remains open where the US government will land. A ban on Chinese open models is still on the table, while at the same time there is movement towards mandatory safety testing – the very instrument Amodei is betting on, and the one where he sees the largest common ground with the rest of the industry.

Rank My Startup: Erobere die Liga der Top Founder!
Advertisement
Advertisement

Specials from our Partners

Top Posts from our Network

Deep Dives

© Wiener Börse

IPO Spotlight

powered by Wiener Börse

Europe's Top Unicorn Investments 2023

The full list of companies that reached a valuation of € 1B+ this year
© Behnam Norouzi on Unsplash

Crypto Investment Tracker 2022

The biggest deals in the industry, ranked by Trending Topics
ThisisEngineering RAEng on Unsplash

Technology explained

Powered by PwC
© addendum

Inside the Blockchain

Die revolutionäre Technologie von Experten erklärt

Trending Topics Tech Talk

Der Podcast mit smarten Köpfen für smarte Köpfe
© Shannon Rowies on Unsplash

We ❤️ Founders

Die spannendsten Persönlichkeiten der Startup-Szene
Tokio bei Nacht und Regen. © Unsplash

🤖Big in Japan🤖

Startups - Robots - Entrepreneurs - Tech - Trends

Continue Reading