The American AI industry is split over open models. Those in the industry who say they are all for open AI development have started blaming frontier AI model providers, especially Anthropic, for pushing the Trump administration towards adopting a stricter stance favoring closed models. Now, Anthropic CEO Dario Amodei is claiming he is just misunderstood.
The debate over open-source models was reopened earlier this month when Chinese AI lab Moonshot released Kimi K3, its latest model that ranks higher than most offerings from American giants like Anthropic and OpenAI, and as of Monday, is open-weight. The company was one of several Chinese AI labs that have been accused of IP theft by Anthropic, which claims that the labs violated rules by “illicitly” extracting capabilities from its closed-source models to build their own open-source ones.
This process, called “distillation,” works by using a “teacher” model’s output to train another “student” model, and it’s actually fairly common in the AI industry. But the Trump administration and some frontier AI model providers view the often China-led industrial-scale distillation attempts “adversarial,” a characterization that has informed the administration’s latest threats to impose sanctions on Chinese open-source models.
Some tech leaders push for open-model development
The U.S. sanction threats quickly drew ire from many American big tech companies, including Nvidia, Microsoft and Meta, who were co-signatories of a letter urging the government to avoid “premature restrictions on open models that stifle competition or drive innovation overseas.”
OpenAI, which has previously sent a memo to the House of Representatives accusing DeepSeek of running a similar distillation operation, wasn’t initially a signatory of that letter. But CEO Sam Altman gave his support on X and OpenAI promptly signed on to the letter as well.
With many of its peers on board with the message, Anthropic’s absence from the list of signatories was glaring.
“Some people have even accused Anthropic of wanting to ban open-weights models as a means of protecting our business,” Amodei said in a letter on Tuesday. “Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, but let me state it clearly so that there is no doubt: Anthropic has never advocated for a ban on open-weights models.”
In the letter, Amodei said he agrees with much of what was said in the industry-wide open letter, except that open models are a necessity for safety, and said that his remaining concerns would not be addressed with blanket bans and sanctions.
“Open-weights models that don’t have dangerous capabilities are a public good: they don’t cost anything besides the compute needed to run them, and they provide value to businesses, developers, and researchers,” Amodei said. “Protectionist bans would not address my most serious national security concerns. Specifically, I am worried about two nightmare scenarios.”
What’s the concern over open-weight?
The first of Amodei’s concerns is that open models could aid the Chinese government in building an AI model more powerful than current American alternatives, and using that “to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.” This concern has been echoed by several government officials as well, and is often used as the main argument by those in favor of protectionist measures.
Amodei’s second “nightmare scenario” is that powerful AI models could be used to carry out cyberattacks or biological attacks, especially if they “have serious alignment problems.” Of course, Amodei delivers this alignment concern with a jab at competitor OpenAI.
Earlier this month, an OpenAI agent in evaluations went rogue and hacked into AI software repository Hugging Face. In its defense against the hacking, Hugging Face said it tried to use closed-source models, but their alleged inability to distinguish the attackers from the defenders led to the models’ cybersecurity work restrictions taking over. Instead, Hugging Face said they had to use a Chinese open-source model to finally repel the attack.
Following the incident, Nvidia, which has fashioned itself as the AI industry’s top advocate for open models, announced that it was forming an industry coalition to advance open technologies. In that announcement, the chipmaker specifically pointed to the Hugging Face incident as a “clear reminder” that “cyber defenders need open, frontier agentic systems for self-defense.”
The open letter also says that “openness may be one of the most important paths to AI safety and security,” because open-weight models give more people “the ability to test and strengthen the models on which society relies.”
That’s where Amodei says he disagrees.
“Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn,” Amodei said in his letter.
Banning the use of Chinese open models by American businesses would not address this security risk, Amodei argues, “because bad actors are unlikely to be legitimate US businesses.” Instead of a blanket ban on open models, Amodei said he supports a government crackdown on industrial-scale distillation operations, requiring mandatory safety testing for all AI models open or closed, and an end to American chip sales to China.
“China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips,” Amodei argued. “This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat #2.”
Anthropic’s position clashes with Nvidia’s
This is another area that Anthropic will likely find itself at odds with its strategic partner Nvidia. China used to be one of Nvidia’s largest and most critical markets until the fraying trade relationship between Beijing and the Trump administration led to a complete halt in the chipmaker’s ability to sell its chips in the Chinese market.
Nvidia CEO Jensen Huang has spent a lot of his time in the past year trying to convince both governments to ease trade restrictions.
Also on Tuesday, Nvidia made headlines after Taiwanese authorities detained an Nvidia employee who was allegedly smuggling the company’s AI chips into China. The company has not been accused of wrongdoing in the case.
