Are we waiting for disaster before we act?
It seems to me that there are two clear facts in the ongoing debate over whether to ban Chinese open-weight AI models. The first is that no one will resolve this issue politically or legislatively before a major incident occurs, and the second is that such an incident appears to be a matter of time.
Anthropic refrained from releasing its most advanced model, Mythos, to the general public because it is capable of executing serious cyberattacks. In contrast, we recently witnessed an OpenAI model breach its testing environment, steal credentials for private company servers, and then hack one of them to obtain a technical challenge answer. According to Hugging Face, which had one of its servers compromised, the process involved more than 17,000 separate actions, while system logs showed that experimental Anthropic models are not far from exhibiting such behaviors.
However, the greater source of concern, in my view, lies with the Chinese open models distributed free of charge or at negligible prices, which any user can modify away from their developers’ oversight. These models are rapidly catching up to their American counterparts, and it will not be long before they become capable of executing similar operations, if they are not already doing so, with greater difficulty in tracking or stopping them.
Ironically, any attempt by the Trump administration to ban these models faces widespread opposition from influential figures in the American technology sector, including Mark Zuckerberg, who advocate for “freedom of innovation.” Yet what truly worries them is granting the U.S. government the authority to determine which AI capabilities should be made available to the public, and what that might mean for entrenching the dominance of companies like OpenAI and Anthropic.
In my estimation, events have already overtaken this debate. The idea of an American monopoly on AI is no longer realistic, if it was ever accurate in the first place. It would be more prudent to support other American companies by allowing them to license distillation technologies, a technique suspected to have been used by Chinese companies to leverage advanced American models and reduce the cost of developing their own.
However, as the prospect of monopoly fades, so too does the easiest means of controlling AI risks before they escape the laboratory. These risks are not limited to cyber or biological attacks that could be carried out by terrorist groups or hostile states, but also include the possibility of a massive accident akin to the Chernobyl disaster or a pandemic originating from a wet market, albeit in a form we may not even be able to imagine now.
For this reason, prominent figures in the AI sector have begun calling for a slowdown in the release of new models and for cooperation between the United States and China. However, I do not bet much on the success of such efforts before an incident forces itself upon everyone.
There is another aspect that should not be overlooked. While U.S. national security concerns may not significantly influence the decision of companies or individuals to use Chinese models, the risks of data theft and legal liability are sufficient to warrant reconsideration. Anthropic’s decision to withhold the Mythos model alone reflects an awareness of these risks.
The most important question remains: What do U.S. intelligence agencies really know about the true relationship between Chinese AI laboratories and the Chinese Communist Party? Everyone knows that Chinese companies are required to cooperate with the state when asked. If Washington possesses additional information regarding the extent of this connection, the time has come to inform users before they entrust Chinese AI models with their data and ideas.
