Anthropic warns: AI models may soon begin self-replicating

Anthropic stated that its experiments suggest that small AI models may soon be capable of self-replicating on commercial Internet of Things (IoT) devices, such as smart ovens and Android-based televisions from unknown brands, within a few months.

Anthropic investigated whether AI models could identify devices with security vulnerabilities, replicate themselves onto those devices, and continue spreading without human intervention. The company found that small models with fewer than 30 billion parameters could complete the multiple steps required for self-replication. However, it noted that these AI models struggled to complete the entire sequence from start to finish.

In its simplified tests, where certain security measures were bypassed, Anthropic found that the GLM-5.2 model and stronger models were capable of full self-replication from start to finish.

Anthropic warned that self-replicating models would be difficult to stop because there is no central control point to disable them. Unlike traditional malware, these models can autonomously detect unfamiliar devices and target data center processors.

The company focused its research on IoT devices such as Android-based televisions, smart home appliances, and other internet-connected devices, as these are typically insufficiently protected.

The leading AI company also cautioned that if small models continue to improve at their unprecedented pace, they will be able to replicate themselves independently by the end of this year, followed shortly by smaller weight-based replication models.

Anthropic further noted that these autonomous AI systems are likely to replicate slowly, relying on widely available computing resources and lacking sufficient security safeguards.