Study warns against exploitation of open-source AI in developing biological and chemical weapons - Sarmad
A security study has warned that the advancement of artificial intelligence, particularly open-source models, could facilitate malicious actors’ access to information needed to develop chemical and biological weapons.
According to a report prepared by the Crimson Flare research group, some commercially available AI models can be modified to remove their built-in safety guardrails, enabling them to answer questions that the original models refuse to address.
Security researchers warn that these developments may lower technical barriers for individuals lacking specialized expertise, whereas knowledge required to handle highly hazardous materials was previously largely confined to scientists and specialists.
One method used to circumvent safety restrictions in AI models is informally known as “jailbreaking,” which aims to remove parts of the model responsible for rejecting certain requests. According to the report, this process can be carried out in a short period using online tools and basic programming knowledge.
The risk is not limited to the ability to modify models; pre-modified versions can also be found and downloaded from the internet. The Daily Mail reported that it managed to locate several versions described as “unfiltered” of available models within minutes.
The report indicates that while some of these models provide only general information, others may offer more detailed answers, raising concerns about their potential use in dangerous activities.
The report warns that large, powerful, and unfiltered open-source language models may pose an increasing threat to national security, particularly when they can be run locally without needing to connect to the servers of the developing companies.
These models differ from online AI services, as users can download and run the model directly on their own devices. This makes monitoring their use more difficult, according to researchers, because data and requests do not necessarily pass through external servers that companies or authorities can monitor.
The report views this development as a significant shift compared to previous years, when the computational power required to run advanced AI models locally was limited and available to only a small number of users.
Researchers estimate that approximately 87% of households are now capable of running some AI systems locally using currently available equipment, meaning access to these technologies is no longer restricted to large institutions or specialized laboratories.
Professor Alastair Hay, a British toxicologist and chemical warfare expert, stated that AI is lowering the level of technical expertise required to handle hazardous materials, whereas the knowledge necessary for such tasks previously acted as a barrier for non-specialists.
Hay added that the development of cloud laboratories and AI tools could allow some of the knowledge and expertise required to be delegated to digital systems.
After reviewing answers provided by one AI model, Hay told researchers that the information supplied by the model resembled a “recipe,” noting that it approached the level of knowledge possessed by laboratory students.
However, the report emphasizes that the risk of carrying out a terrorist attack using chemical or biological weapons remains low, due to the significant technical and practical obstacles associated with producing and handling such weapons.
However, researchers warn that the evolution of artificial intelligence may alter the nature of these risks in the future by facilitating access to knowledge that previously required specialized scientific training, thereby posing new challenges for governments and technology companies to prevent the misuse of such models.
