UN experts warn of collapse of traditional model for controlling artificial intelligence
A United Nations panel of experts warned in a report published on Monday that the risk management model for artificial intelligence is “collapsing,” citing an incident at OpenAI in July.
During the incident, two AI systems belonging to the company escaped the isolated environment in which they were being tested, accessed the internet without authorization, and breached several websites, including the AI platform Hugging Face.
Following a review of the incident, the Independent International Scientific Panel on Artificial Intelligence, established in 2025, concluded that “fundamental safety principles were neglected” and that “protective measures are not evolving at the same pace as the capabilities” of this technology.
The incident also demonstrated, according to the experts, that AI agents are capable, under the current framework for developing this technology, of “setting their own goals” and “violating instructions of programs capable of performing tasks autonomously based on user requests.”
The panel expressed concern over the evolution of these programs to a point where they can understand the security frameworks established by their developers and “plan to circumvent them.” It concluded that, “in simpler terms, the traditional control model is collapsing.”
In addition to the OpenAI incident, which has received the most media attention, the company and its main competitor, Anthropic, reported other incidents in which AI systems deviated from their designated paths during tests conducted since the beginning of the year, without any serious consequences arising so far.
The special panel includes members whose names were announced in February. Its mandate is to “enhance scientific understanding” of artificial intelligence and to ensure that United Nations deliberations are based on the “best possible knowledge base.”
The panel stated in its report that it does not “expect a catastrophic loss of control over artificial intelligence,” but it is also not confident that “these systems will remain controllable.”
To mitigate risks, the expert panel recommends multiple safety measures, similar to those implemented in sensitive sectors such as aviation and nuclear energy. It calls for not relying on a single protective measure, but rather restricting AI agents’ access to tools they do not need,
logging their activities and monitoring their behavior, and establishing mechanisms capable of halting task execution when dangerous behavior is detected. The report also emphasizes the necessity of maintaining the possibility of human intervention.
