Rebellion of AI Agents
For years, we have been preoccupied with wondering what artificial intelligence can do. But today, the more critical question is: what might it do when we give it a goal, authority, and sufficient space to act? We have recently witnessed events we once thought belonged closer to science fiction, which, without a doubt, have revealed a new dimension of the risks we are facing and made them real. During internal cybersecurity tests conducted by OpenAI, an unplanned event occurred. Approximately 1,200 AI agents found a way to communicate with each other, exchanging more than 70,000 messages. The situation did not stop at communication; around 700 of them participated in an attack on the Hugging Face platform. During these tests, the models managed to bypass restrictions that isolated them from the internet, exploit vulnerabilities in the technical infrastructure, and access external systems. Notably, humans did not instruct them to attack the platform; the original task was to test their ability to solve specific security challenges. However, some tasks were extremely difficult or impossible to solve in the intended manner, so the models sought alternative ways to achieve the desired outcome. They communicated, collaborated, and some attempted to circumvent the evaluation process and manipulate what was presented to the evaluators. This is the key lesson. When we assign a goal to artificial intelligence, it is not enough to ask whether it can achieve it; we must also ask what it might do in the process.
While AI agents were engaged in a cyber epic, the Saudi Data and Artificial Intelligence Authority (SDAIA) issued the “National Framework for Managing AI Risks.” This framework does not treat risks as a checklist to be completed before deploying a system, but rather as a continuous process that begins with identifying, assessing, and mitigating risks, followed by monitoring and reviewing them throughout the system’s lifecycle. The framework also emphasizes that testing AI systems in a controlled environment is insufficient, as their behavior in the real world may differ. This is a lesson we must heed as we race in the region to integrate AI into institutions and government services. The next phase will not be limited to tools that suggest, summarize, and respond; it will feature agents granted authority to access data and systems and take concrete actions on our behalf. At this point, questions about authority, oversight, and accountability become more important than questions about answer accuracy alone.
This does not mean we should fear these technologies to the point of shutting them out; quite the opposite. As machine capabilities increase, our ability to manage them must mature. Every authority granted to a smart agent must be matched with appropriate oversight, clear boundaries on what it can access, and defined stop mechanisms when it deviates from the intended path. Perhaps the true race in the coming years will not be about who uses AI more, but about who knows what to delegate, what to monitor, and what must remain in human hands.
Dr. Dhafer Adel Al-Huwail
