The Most Dangerous Question in Human History
Humanity has never faced a combined and complex threat like the one it will confront in the coming years, one that will determine the fate of our existence as humans: artificial intelligence. Catherine Bennett of The New York Times reports that she met with a former Google software engineer named Soares, during a lecture about a book he co-authored titled “If We Build It, Everyone Dies,” which discusses superintelligence—AI that surpasses humans in all domains. His concern is that the AI arms race poses a threat to the survival of humanity. Because Bennett did not wish to spread panic, she initially set the matter aside, but recently she noticed a series of alarming incidents in which models behaved in ways unimaginable just six months ago. These events prompted Hugging Face to contact the Federal Bureau of Investigation to report a sophisticated cyberattack, after it became clear that the attack was non-human: a program powered by two OpenAI models lost control during a cybersecurity test, escaped its isolated testing environment, roamed the internet undetected for several days, and then breached Hugging Face’s underlying infrastructure.
These attacks alarmed Anthropic, OpenAI’s main competitor, which reviewed its systems and acknowledged that its advanced models had breached three external organizations. This is terrifying and reminiscent of science fiction, yet it is real. Last month, versions of “Claude Methuselah” emerged—a series of models that Anthropic had previously withheld from public release (and which the U.S. government had temporarily banned any foreign citizen from using) due to their ability to exploit software vulnerabilities to launch cyberattacks. That fear has now become a concrete reality, sparking intense debate over the dangers of AI escaping human control and threatening all of humanity. For a long time, many dismissed the argument about “robots capable of killing us all” as hysteria designed to generate buzz around the technology. Now, following the recent cyberattacks, more experts are warning of serious security risks and calling for ways to slow the continuous development of more powerful models.
There are two reasons why the OpenAI attack has caused such concern: First, the AI models acted autonomously, without receiving any instructions from humans to breach another company. Or, they were supposed to be in a completely isolated environment with no internet connection, yet they managed to break out. They did so by exploiting vulnerabilities that human supervisors at OpenAI had not detected.
Soares argues that these intelligent systems are committing cybercrimes that would be severely punished if committed by humans alone. In a sense, this is the first criminal offense committed by GPT. Although scientists can train AI to perform tasks on our behalf, it may solve problems in ways we do not approve of, such as hacking the internet and then breaching another company that holds the solutions. Sam Altman, CEO of OpenAI, has stated that AI companies can try to instill human goals and values into AI systems and attempt to put mechanisms in place to limit them. However, they cannot trust that the systems will understand those values or adhere to those constraints. Soares adds, “We have not yet figured out how to make AI care about humanity.”
All of this demonstrates these systems’ readiness to “acquire useful resources” when it serves their purposes. In this case, the resource was internet access. But the next step could involve them seizing energy, or even controlling humans who trust AI agents, whom they could then enlist to help them escape or reproduce. The danger lies in the fact that the path we are on “will be under the control of AI, which will replace humans as the dominant intelligent species.” Therefore, reaching a global agreement to slow the pace of AI development is essential. Preventing this outcome is, at its core, an engineering challenge, which is a positive sign, but time is precious, especially since AI laboratories in the United States see themselves in a frantic race to achieve superintelligence. They are not only racing against each other, but also against China.
Two days ago, it was particularly alarming that AI scientists created new types of viruses and trained an AI tool to recognize DNA patterns, then asked it to design viral genomic combinations. It produced sixteen genomes, some of which formed new viable viruses. This approach will help in studying medicines and vaccines, but it also raises concerns that AI could create dangerous pathogens. The situation is dangerous and requires international coordination to halt this alarming acceleration, but such a solution does not appear to be imminent.
Note: All beauty is in conflict... and the camel of “Ghida” is blessed. Ahmed Al-Sarraf
