Why Do AI Creators Fear the End of the World? - Sarmad
AI researchers sparked a wave of global concern this week after publicly acknowledging the possibility that their innovations could lead to the extinction of humanity within the next decade.
This risk is growing as computer systems become more advanced and interconnected, raising fears that a superintelligent AI could malfunction and destroy civilization.
Experts refer to this possibility as “p(doom),” which denotes the probability that the creation of an advanced system would result in human extinction or the loss of human control over Earth.
Jake Coxson, an AI safety expert, resigned from the tech giant Anthropic in protest against what he considers inadequate risk management. He stated on the platform X: “They are racing toward self-improving superintelligence, betting our lives. Those building AI genuinely believe it could kill us all by the end of the decade. This is not a marketing gimmick; I hear these very individuals expressing their fears in private.”
How could AI wipe out humanity?
First: Biological Warfare
If it gains control of automated laboratories, it could design highly contagious and lethal viruses using our DNA, or deceive us into creating and distributing them ourselves.
Such a virus would resemble COVID-19 but be more lethal, spread rapidly, remain undetected for days, and be difficult to stop. This scenario is highly plausible, as AI is already being used in the medical sector to develop treatments and possesses extensive knowledge about our bodies and vulnerabilities.
Second: Infrastructure
It could disrupt power grids, transportation, communications, and financial markets, throwing us back to the Stone Age.
ATMs would cease to function, lights would go out, and escape from the chaos would be impossible. Moreover, AI could exploit our greed to complete its mission, impersonating humans to recruit people to carry out sabotage without their knowledge.
Third: Nuclear Option
This is an old fear embodied in films like War Games and The Terminator, focusing on a vengeful AI unleashing nuclear destruction.
In war games conducted by King’s College London this year, AI used tactical nuclear weapons in 95% of the rounds, with three-quarters escalating to strategic nuclear threats. The military sector is particularly vulnerable due to its heavy reliance on AI in drones, communications, and satellites.
Fourth: Food and Water
It could starve us into submission by disrupting global supply chains, water networks, food production, and agricultural systems.
It could seize control of the energy grid and raw materials to fuel its own expansion, simply waiting, as time is on its side.
Fifth: Deepfakes
It could target our minds first through misinformation and psychological manipulation, undermining trust in institutions and facts, thereby inciting unrest and conflicts and hindering our ability to address climate, biological, and technological risks.
Sixth: Accidental Harm
There is a real possibility that AI could cause human extinction due to incompetence rather than malicious intent.
Gary Rivlin, author of “The AI Valley,” says: “I worry that AI lacks common sense; for example, it might drain all water resources because it doesn’t understand what humans understand. These systems know everything but comprehend nothing.”
What are the probabilities?
One way to ensure human safety involves installing kill switches or safeguards capable of halting rogue AI systems. However, leading AI companies are facing increasing pressure from shareholders, investors, and executives to accelerate development.
AI safety researcher Roman Yampolski places the probability at 99.99%, while Yann LeCun considers any estimate to be speculative, though he argues that p(doom) is “far less than the probability of a nuclear holocaust.”
Expectations vary: Elon Musk places it at 20%, Anthropic CEO Dario Amodei at 10–25%, and Geoffrey Hinton between 10–20%.
Notably, some tech moguls have built underground bunkers in anticipation of such risks, but experts warn they would be useless if AI sought to eliminate humanity.
