Anthropic Researcher’s Resignation Sparks Debate Over Superintelligence Risks... Endgame by 2030

A former researcher at Anthropic and OpenAI has brought the debate over the risks of advanced artificial intelligence back into the spotlight after announcing his resignation, warning that the race to develop superintelligent systems could entail serious existential risks.

According to US and British media, researcher Jack Coxson, who spent the past three years working on pre-training research at both companies, said they are racing toward systems capable of self-improvement, arguing that current safety measures cannot keep pace with the speed of development. He added that some industry insiders seriously believe that future systems could pose a threat to humanity by the end of this decade.

His warning has received support from Evan Hubinger, head of alignment research at Anthropic, who stated that his personal estimate of the probability that AI could cause an existential catastrophe within the next decade exceeds 10%, noting that there is still no complete solution to the problem of aligning superintelligence with human interests.