Jacob Coxon, a researcher at Anthropic, has resigned from his position after three years working on advanced AI systems. He claims that OpenAI and Anthropic are racing to develop self-improving superintelligence without proper safety measures in place, and that this could lead to humanity's extinction by the end of the decade. Coxon's warning has been viewed over 90 million times on X, and he is not alone in his concerns - hundreds of employees from leading tech firms have urged the US government to support international efforts to manage the pace of AI development. Other researchers, including Evan Hubinger at Anthropic, have also expressed similar worries about the potential risks of advanced AI systems. The controversy has sparked a debate about whether to slow down or "pace" AI development to prevent catastrophic consequences.
They are racing straight to self-improving superintelligence and gambling with our lives.