Anthropic safety researcher quits, citing fears of uncontrollable superintelligence

September 9, 2026

Former OpenAI and Anthropic researcher Jacob Coxon has quit the AI industry, accusing both companies of “racing straight to self-improving superintelligence and gambling with our lives.”

Coxon warned that researchers inside leading labs privately fear advanced AI “could kill us all by the end of the decade,” and called for government intervention, coordination between labs and potentially a temporary halt on making models more powerful.

An Anthropic researcher is quitting the artificial-intelligence industry over fears that the lab and its competitors are racing to build systems that could spiral out of control and destroy humanity. https://t.co/oB7tWEwWkH
— The Wall Street Journal (@WSJ) September 8, 2026

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
— Jacob Coxon (@hilbertspaess) September 9, 2026

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear…
— Jacob Coxon (@hilbertspaess) September 9, 2026

A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else…
— Jacob Coxon (@hilbertspaess) September 9, 2026

Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.
— Jacob Coxon (@hilbertspaess) September 9, 2026

I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on…
— Jacob Coxon (@hilbertspaess) September 9, 2026

If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” – or take this moment…
— Jacob Coxon (@hilbertspaess) September 9, 2026

POPULAR

Leave a Reply Cancel reply