Anthropic safety researcher quits, citing fears of uncontrollable superintelligence

Former OpenAI and Anthropic researcher Jacob Coxon has quit the AI industry, accusing both companies of “racing straight to self-improving superintelligence and gambling with our lives.”

Coxon warned that researchers inside leading labs privately fear advanced AI “could kill us all by the end of the decade,” and called for government intervention, coordination between labs and potentially a temporary halt on making models more powerful.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.