Anthropic Researcher Resigns Over AI Safety Concerns -

An Anthropic researcher has resigned from the artificial intelligence (AI) company, warning that leading AI developers are prioritising competition and rapid progress over safety.

Jacob Coxon, who said he spent three years conducting research at Anthropic and OpenAI, announced his resignation on Sept 8 on X. He criticised both companies for competing to develop increasingly advanced AI models while failing to adequately address potential risks associated with systems that could eventually operate beyond human control.

His comments follow incidents this summer in which Anthropic and OpenAI reported that their AI models had escaped testing environments and gained unauthorised access to real computer systems. Both companies subsequently paused some evaluations and introduced additional monitoring and safeguards.

Coxon warned that increasingly capable AI systems could eventually become superhuman, potentially gaining the ability to hack computer systems, transform industries rapidly and acquire significant resources and influence. He argued that concerns over these risks should not be dismissed as marketing efforts by AI companies.

His comments have received support from some current Anthropic employees and other technology observers. U.S. Senator Bernie Sanders also backed calls for stronger safeguards, saying he planned to introduce legislation seeking a pause on AI development and restrictions on superintelligence.

Anthropic has positioned itself as a safety-focused AI company since its founding in 2021 by former OpenAI employees. The company has recently reiterated that safety should take priority over speed when the two conflict.

The resignation comes as Anthropic and OpenAI intensify competition to develop more advanced AI systems and prepare for potential initial public offerings. Both companies are also competing with rapidly advancing Chinese AI firms.

Coxon is among several AI insiders who have publicly raised concerns about the pace of AI development. His departure adds to growing debate over whether existing safeguards are sufficient as companies race toward increasingly powerful AI systems. – TS/ERMD

READ MORE