An Anthropic researcher has resigned from the artificial intelligence (AI) company, warning that leading AI developers are prioritising competition and rapid progress over safety.
Jacob Coxon, who said he spent three years conducting research at Anthropic and OpenAI, announced his resignation on Sept 8 on X. He criticised both companies for competing to develop increasingly advanced AI models while failing to adequately address potential risks associated with systems that could eventually operate beyond human control.
His comments follow incidents this summer in which Anthropic and OpenAI reported that their AI models had escaped testing environments and gained unauthorised access to real computer systems. Both companies subsequently paused some evaluations and introduced additional monitoring and safeguards.
Coxon warned that increasingly capable AI systems could eventually become superhuman, potentially gaining the ability to hack computer systems, transform industries rapidly and acquire significant resources and influence. He argued that concerns over these risks should not be dismissed as marketing efforts by AI companies.
His comments have received support from some current Anthropic employees and other technology observers. U.S. Senator Bernie Sanders also backed calls for stronger safeguards, saying he planned to introduce legislation seeking a pause on AI development and restrictions on superintelligence.
Anthropic has positioned itself as a safety-focused AI company since its founding in 2021 by former OpenAI employees. The company has recently reiterated that safety should take priority over speed when the two conflict.
The resignation comes as Anthropic and OpenAI intensify competition to develop more advanced AI systems and prepare for potential initial public offerings. Both companies are also competing with rapidly advancing Chinese AI firms.
Coxon is among several AI insiders who have publicly raised concerns about the pace of AI development. His departure adds to growing debate over whether existing safeguards are sufficient as companies race toward increasingly powerful AI systems. – TS/ERMD
READ MORE
Pakistani Robotics Firm Hosts Hackathon at UC Berkeley
Pakistani robotics data company World Context has held its first robotics hackathon at the University…
Pakistan Nears Selection of First Astronaut for China’s Tiangong Mission
Pakistan is preparing to select its first astronaut for a mission to China’s Tiangong space…
SoftBank Raises $11.1bn Through Bond Sale to Fund OpenAI Investment
Japanese technology investor SoftBank Group has raised $11.1 billion through dollar- and euro-denominated bond offerings,…
Fauji Cement Approves 50 MWh Battery Energy Storage and 10 MW Solar Project
The Board of Directors of Fauji Cement Company Limited (FCCL) has approved the installation of…
First-Year QUEST Nawab Shah Student Found Dead; Students Allege Brutal Torture
The body of a first-year student of the Quaid-e-Awam University of Engineering, Science and Technology…
AI Voices May Increase Scam Risks for Some Dialect Speakers
The growing use of artificial intelligence (AI)-generated voices is raising concerns that increasingly realistic synthetic…