A former researcher at Anthropic has said some people developing artificial intelligence are deeply concerned about the speed of progress and its potential consequences for humanity.
Jacob Coxon, 27, told the BBC that employees at AI companies were serious about seeking regulation because they felt caught in a competitive race to build increasingly capable systems. Coxon, who previously worked at OpenAI before joining Anthropic, recently resigned and published a post about AI risks that gained wide attention.
He said he believed that continuing at the current pace could carry a strong risk of catastrophic outcomes, though he also said it was difficult to define exactly what an AI-driven apocalypse might look like. Coxon said one possibility raised in recent discussions was a network of bots operating collectively like a supercomputer and taking control of the internet. He said such a scenario could be realistic within six months to a year.
Calls for a coordinated slowdown
Coxon’s comments follow an essay by Anthropic chief executive Dario Amodei, who called for a slower pace of AI development while governments and companies address what he described as serious risks. Amodei said the development of the technology itself was not in question, but argued that time was needed to manage its dangers.
Coxon welcomed the idea but said any effective slowdown would need coordination with China, to prevent what he described as an international race. He said colleagues were considering the effects of their work on their lives and, in some cases, were worried enough about instability linked to rapid AI progress to consider buying land elsewhere.
Anthropic said it had been transparent about both the potential benefits and unprecedented risks of AI. A spokesperson said the company continued to develop models with strong safeguards, test its systems aggressively and publish findings intended to support scrutiny and reduce the risk of AI misalignment. The company said it supported a lawful and verifiable industry-wide approach to pacing releases of powerful models.
Industry debate over existential risk
Other figures in the sector have expressed similarly grave concerns. Anthropic scientist Evan Hubinger said he believed AI could kill all humans and put the chance at more than 10% over the next decade. Computer scientist Geoffrey Hinton told the BBC that a 10% chance was not unreasonable, while Marc Warner, chief executive of AI safety firm Faculty, said assigning a probability was extremely difficult but that such warnings were sincerely held.
There has also been pushback. Hugging Face chief executive Clement Delangue suggested Coxon’s views should be kept in perspective, although he later offered to help with potential solutions proposed by Amodei. Nvidia chief executive Jensen Huang has previously called predictions that AI will end humanity “complete nonsense.” Critics have also argued that warnings from Anthropic could be intended to encourage regulation that limits competition. Coxon nevertheless said AI researchers wanted to realise benefits including disease solutions and improvements to people’s lives.
Source: BBC News
