'AI race could put humanity at risk'

A former researcher at AI company Anthropic has warned that the rapid pace of artificial intelligence development could pose a serious threat to humanity if it is not slowed.

Jacob Coxon, 27, who recently resigned from Anthropic, told BBC interviewer Laura Kuenssberg that many people working in the technology sector are “genuinely scared” by the speed of AI advances. He said there is a significant risk that humanity could face an existential threat in the near future if the development race continues unchecked.

Coxon, who previously worked at OpenAI, said efforts to slow AI development should involve international coordination, including with China, to prevent an intensifying global race over increasingly powerful systems.

He pointed to a potential scenario in which vast numbers of AI bots could operate together with supercomputer-level capabilities and potentially take control of parts of the internet. He said such a situation could emerge within six months to a year, citing risks previously raised by Anthropic CEO Dario Amodei.

Amodei has also argued that AI development should not be halted but that companies and governments need time to address its serious risks. OpenAI CEO Sam Altman and xAI chief Elon Musk have similarly discussed the need for greater oversight and controls.

Anthropic said it has always been transparent about both AI’s potential and its risks and is working to develop strong safeguards. Critics, however, have questioned whether dire warnings could also serve companies’ commercial interests by encouraging regulation that limits competition.

Coxon’s views have drawn criticism from AI figures including Hugging Face CEO Clément Delangue and Nvidia CEO Jensen Huang. Huang has previously dismissed the idea that AI could cause humanity’s extinction as “absurd.”

Coxon said some of his former colleagues believe major dangers could emerge within two years, while acknowledging AI’s potential to transform healthcare and improve lives.

Anthropic scientist Evan Hubinger has also expressed concern, estimating that AI could cause human extinction within the next decade with a probability of more than 10 percent.

Concerns over AI safety have been raised by leading technology companies since at least 2023, but the debate has intensified as newer AI models become increasingly capable.