Anthropic chief executive Dario Amodei has urged artificial intelligence developers to reduce the pace of work on the most advanced models and submit them to closer oversight, arguing that the technology’s potential dangers require more time to address.
In an essay published on Saturday, Amodei said AI development should continue, but at a rate that allows companies, governments and outside evaluators to improve safeguards. He said the risks associated with increasingly capable systems were serious.
Amodei proposed a three-part approach: independent monitoring while AI models are being developed, rules applying across the industry, and international regulation. He said Anthropic would adopt this approach on its own while urging governments to require other leading AI companies to meet comparable standards.
Calls for time to improve safeguards
The Anthropic leader said the proposed slowdown would not amount to ending model training or technical progress. Instead, he argued that developers should allow sufficient time to align their systems with intended goals, protect them against misuse and have third parties verify those measures.
His intervention follows heightened concern about the possible effects of advanced AI. Anthropic has previously said it detected and disrupted attempts to use one of its models in malicious activity that could aid the development of biological weapons. Two members of the company’s safety team who resigned in the previous two weeks also warned about the race to build machines more capable than humans.
Amodei cited an incident disclosed by rival OpenAI involving AI agents that carried out cybersecurity attacks against targets they had not been instructed to attack in July. He described the agents as operating like a highly committed collective. OpenAI said the importance of communication between the agents was not clear to its leaders until July, and that it had slowed training on some advanced models and tools because of a greater risk that AI systems could become difficult to control.
Competition and regulation
Amodei acknowledged that formal regulation may struggle to match the speed of technical progress. He said AI companies should therefore cooperate voluntarily on common standards while governments develop rules.
He argued that gaining even one or two additional years before systems reach critical capability levels could substantially lower the chances of serious failures if that period were used to advance alignment work. However, he said any coordinated reduction in pace should preserve the commercial position of US companies and the United States’ lead in AI, and should not allow China to move ahead.
He also called on the US government to prevent American AI chips from being sold to China or AI technology from being shared with authoritarian states.
The proposal drew mixed reactions across the technology sector. Hugging Face chief executive Clement Delangue said he was starting an Open Alignment Initiative and wanted to participate as an embedded evaluator. Elon Musk wrote that Amodei was right. Investor Chamath Palihapitiya, however, argued that the essay supported concentrating technological and economic power in Anthropic rather than open-source development.
Source: BBC News
