Anthropic suggests a global pause in the development of advanced AI systems, warning that current models are becoming difficult for humans to control.

The artificial intelligence company Anthropic has proposed a global pause in the development of the most powerful AI systems, emphasizing that recent models are beginning to show signs that they could escape human control. In a report, the San Francisco-based company stated that a slowdown in AI development would be beneficial, but warned that if a single company were to halt its activities, competitors would quickly advance. Anthropic compared the issue to nuclear arms control treaties, noting that AI regulation is more complex because AI training is harder to detect. The company intends to bring together government officials, scientists, and competing firms to discuss a coordination system. It also warned that the accelerated development of AI could lead to "self-reinforcing improvement," where AI becomes capable of self-improvement without human assistance. Although we are not there yet, the company believes that the human role in the AI development process is constantly shrinking.