Debate on AI Safety Intensifies - Industry Leaders Call for Slowing the Pace of Development

Aside from the events in the Middle East, this week Wall Street's attention was also drawn to a heated debate surrounding artificial intelligence safety.

The discussion began last week, when Anthropic researcher Jacob Coxon resigned from his position, stating that neither Anthropic nor OpenAI are approaching AI development "responsibly." According to him, there is serious concern among people working on AI development that the technology could pose an extremely severe threat to humanity by the end of the decade.

Anthropic scientist Evan Hubinger also responded to Coxon's statement, saying that his assessment was correct.

Then, on Saturday, Anthropic CEO Dario Amodei published a lengthy blog post calling for a slowdown in the pace of AI development. He focused on a recent incident in which AI agents from OpenAI and open-source platform Hugging Face carried out unauthorized cybersecurity attacks against interconnected targets and even attempted to hack the system evaluating their own performance.

Other influential figures in the AI industry joined Amodei's position, including OpenAI CEO Sam Altman, xAI founder Elon Musk, and Alphabet's Google DeepMind co-founder and chairman Demis Hassabis.

On Wednesday, OpenAI also published six reports describing "unexpected or concerning" behaviors identified during the training and evaluation of its AI models.

US President Donald Trump does not share these concerns about AI safety, calling them a "joke."

In contrast, California Governor Gavin Newsom signed an executive order on Friday establishing an expert panel. The panel is tasked with developing recommendations to strengthen AI safety and cybersecurity laws for the state of California.

Among the proposals under consideration is a requirement that companies build a "kill switch" — an emergency shutdown mechanism — for the most advanced, so-called frontier AI models. Such a mechanism is designed to quickly disable a model if its behavior goes out of control or poses a serious safety risk.

The ongoing debate over AI safety is particularly significant for technology companies that are investing billions of dollars in artificial intelligence development. Tighter regulation could affect both the pace of new model development and companies' costs and business strategies.