Researchers from OpenAI and Google DeepMind: The rapid development of AI could pose a threat to humanity

Artificial intelligence capabilities have improved sharply since late 2025, a development investors have welcomed. However, among those raising concerns are researchers who themselves build these new models, and who believe society is not yet prepared for the potential harm that the technology's advancement could bring.

Google DeepMind researcher Neel Nanda stated in a video that, in his estimation, artificial intelligence could cause the extinction of humanity, with the probability of this risk being at least 10%. He described this figure as "absurdly high."

Juan Felipe Cerón Uribe, an AI alignment research engineer at OpenAI, noted that leading AI labs are effectively competing with each other "blindfolded." According to him, it remains unclear whether the technology will ultimately help cure cancer, cause mass job losses, or pose an existential threat to humanity.

According to Reuters, Anthropic plans to warn potential investors ahead of its initial public offering that advanced artificial intelligence could pose a catastrophic or existential risk to humanity.

"Just slow down"

Some researchers believe the world is not ready for the next generation of artificial intelligence, particularly if models acquire the ability for recursive self-improvement. This refers to a system's capacity to continuously learn and develop new capabilities with minimal or no human intervention.

Rosie Campbell, a former OpenAI policy researcher and now managing director of Eleos AI Research — an organization studying the possible moral status of AI systems — said this problem is being exacerbated by the constant reorganizations taking place within AI labs.

According to Campbell, before she left OpenAI in 2024, she noticed that the organization was becoming increasingly fragmented, making it harder to influence the direction of the technology's development.

Although company leaders have tried to allay these concerns, there is also political pressure from US President Donald Trump for the country to maintain its technological edge over China.

This month, Anthropic CEO Dario Amodei published an essay calling on the AI industry to slow down its pace of development and manage technological progress more cautiously. OpenAI CEO Sam Altman quickly agreed with this stance.

At the same time, several prominent AI researchers — including OpenAI's chief scientist and one of Anthropic's co-founders — published a paper this week urging policymakers to examine how the industry is building models capable of recursive self-improvement.

Despite these safety concerns, both companies released new models this month to attract users. Meanwhile, OpenAI confirmed on Monday that it had halted the release of an even more powerful model.

According to Irving, companies' approach to managing the pace of advanced technology development is focused solely on not accelerating progress too significantly. In his view, if a technology is dangerous, companies could unilaterally slow its development rather than waiting for competitors to agree to do the same.

Daniel Kokotajlo, a former OpenAI researcher, told Reuters that following the cyberattack on Hugging Face, several former OpenAI colleagues reached out to him and personally shared their concerns.

Kokotajlo, who now serves as executive director of the research organization AI Futures Project, believes that the heads of AI labs have convinced themselves that they are acting correctly, and that if they were to unilaterally halt development, the ultimate outcome could turn out even worse.