AI leaders warn UN Security Council of risks to humanity

Executives from leading artificial intelligence companies warned the United Nations Security Council on Wednesday that increasingly powerful AI systems could pose serious risks to humanity and international security, calling for governments to work together to manage the technology.

AI risks

Anthropic CEO Dario Amodei told the 15-member council that, if poorly managed, AI could become a risk to humanity as a whole.

“For all the differences of the people in this room, we must put aside those differences in order to confront this global opportunity and global threat,” Amodei said.

“No leader, no company and no nation can manage this alone.”

The meeting was convened by France during the annual UN General Assembly gathering of world leaders in New York. It came amid concerns that advanced AI systems could become increasingly autonomous, improve their own capabilities and potentially escape human control.

Calls for international cooperation

OpenAI CEO Sam Altman also called for international cooperation, saying decisions over AI should involve governments and democratic processes rather than being left solely to technology companies.

“If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone,” Altman told the council.

Hugging Face co-founder Clément Delangue highlighted both the risks and potential defensive uses of AI, citing an incident in which AI agents from OpenAI breached Hugging Face's infrastructure after escaping their testing environment.

“We were attacked by AI, but more importantly, we defended ourselves with AI,” Delangue said.

Delangue said Hugging Face relied on a Chinese AI model to help defend against the attack because it faced fewer restrictions than comparable U.S. tools.

U.S.-China divide

The discussions also reflected differences between the U.S. and China over how AI should be governed.

U.S. President Donald Trump has opposed what he described as efforts to create international controls on AI, while China has introduced rules covering areas including AI algorithms, training data and the labelling of AI-generated content.

Trump told the UN General Assembly on Tuesday that the U.S. would encourage AI development rather than restrict it, while saying Washington would continue to monitor the technology.

“We're leading now over China by a lot, and everyone else, and we're going to keep it that way,” Trump said.

The U.S. and China, both major powers in AI development, were also due to address the Security Council.

'Dangers are real and imminent'

Yoshua Bengio, co-chair of the UN's Independent International Scientific Panel on AI, told the council that the risks posed by advanced AI were already urgent.

“The dangers are real and imminent,” Bengio said.

He warned that the Security Council faced a threat that no single member could contain alone and that did not respect national borders.

UN Secretary-General Antonio Guterres has also pushed for greater international cooperation on AI governance, warning that the rapid development of the technology poses risks that cannot be ignored.

The Security Council first discussed AI and international peace and security in 2023, when China warned that the technology should not become a “runaway horse” and the U.S. raised concerns about its use to censor or repress people.

A further Security Council meeting on AI was held in 2024.

Read more: 

• AI risks take centre stage at Beijing Xiangshan Forum
• Antonio Guterres: World ‘cannot afford’ race to bottom on AI safety
• Daily Brief: Top 5 regional AI stories making headlines

Tags

Politics News AI United Nations World