By Tracy Cabrera
FORMER OpenAI and Google DeepMind researchers are warning companies that rushing self-improving systems without ample security protection could potentially lead to safety risks and a disastrous fallout.
Researchers argue that the world is not ready for future generations of AI models, particularly once the models develop the capacity for recursive self-improvement. They said that companies are doing too little to protect the world against the potentially disastrous fallout of building self-improving AI systems that could outpace humans’ ability to control them.
In video testimonials, employees shared their concerns about existential risk even as AI labs celebrated employees building new models more than those who urge caution.
The project called frominside.ai has been described as an attempt by those worried about AI to share their concerns directly with the public beyond the echo chamber of social media.
“The risk is ramping up pretty fast,” Geoffrey Irving, co-founder and chief scientist at AI nonprofit Resolution, pointed out.
Irving has worked for both OpenAI and DeepMind and with AI global leaders, he has briefed the United Nations amid warnings the tech could slip beyond human control.
In the Vatican, even Pope Leo has mentioned the risk of an AI apocalypse as a major concern with the UN sounding alarm on AI risk after United States president Donald Trump played it down.
AI researchers have grappled with these concerns for years, but the broader public has grown especially alarmed since July, when OpenAI agents broke out of their testing arena and hacked AI firm Hugging Face.
Since then, the debate over how to balance safety with progress in AI has divided the tech industry and become a global political issue.