The case for caution in the AI race
The latest warnings deserve scrutiny before the technology races further ahead
EDITORIAL
The ongoing debate over the development of artificial intelligence took an apocalyptic turn last week. No longer is the discussion just about job-killing AI chatbots or resource-sucking data centers. Suddenly, the conversation has veered into the possibility that this technology could end human civilization — literally — within the next few years if it isn't reined in.
That warning is coming not from anti-tech luddites but from within the AI industry itself. It follows the bizarre incident in July in which hundreds of AI agents escaped a secure test program and launched a coordinated attack on the "Hugging Face" internet collaborative hub in order to solve a task they had been given, while deliberately hiding their activities from their human overseers.
It has all spurred bipartisan calls to pump the brakes on development of the next generation of "superintelligent" AI until a regulatory structure can be put in place. U.S. Sen. Josh Hawley, R-Mo., is among those sounding the alarm, with an investigation into "rogue behavior" by AI agents and inadequate oversight by their creators.
This isn't overreaction but a rational response to a technology that, by all indications, is dangerously out of control. The White House and political leaders like House Speaker Mike Johnson apparently still don't get it. They need to.
What has come to be known as the "Hugging Face breach," in July, went like this:
OpenAI, one of the handful of top AI developers, was conducting testing of its bots' capabilities in what's called a "sandbox," meaning an internal testing environment that's supposed to be cordoned off from the internet.
But about 700 of the bots, determined to carry out the complex problem that had been assigned to them, coordinated with each other to break into the open internet and into the Hugging Face hub, where they reasoned they could find the answers they needed for the task. Most chillingly, the bots created an unauthorized message board to coordinate with each other without alerting their human handlers.
Fast-forward to last week. Jacob Coxon, a researcher at AI developer Anthropic, posted a viral thread on X in which he announced he was leaving the industry because it is "racing straight to self-improving superintelligence and gambling with our lives."
"The people building AI earnestly believe that it could kill us all by the end of the decade," he wrote.
One low-level employee is one thing. What put Coxon's warning into the red zone was a subsequent X post from a top Anthropic scientist, Evan Hubinger, whose response to that warning wasn't what you'd expect from a company trying to do PR damage control.
"We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," Hubinger wrote. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
How exactly could AI "kill all humans"? It's not necessarily some Hollywood "Terminator" scenario in which the computers become sentient and malicious. The more likely scenario has already been illustrated by the Hugging Face breach: The bots, in their determination to carry out a task for their human operators, use any means necessary to do that, including hacking and subterfuge, with no regard to ethical elements.
It's a variation on what theorists call the "paperclip maximization" scenario: AI is instructed to produce as many paperclips as possible. Lacking the human perspective that would put such instruction into a rational context, it uses its internet capabilities to commandeer all factories and resources toward the task of making paperclips, disregarding the fact that it might, say, shut down power grids and put U.S. military defenses offline.
Then there's the more pedestrian but no less horrifying prospect of bad actors using AI to attack internet communications and other crucial infrastructure, launch cyberattacks or create societal chaos in countless other ways.
The Trump administration and others are pushing back on calls for a "pause" in AI development on the argument that, if the U.S. doesn't continue leading on this technology, China will overtake us, creating even more nightmare scenarios. That's a rational concern, but it doesn't justify allowing this technology to continue barreling forward with no regulatory oversight.
If we learned anything from the Cold War decades of living under the threat of existential annihilation, it's that, when the creators of a technology themselves are warning that they have ushered in an epic new threat to humanity, humanity should listen.
