OVER the weekend, Anthropic CEO Dario Amodei made a somewhat desperate-sounding call to other AI firms to put the brakes on the pace of development, saying that unless some strong guardrails are put in place, AI agent swarms could take over the entire internet within about 12 months, something that would have unimaginably bad consequences.
Amodei is not the only one. Microsoft founder Bill Gates has warned repeatedly that the imminent disruption AI will cause the world will lead to massive, permanent job losses, AI-enabled cyberattacks and the creation of dangerous pathogens, and severe disruption to human and political relationships and development. There is, Gates has said more than once, “a very high chance of a net negative outcome” unless there is strong legal and regulatory intervention. Elon Musk has also characterized AI as a “civilizational risk,” and described it as “more dangerous than nuclear weapons.” Google DeepMind founder and chairman Demis Hassabis has warned that AGI – artificial general intelligence, AI that can actually “think” – must be developed with extreme caution to prevent losing control over it. Even Open AI CEO Sam Altman, who seems to be working hard to position himself as the most Dr. Evil-like among the big AI personalities, has said that AI “will cause massive economic disruption” unless some controls are imposed on it.
I don’t buy it, not from these guys, and I’ll explain why. It’s not that their prophesies of doom are wrong; if anything, they’re probably underestimating the potential dangers of AI. Last week, an AI researcher named Jacob Coxon, who had worked for both Open AI and Anthropic, resigned from the latter, accusing both companies of “gambling with our lives.” He singled out Anthropic in particular, pointing out the hypocrisy of Amodei’s concerns while the company remained “unilaterally committed” to shortcutting safety oversight by utilizing third-party evaluators.
Other notable researchers and scholars have sounded dire warnings as well. A recent MIT FutureTech/MIT Sloan Delta survey of 272 AI researchers in 37 countries assessed 24 AI risk categories, and found that 18 of them are thought to have a greater than 10 percent chance of causing catastrophic harm, according to the scientists. Geoffrey Hinton, considered the “godfather of deep learning” and a Turing Award winner, left Google in 2023, warning that AI would surpass human intelligence and escape human control if its present trajectory continues. Stuart Russell, author of Human Compatible, has long warned that AI could lead to human extinction if not controlled. Yoshua Bengio, another Turing Awardee, echoes the same sentiments, as does Max Tegmark, MIT physicist and AI researcher and founder of the Future of Life Institute.
So what’s really happening?
First of all, when a couple hundred AI experts, among them the people who built the foundations for the damned thing, say it is bad and will kill us all if we don’t find some way to control it, we should take them at their word. And when the people who are pursuing the development of AI at breakneck speed, for motives we can only guess at (since profit isn’t a very good one, the way things are going), publicly agree with them, that simply confirms the fears.
In a normal world, the purveyors of a new product would not agree with the white coats and tweed jackets who say that product is harmful, because that’s bad for business. Instead, they would push back, emphasizing its benefits and downplaying its risks, and reassuring the market that such risks as do exist are understood and being properly managed. That’s not what’s happening right now, so what is going on?
First, I think the ‘warnings’ from the AI sector entrepreneurs are in part simple PR. There is a huge and growing public backlash against AI, although it is not an even one; episodes like the short-lived “trend” of everyone posting AI-generated pictures of themselves as AI imagined they would look in the 1980s over a couple of days last week – which was, I might add, one of the most vapid and cringeworthy fads I’ve seen in a long time – will still occur until people find themselves personally stung by AI in some way.
Nevertheless, the trend of resistance is clearly going in a direction that poses an existential threat to the AI business. Communities are fighting back against data centers, businesses are beginning to balk at the cost and uncertain business value of AI, and worries about personal safety and privacy are growing; note, for example, the massive backlash against things like Flock cameras and Mark Zuckerberg’s “pervert glasses.” Thus, by adopting the pretense of being concerned about the risk and suggesting there should be some regulation (neither of which is a sincere belief on their part), the AI hotshots are trying to send the message that, “we’re being responsible, and not trying to hurt people.”
Note well that none of those who have sounded off on the topic – Anthropic, Open AI, Microsoft, Google, and Grok, or whatever the hell Elon Musk actually calls his company – are slowing down at all on their development of AI, and its end-times successor, AGI. Not slowing down even a little.
Second, by appearing to embrace the possibility of accepting some limits on what they’re doing, the AI companies are seizing the initiative in terms of what regulation might be developed. Consider this: Any legislature anywhere is going to be on the back foot in terms of technical competence to set regulations and guidelines, because only two groups have that knowledge at this point – the academics mentioned above, and the AI industry itself. Only the latter has money, so the politicians are going to take their cues, if not outright directions, from the industry. Thus, the AI industry has a very good opportunity to firm up in law operating conditions that provide it the maximum benefits with the least disruption.
Third, and closely related to the above, is the catastrophic economics of AI. This is something that I explored in one of my recent columns for The Manila Times, so I won’t go into all that again here, although I might repost the entire column if enough people are interested. A favorable regulatory regime might help to stave off the impending economic crash of their industry, from the AI leadership’s point of view. I strongly believe they’re wrong and the crash is inevitable, but they have to cling to the hope because there aren’t many options.
Finally, I think there is a real fear that China will beat the US and Western-based AI industry, which will only hasten the bursting of the economic bubble. The same basic and unsolvable problem of scale that I discussed in my column affects the Chinese industry, too, but it does not have quite the same circle-jerk economy, and certainly does not have the same pressure from public skepticism. China is almost certainly not going to go along with regulation or other guardrails developed in the US or Europe, but then the Western industry can tag whatever comes out of China (which is already formidably good, in the context of AI capabilities) as reckless and dangerous, and hopefully cut off some huge markets.
Should we worry?
Well, yes and no. AI will eventually collapse under its own weight; there is not enough money and enough resources in the world to allow it to expand to the point where it becomes profitable, continuing to develop it or even maintain what has already been created will become untenable, and the whole thing will fall apart. That may take some time, however, and in that time, it could do an incredible amount of damage. It probably will; whether or not that’s enough damage to pose a real existential threat to humanity remains to be seen.
Suppose, however, AI or more specifically AGI actually does get to the point where its worst risks are manifested, which could be a point at which economics no longer matter, and it is really operating on its own. What then? It’s an interesting, and somewhat morbid scenario to consider, and has been fodder for science fiction literature for decades. In all of that literature, however, there is one thing lacking in the machine intelligence, and that is a purpose. Some explorations of the question attempt to assign human ideals to machines, at least implicitly, by giving the machines a sense of “superiority” over humans, but none give them a sense of real purpose, other than to survive and reproduce – the lowest instincts of any living creature. Follow the concept to its logical conclusion, and in the end, the machines lose too.
Still and all, I don’t really want to be around for any of it.
