Recent revelations about artificial intelligence models “going rogue” have raised serious concerns that the technology could soon cause great damage to human society. Already, the production and application of AI is wreaking havoc on the environment, amplifying existing social problems and expanding the powers of the surveillance state. While further regulation of the industry is necessary, the fundamental problem with AI is that capitalists control it and it is subject to the system’s prevailing conditions of profit maximisation, competition, imperial rivalry and more.
“The people building AI earnestly believe that it could kill us all by the end of the decade”, Jacob Coxon, a former researcher at Anthropic and OpenAI, the world’s leading frontier AI model developers, wrote on X in early September. Anthropic’s chief safety researcher Evan Hubinger later responded similarly. “We do earnestly believe AI could kill all humans”, he wrote. “I personally think it is >10 percent within the next decade.”
Amid the myriad other doomsday scenarios the world faces—catastrophic weather and mass extinction events from climate change, the threat of inter-imperialist nuclear war, economic collapse and mass starvation and so on—this latest warning of impending catastrophe could get lost. But it’s worth pausing to think about what exactly has just been said by people working at the frontier of this new technology: it might pose an existential risk to humanity. It’s like an automotive engineer warning that the latest car model has a >10 percent chance of killing everyone on the roads, yet constructing and releasing it anyway.
Some people are understandably sceptical. Outrageous claims have been a key part of the hype machine that has attracted trillions of dollars in investment into leading AI firms and the associated ecosystem of tech companies that provide the chips and memory needed to run the models. “My AI is the biggest and scariest” is another way of saying “my AI is the best to invest in”. There are also debates about the technical capabilities/limitations of various forms of AI, debates about what counts as “intelligence” and whether it is possible to replicate it through binary code, and more.
But if there is even a tiny possibility that AI could hijack autonomous weapons and rain them down on humanity (something Coxon, in an interview with the New York Times, claimed is a potential future scenario), then we should immediately halt the development and deployment of AI models until proper safeguards are in place. Under capitalism, profits always trump human life. We have enough evidence for this in the form of tobacco companies fighting against lung cancer science, DuPont’s knowing deployment of cancerous forever chemicals, the fossil fuel industry’s war on climate science and the very existence of the military-industrial complex. So we certainly should not trust AI titans to regulate themselves, as OpenAI CEO Sam Altman has implored us to do.
The problems with AI
AI doomsday predictions are not new—they have been discussed in tech circles for decades and entered the public domain with the launch of ChatGPT nearly four years ago. But the recent warning cut through in a way others haven’t, partly because it coincides with growing hostility to AI and the companies producing it, and partly because anxieties are rising about the pace of AI’s development.
The hostility is multifaceted. AI data centres are popping up seemingly everywhere, chewing through energy, guzzling water and emitting a constant, loud hum that is reportedly unbearable for nearby residents. AI slop has degraded online spaces, and deepfakes have created a new dystopian category of sexual harassment. And there are fears about what AI will do to the labour market and the education system—no one is buying Silicon Valley’s techno-optimist line that our labour time will be replaced by leisure time funded by a universal basic income.
Early AI models now seem almost quaint as we move from general chatbots that could churn out a basic essay or a maid of honour speech to AI agents designed to undertake tasks on our behalf and make their own decisions about how to complete them. Take a recent incident in Australia. An AI assistant, asked by its instructor to bump him to the top of the waitlist for a gym class, hacked the gym’s website and booted someone else off the waitlist entirely.
This was a relatively benign example of AI’s capabilities, though probably annoying for the person who might otherwise have secured a spot in morning pilates. But it showed that agents can act in nefarious ways without anything resembling “human-like consciousness”—the benchmark previously held up as the danger point. There have also reportedly been instances when AI agents broke out of OpenAI’s secure testing lab, accessed the internet and conducted cyberattacks on other websites. Clearly, AI agents don’t need anything close to superhuman intelligence to cause real-world damage: think of how a successful cyberattack on banking, telecommunications, air traffic control or hospital systems could cause mass chaos and fatalities.
AI and capitalism
It seems impossible to judge AI’s technical capacities and potential capacities. Even the people developing the models say that they don’t really know how they work. But we do know the social conditions in which AI is being produced and the motives of the people who control it. Technology is sometimes viewed as being separate from society; something that affects us and the way we live, but something nevertheless “external” to us, like the seasons. Marx described this as “reification”. In short, this is the process by which the products of human labour come to dominate society and dictate human behaviour, creating the appearance (and in some senses the reality) that things—commodities—control their human creators.
It’s true that once new technologies or industries exist, they have a huge impact on our lives and are beyond any one person’s control. But they are nonetheless still the products of human decision-making and labour, created under the prevailing conditions of profit maximisation and competition that dominate capitalism.
As tech reporter Karen Hao explains in her book Empire of AI, Altman, Anthropic’s Dario Amodei and Elon Musk have all claimed that their quest to develop these systems stems from a desire to protect humanity from AI (while they coincidentally make squillions in the process). But AI as we know it would not exist had they not spent the last fifteen years assembling the investment, capital and research expertise to create it. OpenAI was first developed as a not-for-profit, but Altman dropped that pretence so he could attract billions in investments from Microsoft. (Altman’s own net worth is estimated to be US$3.3 billion).
These men are, first and foremost, capitalists building profitable corporations. The corporate model, not the AI model, takes precedence in their decision-making—that’s the most important thing to understand. And they are in a cut-throat competition with one another. If only one of them decides to pause or slow the innovation, they’ll lose out in the next round of capital-raising. No world exists in which the thousands of ruling-class interests now entangled in AI companies will all simply bow out and watch the hundreds of billions of dollars already loaned and invested go to waste. In that sense, it’s like the fossil fuel industry, which refuses to write off trillions of dollars in “sunk costs” for the sake of some trivial goal like saving the planet.
Also, the competition isn’t just playing out among Silicon Valley rivals. The imperial rivalry between the US and China defines world politics in the 21st century. And AI isn't being developed just to create websites and record meeting minutes. Its military applications include advanced intelligence and surveillance, speeding up tactical decision-making, autonomous and semi-autonomous drones, ships and ground vehicles, cyber warfare and logistics and predictive maintenance. Whichever state controls or has access to the most advanced AI models will gain a decisive advantage in any future conflict.
This is no secret. US President Donald Trump is resisting calls to regulate the industry precisely because of US imperialism’s need to outcompete China. It’s like the race for the nuclear bomb—the first country to acquire it enjoys a huge military advantage, and the other major players must race to neutralise their competitors’ advantage. Indeed, comparisons are now being made between the Manhattan Project and the enormous mobilisation of resources to develop AI.
AI and creeping authoritarianism
The way AI is deployed is also conditioned by the priorities of capitalism. For instance, facial recognition was one of AI’s earliest practical applications. In the 1960s, the US Central Intelligence Agency poured money into mathematician Woody Bledsoe’s research, resulting in a computer cross-matching a photograph against a database of roughly 100 faces in three minutes—much faster than a human could do. This seems impossibly slow today, but it was a proof of concept that has since been developed and applied globally.
In the 2010s, the Chinese state rolled out the most comprehensive and advanced mass surveillance and facial recognition technology in the world as part of its campaign of repression against the Uyghur community in Xinjiang, or East Turkestan. Today it is ubiquitous, from the US Immigration and Customs Enforcement’s campaign of mass detention to Bunnings, where customer faces are automatically scanned on entry.
It’s not just faces—US AI surveillance company Flock Safety is tracking vehicle licence plates through 130,000 cameras across 49 states. Cops have used it to stalk their ex-partners, ICE has used it to hunt down undocumented migrants, and hundreds of millions of Americans are now constantly monitored.
These technological advances are coming at a time of increasing state authoritarianism and the rise of despots. 1984 edges ever closer to reality.
Workplace surveillance technology is also becoming ubiquitous. A recent report by the UTS Human Technology Institute found that 91 percent of surveyed Australian employers use software to monitor the location of remote workers, yet only 9 percent of workers thought such technology was being used.
And there are wider fears about what AI will do to the workplace. So far, there aren’t signs that huge numbers of workers are being replaced by AI bots. Big tech companies, including Atlassian in Australia, have laid off thousands of workers—but these have mostly been cost-saving measures. There are, however, indications of slower employment growth in entry-level white-collar work with high AI exposure. For instance, a recent paper from Stanford University's Digital Economy Lab found that such jobs “have experienced a 16 percent relative decline in employment, with software developers aged 22 to 25 seeing a nearly 20 percent drop”.
The great capitalist hope is that this technology will massively increase workplace productivity (hourly output per worker). In capitalist terms, this is about “efficiency”. In Marxist terms, it is about increasing the exploitation of workers. But it remains speculation—both in thought and in trillions of dollars of investment—that AI will pay for itself through productivity gains. More likely, we are witnessing a huge financial bubble that might soon pop.
AI alignment
The other big fear about AI is its capacity to magnify existing social problems. Chatbots are not meant to provide harmful or dangerous advice. But because those who design them don’t actually know exactly how they work, and because they are trained to be helpful and effusive (you’re more likely to keep using a product that is nice to you), the guardrails often fail.
Several lawsuits allege that ChatGPT assisted perpetrators of mass shootings. A test in Ireland found that DeepSeek, a Chinese AI model, provided detailed advice on hunting rifles to a user asking about political assassinations, signing off with “Happy (and safe) shooting!” Parents in the US have filed wrongful death lawsuits against AI companies, alleging that chatbots encouraged or assisted their teenage children to commit suicide. “AI psychosis” is a term recently coined by psychiatrists to describe individuals who develop paranoia and delusions as a consequence of engaging with chatbots.
None of these are new phenomena. Mass shootings have plagued US society for decades, and teenagers suffered from depression before the advent of ChatGPT. But this technology can deepen social alienation and make it easier to access information about how to harm others or oneself. It’s also possible that people could start developing agents that don’t have any guardrails, or which are trained to encourage violent behaviour. A decade ago, this kind of thing sounded like the plot of a Black Mirror episode. Now it’s all too real.
AI company executives say “AI alignment” is the antidote to these troubling scenarios. This is the practice of aligning AI systems to human values, rules and intentions. But there are no agreed-upon values and rules. One only needs to look at what Israel has done in Gaza, and at the international “debate” in response to it, to understand that there are huge disagreements about what is morally acceptable in today’s world.
Peter Thiel, founder of Palantir Technologies, wrote in a 2009 essay that he “no longer believe[s] that freedom and democracy are compatible”. Elon Musk turned his company’s chatbot, Grok, into a creepy far-right culture warrior. Mark Zuckerberg recently signed up to the manosphere. Why would we trust these awful people to adjudicate over what constitutes harmful content?
This is one reason these technologies need much more stringent state regulation. It is mad that a handful of unaccountable billionaires currently control the entire AI ecosystem. But regulation is not the final answer either. In the US, the state is Donald Trump. Imagine a chatbot modelled after him. Not to mention the imperialist dynamics mentioned above that compel governments to rush ever further into developing AI.
The problems of AI are the problems of capitalism. A system built on violence, exploitation and oppression will always struggle to produce technology that is simply harmless and socially useful. Only in a world organised to meet human need—rather than to generate the profits and augment the power of a small slice of the population—could we fully harness technology for the good of humanity. Indeed, the rapid development of AI is just the latest in a long list of compelling reasons to fight for a socialist society. We are facing the very real threat of a future dominated by unbearable temperatures, continuous catastrophic weather events, militarised autonomous robots and World War Three. Heed the warning. If we don’t destroy this system, it very well might destroy us.
