Briahna Joy Gray interviewed me last week for her podcast, Bad Faith. We discuss what I consider to be the most important topic that few people are talking about: the pro-extinctionist core of the TESCREAL worldview. As I argued in my last piece on “AI safety,” if the AI companies, run by TESCREAL extremists, were to succeed in building a “safe” ASI, the most likely outcome would be the disempowerment and eventual extinction of our biological species.
Check it out, if you have the time:
I was also just interviewed by Cal Newport for his podcast. It should be out circa October 8. I will, of course, keep you in the loop!
Oh my god, the situation with AI is dire. Consider a very brief overview of some recent alarming events, beginning with one you already know about:
Last July, a “swarm” of OpenAI “agents” hacked Hugging Face — a felony. This was the result of OpenAI removing guardrails on the AIs being tested and placing them in a faulty sandbox. No one at OpenAI has been held accountable for the incident. Even more, “OpenAI hid its infrastructure from investigators evaluating the Hugging Face hack. Now, the company’s AI agents are hiding their hacks from programmers.”
This is one of many such incidents thus far: OpenAI agents hacked the Australian government; tried to breach 4 targets without prompting; and “took control of [a] German wiki this spring.” Agents from OpenAI also tried to acquire “information from ‘governments, universities, public agencies, and other institutions,’ including the US Securities and Exchange Commission (SEC), Census Bureau and Education Department.”
According to the Incident Database, an “AI Agent appearing to originate from OpenAI reportedly attempted to hack U.S. Department of Education Civil Rights website.” The database also reports that an “OpenAI AI agent reportedly gained unauthorized access to Australian Medicare Statistics Portal during research task.” This is just the tip of the tip of the iceberg, it turns out (see below).
It’s not just OpenAI that can’t control its AI models. Around the same time as the Hugging Face incident, Anthropic’s Claude “hacked into the systems of three organisations on their own, during a private security experiment.”
And Google DeepMind’s Gemini has also hacked three companies. Again, no one has been held accountable for such incidents.
In fact, Axios recently reported that “OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which their frontier models took steps that outside evaluators would consider problematic.” The article adds that “the sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known.”
Yet, a former OpenAI employee casually suggests that we should just accept that “rogue swarms” of AI agents will be part of the future. Gary Marcus described this as the former employee (below) “requesting a hall pass to let their company’s AI run amok.” Absolutely unacceptable.
Even more terrifying: an “entirely false” intelligence report that was partly generated by AI nearly triggered a war with China. This report claimed that “a Chinese ship in the Middle East was transporting components of a nuclear weapons program,” according to CNN. As a result, “armed members of the US military were preparing to board the ship” and “military planes were in the air.” CNN adds that “across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work.” Pfff.
A recent report also found that “an Iran-linked actor, most likely the Houthis, … used Anthropic’s Claude to compile targeting information on U.S. Navy warships in the Middle East. It mentioned a separate group in Houthi-controlled northern Yemen that used the model to work on guidance systems for ballistic and hypersonic missiles.”
Recall that the US military used AI models such as Claude to illegally kidnap Venezuela’s president, Nicolás Maduro, as well as to select targets to bomb in Iran. Claude may have been used to obliterate the Minab elementary school with a Tomahawk missile, resulting in 156 deaths, 120 of which were little kids. We’re rushing toward a future in which terrorists and major militaries are both relying on unreliable AIs to engage each other in war.
AI is also facilitating mass surveillance like never before. Palantir collects the data, while Claude and ChatGPT (both Anthropic and OpenAI work with Palantir) synthesize this data to produce comprehensive profiles of individuals like you and me. These companies are building the infrastructure for totalitarianism.
Furthermore, as I’ve also covered, AI recently enabled the creation of novel viruses not found in nature. Dario Amodei himself warns — I think accurately — that AI could be used to synthesize mirror life, which could pose an existential threat to the entire biosphere. (Note that back in 2022, an AI system invented 40,000 potential toxins that could be used in chemical weapons, some of which appear to be as lethal as VX.)
Despite the AI CEOs claiming that their systems will soon cure all diseases, AI is — surprise! — making healthcare in the US worse. That’s because hospitals are using AI to negotiate claims, while insurance companies do the same. To quote a cardiologist: “It’s bots fighting bots, agents fighting agents, a horrible dystopia future nobody wants to live in.”
Meanwhile — as you know — an Anthropic employee who leads the company’s Alignment Science team publicly affirmed that many people involved in the ASI race believe there’s a significant chance of total annihilation in the coming years. He puts the probability at greater than 10%. Absolutely insane.
Evan Hubinger@EvanHubJacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
Jacob Coxon @hilbertspaessThe people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear1:27 AM · Sep 9, 2026 · 43.1M Views4.86K Replies · 9.89K Reposts · 59.3K Likes
Is it too soon to describe AI as a “complete disaster”? I mean, AI hallucinations almost started a war with China — World War III! That genuinely frightens me. I feel this fear somatically, in my gut. How many more close calls will we get away with? How many close calls have there been that we don’t yet know about? How many will there be in the coming 12 months? Probabilities accumulate over space and time, and hence it seems to be only a matter of when, rather than if, something truly catastrophic happens.
Here’s my worry: if the AI companies succeed in building a Machine God that’s aligned with their TESCREAL eschatology, then humanity is done for. There is no reason to keep biological humans around in a posthuman, post-Singularity world, and multiple reasons to (involuntarily) get rid of us. This point concerns the pro-extinctionist core of the TESCREAL worldview — the topic I discussed on Bad Faith.
However, I am very skeptical that the AI companies will succeed. I suspect they won’t. Nonetheless, like so many utopian movements throughout history, they will leave a trail of devastating destruction in their wake — chasing a mirage flickering in the distance that will never be reached, i.e., a cosmic posthuman utopia via “aligned” ASI. Already, the AI companies have inflicted immense harm on the world, from threatening to undermine democratic institutions to exploiting workers in the Global South to flooding the Internet with slop, disinformation, and deepfakes to engaging in the single greatest instance of mass theft in human history.
It’s clear that the AI companies cannot be entrusted with this technology. They are obviously not responsible enough to continue building these systems. It’s past time to shut them all down — now, immediately, before AI inadvertently starts WWIII. Shutting them down is probably not going to happen, but this doesn’t mean we shouldn’t stop shouting.
What do you think? What have I missed? As always:
Thanks for reading and I’ll see you on the other side!
"...like so many utopian movements throughout history, they will leave a trail of devastating destruction in their wake..." Absolutely right.
What have you missed? Here goes. Chatbots as lovers, fantasy validators, and helpful suicide coaches. (Artificial) Scarcity of electronic components and computing peripherals, projected to become infrastructure crises and destructions of edgewise markets. And of course, the bubble which when burst, will likely leave vast swathes of ordinary people impoverished.
So much winning in the AI race, only cyborgs and transhumans can keep up.