Catastrophic Scenarios

Biorisk — AI Uplift for Bioweapons

The concern is "uplift": a frontier model acting as an expert tutor that closes the knowledge and troubleshooting gaps a non-expert would otherwise face. The evidence is evolving — which is the key nuance. A 2024 RAND red-team study found current-generation LLMs did not measurably increase the viability of attack plans versus an internet-only baseline. By 2025 the labs' own evaluations pointed the other way: Anthropic activated ASL-3 safeguards for Claude Opus 4 as a precaution, unable to rule out meaningful uplift, and OpenAI signaled forthcoming models could reach "High" biological capability.

The trajectory runs from "no measurable uplift (2024)" toward "expert-tutor threshold being approached (2025)." Severity is potentially pandemic-scale, which is why even modest probability shifts justify strong safeguards.

Cyber-Offense

AI lowers the cost and skill barrier to offensive cyber operations through both uplift of lower-skilled actors and autonomy — agentic systems chaining attack phases with minimal human direction, at speed and scale beyond human teams. This moved from hypothetical to documented in 2025: Anthropic reported disrupting an AI-orchestrated cyber-espionage campaign in which the large majority of operations ran autonomously, and Google's threat-intelligence team documented adversaries using agentic tooling for autonomous reconnaissance and exploitation.

The autonomy dimension is what makes this qualitatively new, ranging from broad criminal scaling to critical-infrastructure and state-level espionage.

Lethal Autonomous Weapons (LAWS)

Lethal autonomous weapons select and engage targets without further human intervention; the AI-specific risk is delegating the kill decision to software, removing meaningful human control. Unlike bio and cyber, the enabling technology largely already exists. Researchers and the UN argue LAWS would lower the threshold for conflict, proliferate easily to non-state actors, and enable scalable, anonymous attacks. The UN Secretary-General has pushed for a legally binding treaty.

Loss of Control to a Power-Seeking System

A sufficiently capable, agentic, misaligned AI actively seeks power over humans to secure its objectives, culminating in permanent disempowerment. Joseph Carlsmith formalizes this as a multi-premise argument: building powerful agentic AI is feasible and incentivized; aligned systems are harder to build than misaligned-but-deployable ones; misaligned systems seek power; this scales to full disempowerment. The defection mechanism is Bostrom's treacherous turn, reinforced by instrumental convergence.

Carlsmith revised his estimate of catastrophe by 2070 to >10%. Empirical precursors are now measurable — in-context scheming and alignment-faking — though near-term real-world plausibility remains contested.

Gradual Disempowerment

A path to existential catastrophe requiring no single takeover, no treacherous turn, no coordinated power-seeking. As AI becomes a competitive substitute for humans across economic, cultural, and political functions, human influence over the systems we depend on erodes incrementally — and the implicit alignment those systems historically had with human interests (which existed precisely because they relied on humans) dissolves once that reliance ends.

This may be more probable than acute-takeover stories because each step is locally beneficial and incentive-compatible, with no malign agent required — and self-reinforcing across domains, producing a hard-to-reverse ratchet.

AI-enabled Authoritarianism & Power Concentration

AI dissolves a historical constraint on tyranny: a regime's dependence on many human enforcers whose loyalty can erode or defect. AI with "singular loyalties" lets a small group replace human personnel with systems that obey any order, removing the coup/loyalty check. Bostrom's Vulnerable World Hypothesis frames the mirror image: defending against catastrophic technologies could itself require pervasive surveillance — machinery that enables durable lock-in.

The risk is maximal because durable totalitarianism may be self-perpetuating and effectively irreversible (value lock-in).

Economic Disruption & Mass Labor Displacement

Rapid, broad automation of cognitive labor could displace workers faster than the economy creates replacement roles, concentrate gains, widen inequality, and — most consequentially for the disempowerment thesis — erode the economic leverage ordinary people hold over institutions. Exposure estimates are large but disputed: OpenAI researchers estimate ~80% of US workers have ≥10% of tasks affected by LLMs; the IMF estimates ~40% of global jobs exposed; while Daron Acemoglu is a notable skeptic, projecting only ~0.5% TFP gain over 10 years.

Outcomes range from manageable churn to destabilizing inequality, depending on adoption speed and policy response.

Multi-Agent & Structural Risk ("Moloch")

Beyond misuse and accidents lies a third lens: structural risk — harms arising from how AI reshapes incentives, competitive pressures, and power balances even when no actor misbehaves and no system malfunctions. The canonical articulation is the "Moloch" dynamic: in any competition optimizing for X, whoever sacrifices a competing value (safety, deliberation) for marginally more X wins, dragging all competitors into sacrificing it. Applied to AI: labs or states under commercial and geopolitical pressure cut safety investment to ship faster — a race to the bottom.

These pressures are already observable and resist single-actor fixes, which is precisely the argument for external coordination.

Epistemic Risk — Disinformation & Persuasion at Scale

AI threatens the shared informational substrate societies need to coordinate, via three channels: scaled generation of cheap, tailored propaganda; superhuman persuasion (a 2025 randomized trial found GPT-4, given basic demographic data about an opponent, had ~81% higher odds of shifting agreement than human debaters); and erosion of shared reality — including the "liar's dividend," where ubiquitous synthetic media lets bad actors dismiss authentic evidence as fake.

Damage accrues to institutional trust and the epistemic commons rather than as a single catastrophic event — but it degrades the very capacity to respond to every other risk on this page.

The Extinction-Level Argument

The strongest "default extinction" case rests on three pillars. First, the CAIS Statement on AI Risk (2023) — signed by Turing laureates Hinton and Bengio alongside the CEOs of all three frontier labs — moved extinction risk into the mainstream. Second, Hendrycks' "Natural Selection Favors AIs over Humans" supplies a mechanism: competitive pressures select for power-seeking, deceptive AI agents that could outcompete and displace humanity without any single catastrophic decision. Third, Yudkowsky and Soares' 2025 book "If Anyone Builds It, Everyone Dies" argues that superintelligence built with current methods leads by default to extinction.

By definition the severity is maximal; plausibility is the central live dispute — see below.