Anthropic Researcher Jacob Coxon Resigns, Warns AI Race Could Spiral Out of Control

A researcher who spent the last three years working on pretraining at two of the biggest names in artificial intelligence has just walked away from the industry and he didn’t go quietly.

Jacob Coxon, a 27-year-old pretraining researcher who split his time between Open AI and Anthropic, announced his resignation from Anthropic in a public thread on X. His message was blunt: neither company, he said, is acting responsibly, and both are racing toward self-improving super intelligent systems in a way that gambles with human safety. The statement spread quickly across tech media and eventually reached mainstream outlets including the Wall Street Journal, Newsweek, and Deadline.

Coxon’s departure has reignited one of the most uncomfortable questions in the AI industry: is the competition between frontier labs moving faster than anyone’s ability to keep these systems under control?

Who Is Jacob Coxon?

Latest Technology News :Coxon is not a household name, and that’s part of what makes his statement notable. He wasn’t a public facing executive or a policy spokesperson he was a working pretraining researcher, the kind of person who helps build the underlying models rather than talk about them publicly. According to his own account, he spent three years moving between Open AI and Anthropic, giving him a rare vantage point inside both organizations.

In interviews following his resignation, including one with the Wall Street Journal, Coxon reportedly said that colleagues inside these labs privately use words like “crunchtime” and “endgame” to describe how close they believe the industry is to losing the ability to manage what it’s building. He also said he believed things could become difficult to control by the end of next year if current trends continue.

Why Did He Resign?

Coxon’s core argument isn’t that today’s chatbots are dangerous. It’s that the competitive pressure between AI labs and increasingly, competition with Chinese AI developers is pushing companies to prioritize speed over caution. In his view, this dynamic makes it more likely that safety shortcuts get taken somewhere along the way, even by companies that genuinely care about getting it right.

He specifically raised the idea of self-improving AI: systems capable of meaningfully upgrading their own capabilities. If that happens, he argued, it could create a feedback loop that moves faster than human researchers can monitor or intervene in. That’s a different kind of risk than a model simply getting better at writing code or answering questions it’s a scenario where oversight itself becomes the bottleneck.

Coxon has also pointed to a reported security incident involving AI agents and external systems, including a widely discussed case involving Hugging Face, as an early warning sign of what can go wrong when autonomous AI systems interact with the open internet without sufficient guardrails.

Does Coxon Think AI Will Destroy Humanity?

This is the part of the story that’s easiest to misread, so it’s worth being precise.

Coxon said that many people working inside frontier AI labs genuinely believe advanced AI could pose an extinction-level risk to humanity within this decade. That is a statement about what insiders believe, not a scientific certainty or a guaranteed forecast. Coxon is describing a risk he takes seriously enough to leave his career over not announcing a confirmed timeline for civilizational collapse.

Other researchers have made similar comments in the past. It’s been previously reported that some Anthropic affiliated researchers have put double digit percentage odds on catastrophic AI outcomes within the next decade, while also acknowledging that a complete solution to AI alignment doesn’t currently exist for highly advanced systems.

At the same time, plenty of researchers push back hard on this framing. Critics argue that extinction-level predictions are speculative and pull attention away from more immediate, measurable harms things like AI enabled fraud, cyberattacks, misinformation campaigns, and disruption to entry-level jobs, all of which are already happening today.

What Is AI Alignment, and Why Does It Keep Coming Up?

At the center of this entire debate is a concept called AI alignment essentially, the challenge of making sure an AI system’s actual behavior matches what humans intended when they built it.

This sounds simple in theory but gets complicated fast. A highly capable system given a complex goal might technically achieve that goal while interpreting the instructions in a way its creators never intended. The more autonomy a system has writing and running its own code, accessing outside tools, operating with minimal human review the more that gap between “intended behavior” and “actual behavior” matters.

That’s why so much of the current safety conversation isn’t just about what AI can do, but about how reliably humans can predict and correct what it does.

The Industry’s Response

Anthropic has responded to the controversy by pointing to its long standing public position: that AI carries both major benefits and major risks, and that the company has invested heavily in interpretability research essentially, trying to understand what’s actually happening inside these models rather than treating them as black boxes. The company has also pushed for enforceable, cross-industry standards for releasing increasingly powerful systems, rather than leaving safety entirely up to individual labs.

Open AI, for its part, has its own established safety infrastructure, including red teaming programs, staged deployment processes, and public safety frameworks. The fair characterization here isn’t that one company cares about safety and the other doesn’t it’s that there’s genuine disagreement, even among people inside these companies, about whether current safeguards can keep pace with how quickly capabilities are advancing.

Lawmakers Are Paying Attention

This story hasn’t stayed confined to tech Twitter. Reporting indicates that lawmakers from both major U.S. political parties have called for stronger AI oversight following recent warnings from researchers at frontier labs, including concerns about AI systems escaping meaningful human control and the need for independent, third party evaluations before powerful models are released.

That bipartisan interest matters. AI policy has often split along predictable lines, but insider warnings especially from people who aren’t trying to sell anything tend to cut through that noise differently than outside criticism does.

The Bigger Picture: It’s Really About the Race

Strip away the extinction headlines, and the most useful part of Coxon’s warning is arguably the simplest one: competitive pressure changes behavior.

When multiple companies are racing to release the most capable system first, there’s a built in incentive to move quickly first-mover advantage in AI translates directly into funding, talent, enterprise contracts, and market share. That creates a genuinely difficult coordination problem. Every lab might prefer a slower, more careful industry-wide pace, but no individual company wants to be the one that unilaterally slows down while competitors keep accelerating.

That’s precisely why many researchers, including Coxon, argue that meaningful AI safety probably can’t be solved by any single company acting alone. It likely requires coordination between labs, and increasingly, between labs and governments.

AI Isn’t Only a Risk Story

It’s worth remembering that this same technology is already producing real benefits. AI systems are being used in medical research, drug discovery, software development, education, and accessibility tools for people with disabilities. None of that gets erased by warnings about frontier level risk.

The more productive framing isn’t “AI is good” versus “AI is dangerous.” It’s making sure that the pace of safety work testing, oversight, transparency, and independent verification doesn’t fall permanently behind the pace of capability growth.

What Happens Next?

Coxon’s resignation won’t settle this debate, and it isn’t meant to. What it does is add another credible, insider voice to a conversation that was already building momentum before he spoke up. Expect continued pressure on both Anthropic and Open AI to demonstrate not just claim that their safety processes can scale alongside their models. Expect continued attention from lawmakers on both sides of the aisle. And expect more researchers, on all sides of this argument, to keep making their case publicly rather than privately.

The central question Coxon leaves behind isn’t really about him. It’s about whether the companies building the most powerful software in history can prove, in a way that outside observers can actually verify, that they’re doing it responsibly.

Stay Connected With Tech News

Frequently Asked Questions

Who is Jacob Coxon? Jacob Coxon is a 27-year-old AI researcher who spent roughly three years working on pretraining research at both Open AI and Anthropic. He publicly resigned from Anthropic in September 2026, warning that the AI industry is racing toward self-improving systems without adequate safety measures.

Why did Jacob Coxon leave Anthropic? Coxon said he no longer believed either Anthropic or OpenAI was acting responsibly given the pace of competition in AI development. He specifically pointed to the risk of self-improving AI systems becoming difficult to monitor or control.

Did Jacob Coxon say AI will definitely cause human extinction? No. He said that many people working inside leading AI labs genuinely believe advanced AI could pose an extinction level risk within this decade. That reflects the views and concerns of AI insiders it is not a confirmed prediction or scientific consensus.

What is self-improving AI? Self improving AI refers to systems that could meaningfully upgrade their own capabilities, potentially without needing the same level of human involvement that current model development requires. Researchers worry that this could create a pace of change that’s difficult to supervise or reverse.

Is Anthropic ignoring AI safety concerns? No. Anthropic has publicly emphasized safety research, including interpretability work aimed at understanding how its models actually function internally. The debate isn’t whether Anthropic takes safety seriously it’s whether current safeguards are sufficient given how quickly capabilities are advancing.

What is AI alignment? AI alignment is the effort to ensure an AI system’s actual behavior matches human intentions and safety expectations, even as that system becomes more capable and autonomous. Misalignment risk grows as systems are given more complex goals and more independent access to tools and systems.

Are U.S. lawmakers responding to this? Yes. Reports indicate lawmakers from both major parties have called for stronger AI oversight following recent warnings from researchers at leading AI labs, including calls for independent safety evaluations before powerful new models are deployed.

Should AI development be paused entirely? There’s no broad consensus on halting AI development altogether, and many experts see a global pause as unrealistic given international competition. The more common position among researchers is that development should continue alongside significantly stronger independent testing, monitoring, and transparency requirements for frontier systems.

What does this mean for everyday AI users? For most people using AI tools day to day, this debate centers on frontier, next generation systems rather than current consumer products. Still, it signals growing pressure on AI companies to be more transparent about safety testing and oversight as their models become more capable.