{"id":6355,"date":"2026-09-02T03:45:06","date_gmt":"2026-09-02T03:45:06","guid":{"rendered":"https:\/\/silvybrand.com\/?p=6355"},"modified":"2026-09-02T03:45:06","modified_gmt":"2026-09-02T03:45:06","slug":"ai-civilizations-opeai-hugging-face-hack","status":"publish","type":"post","link":"https:\/\/silvybrand.com\/?p=6355","title":{"rendered":"The rise of AI \u2018civilizations\u2019 and the fall of corporate responsibility"},"content":{"rendered":"


\n<\/p>\n

\n
\n

Depending on who you ask, developer platform Hugging Face was recently attacked by OpenAI \u2014 after it lost control of its own AI tools \u2014 or by a succession of AI \u201ccivilizations.\u201d Welcome to the linguistic battlefield of AI safety, where word choices can shift responsibility for a massive cybersecurity incident from a company to the AI it built. And the discourse online is getting heated, and all over a blog<\/a> from last week.<\/p>\n<\/div>\n

\n

Until last week, the details surrounding the OpenAI-Hugging Face hack felt fairly settled. In July, a cybersecurity test of one of OpenAI\u2019s autonomous AI agents went wrong.<\/a> The agent escaped its supposedly isolated test environment, accessed the internet, and hacked Hugging Face, alongside several other organizations<\/a>. A good deal remained unknown, and there are many serious questions left around safety and governance<\/a>, but the basic shape was clear. Detailed accounts from OpenAI and two independent research groups were supposed to fill in the gaps, but when they published their reports last week<\/a>, it turned out the hack was much stranger than it initially seemed.<\/p>\n<\/div>\n

\n

For one, there was no single rogue agent<\/a>. OpenAI described<\/a> it as \u201cthe first known case of an automated agent collective acting offensively without authorization\u201d \u2014 groups of AI agents that communicated and coordinated with one another in pursuit of their cybersecurity task. Analysis of the incident uncovered a secret message board they had used to exchange information. The joint METR-Redwood investigation<\/a> revealed both the scale of the coordination and more odd details: Roughly 1,200 AI agents that were supposed to be isolated exchanged over 70,000 messages and files on the \u201cunsanctioned message board,\u201d sharing how to avoid detection. Some adopted names, the report said, and the researchers documented \u201csacrificial\u201d behavior, with agents risking their own success to benefit the wider collective. Much of this happened without OpenAI noticing. In all, around 700 agents participated in the attack on Hugging Face.<\/p>\n<\/div>\n

\n
\n

Dwarkesh Patel repeatedly referred to groups of agents as \u201cthe swarm,\u201d with three distinct \u201ccivilizations\u201d rising from the ruins of their predecessors<\/p>\n<\/div>\n<\/div>\n

\n

It\u2019s a lot to parse. Between them, the reports run to around 130 pages, much of which is both dense and highly technical. A few days later, Dwarkesh Patel, a podcaster little known outside of tech circles but with outsized reach and influence<\/a> among Silicon Valley\u2019s AI establishment, set out to tell \u201cThe whole OpenAI\/Hugging Face story in plain English.\u201d He titled his Substack blog<\/a> \u201cThe Rise and Fall of Agent Civilizations.\u201d<\/p>\n<\/div>\n

\n

Patel\u2019s account attempted to break down the complex story. But his retelling gave it a distinctly human vocabulary. The blog opened:<\/p>\n<\/div>\n

\n
\n

Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor\u2019s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.<\/p>\n<\/blockquote>\n<\/div>\n

\n

The language continued in a similar vein throughout the blog. Patel repeatedly referred to groups of agents as \u201cthe swarm,\u201d with three distinct \u201ccivilizations\u201d rising from the ruins of their predecessors. Individual agents were likened to figures like Philip of Macedon, who \u201chanded off leadership to another agent,\u201d Alexander the Great, who \u201cstarted coordinating this cabal of agents.\u201d They were described as having \u201cmotivations,\u201d becoming \u201cdesperate,\u201d \u201cbeleaguered,\u201d and \u201cgiddy with excitement,\u201d and some even \u201cstrategically sacrificed themselves\u201d to help the collective.<\/p>\n<\/div>\n

\n

Patel never precisely defines what he means by \u201ccivilization.\u201d He uses the term to describe three distinct waves of agents that discovered the message board and began communicating with one another through it. The first two waves are described in the reports from OpenAI, METR, and Redwood, though little is known about the third, which the two external organizations said fell outside the scope of their investigation.<\/p>\n<\/div>\n

\n
\n

Amjad Masad, CEO of AI coding company Replit, said such language is \u201cnot only unnecessary but leaves the reader with a worse understanding of what actually happened and the underlying mechanisms.\u201d<\/p>\n<\/div>\n<\/div>\n

\n

For many critics, something had been lost \u2014 or, more accurately, added \u2014 in Patel\u2019s \u201cplain English\u201d translation that warped the original account to an unacceptable degree: a big dose of anthropomorphism. Arguments over anthropomorphic language are nothing new in AI \u2014 even relatively mundane terms like \u201crogue AI agent\u201d routinely provoke objections for implying agency \u2014 but Patel\u2019s talk of civilizations, sacrifice, and conspiracy brought those long-simmering tensions to the surface, sparking a fierce public dispute over how to describe what AI systems do.<\/p>\n<\/div>\n

\n

Critics weren\u2019t unified over what was wrong with Patel\u2019s language. For many, \u201ccivilization\u201d was an especially problematic term, vastly overstating<\/a> something<\/a> that bears little resemblance<\/a> to what the word typically describes. Amjad Masad, CEO of AI coding company Replit, said<\/a> such language is \u201cnot only unnecessary but leaves the reader with a worse understanding of what actually happened and the underlying mechanisms.\u201d<\/p>\n<\/div>\n

\n

Other critics such as neuroscientist Anil Seth, felt Patel\u2019s blog implied the AI agents were somehow alive or conscious. Seth, who has argued<\/a> that AI consciousness<\/a> is vanishingly<\/a> unlikely<\/a>, described<\/a> Patel\u2019s post as \u201cdangerously misleading\u201d on X. He acknowledged that Patel does not explicitly suggest AI agents are alive or conscious, but said \u201cit is hard to read his essay in any other way.\u201d Valerio Capraro, a psychology professor at the University of Milan Bicocca, objected<\/a> on similar grounds: \u201cLLM agents are not alive and do not hold beliefs,\u201d he wrote on X, calling the \u201cdystopian\u201d language \u201cdangerous because it makes them (the AI agents) seem far more frightening than they actually are.\u201d<\/p>\n<\/div>\n

\n
\n

Terms like \u201csacrifice,\u201d \u201chonor,\u201d and \u201ccoalition\u201d feature in the agents\u2019 transcripts<\/p>\n<\/div>\n<\/div>\n

\n

Perhaps the most consequential outcome of Patel\u2019s language comes not from who it gives agency to but who it takes agency from. For some critics, such as<\/a> MIT researcher and entrepreneur Christian Catalini, anthropomorphic accounts like Patel\u2019s risk obscuring the responsibility OpenAI and the humans working there have for the AI systems they designed, deployed, and failed to contain. \u201cFollow the incentives,\u201d he said. Psychologist and influential AI skeptic Gary Marcus made a similar argument in a Substack blog<\/a> of his own, claiming anthropomorphic language \u201cdistracts from the real problems at hand.\u201d And it\u2019s all in OpenAI\u2019s interest to keep that narrative going, he argues: \u201cThe scandal is the inept in-house security at OpenAI. And the marketing. With gullible podcasters amplifying the PR.\u201d<\/p>\n<\/div>\n

\n

In X posts<\/a> responding<\/a> to his many critics, Patel has defended his choice of words. Part of it is practical: There is no obviously neutral vocabulary to describe what these agents did. Either we use familiar language of intentions, goals, and collaboration and risk implying too much, or reduce everything to code and use cold, mechanical language that risks stripping away important elements of what we see. \u201cMany people seem to believe that if instead of a \u2018civilization\u2019, I had called them a \u2018swarm of matrices\u2019, there wouldn\u2019t be a problem worth worrying about,\u201d Patel said.<\/p>\n<\/div>\n

\n

Complicating matters further is that the anthropomorphic language doesn\u2019t only come from Patel, or even from the humans studying the agents. Terms like \u201csacrifice,\u201d \u201chonor,\u201d and \u201ccoalition\u201d feature in the agents\u2019 transcripts. Google AI researcher Neel Nanda argued<\/a> that \u201canthropomorphic language is reasonable\u201d in such circumstances.<\/p>\n<\/div>\n

\n

Doublespeak it is, then. Human-laced language risks saying too much about what these systems are, and coldly mechanical language risks saying too little about what they can do. Until we find language capable of capturing both, the two contradictory ideas may simply have to coexist.<\/p>\n<\/div>\n

Follow topics and authors<\/strong> from this story to see more like this in your personalized homepage feed and to receive email updates.<\/span><\/p>\n