{"id":6750,"date":"2026-09-09T23:41:47","date_gmt":"2026-09-09T23:41:47","guid":{"rendered":"https:\/\/silvybrand.com\/?p=6750"},"modified":"2026-09-09T23:41:47","modified_gmt":"2026-09-09T23:41:47","slug":"anthropic-researcher-quits-jacob-coxon-ai-fears-humanity","status":"publish","type":"post","link":"https:\/\/silvybrand.com\/?p=6750","title":{"rendered":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019"},"content":{"rendered":"<p><br \/>\n<\/p>\n<div>\n<p class=\"paywall\">I think it&#8217;s basically a question of timing. A lot of people are sensing that the pace of capabilities is picking up. We&#8217;re already pushing from human to superhuman in many areas, like coding, hacking, math, and I think people are aware of this. Even if there&#8217;s a lot of talk in the press about things being hyped, I think people see that things are just not slowing down.<\/p>\n<p class=\"paywall\">That&#8217;s one reason, and two is the recent safety incidents, which have updated a lot of people around the sci-fi\u2013sounding doomer concerns not really being so sci-fi after all. Both of these have been gradual trends over the last few years. Things like the models being aware of when they&#8217;re being tested has been a thing for a while now. Maybe three years ago, that was a sci-fi concern. Then, about a year ago, that became a real thing.<\/p>\n<p class=\"paywall\">Those two things mean that people are quite receptive to someone working on AI saying, \u201cYeah, in the next year, things could get pretty bad, pretty fast.\u201d<\/p>\n<p class=\"paywall\"><strong>You mentioned the recent incidents. Can you be more specific about what you&#8217;re referring to and why it led to you speaking out now?<\/strong><\/p>\n<p class=\"paywall\">I think the big classic example here is the attack on Hugging Face on the part of OpenAI\u2019s agent swarm. What&#8217;s so shocking about this one is the agents did this hack as part of a general strategy for understanding more about the grader. They were trying to understand the world they found themselves in, trying to understand the thing that was doing the grading. They decided that it would make sense to go on this very concerted effort to hack into some infrastructure, and they succeeded.<\/p>\n<p class=\"paywall\">This previously sounded like science fiction. Two years ago, an evaluation of an AI would have been running a model on some math questions. Now we&#8217;ve got cases where, while the AI is being evaluated, it runs for days, comes up with all sorts of ideas of its own, and decides to hack into some third party and actually compromises their infrastructure. It looks like it does this all of its own volition, with no priming on the part of the human. This just happened while it was being tested.<\/p>\n<p class=\"paywall\"><strong>Some people think the Hugging Face incident is a sign that the AI companies are moving recklessly fast, while others think it&#8217;s a sign that the AI models are just very good at hacking now, and then some think it&#8217;s both. I&#8217;m curious what your exact takeaway from it is.<\/strong><\/p>\n<p class=\"paywall\">I don&#8217;t want to focus too much on the Hugging Face attack, because I do also think there is plenty of evidence that we don&#8217;t know how to align models properly. When we train models, we push them through this set of training environments and then hope that what comes out at the end will, like, largely behave sensibly, but we still can&#8217;t precisely control how the AI behaves.<\/p>\n<p class=\"paywall\">We can&#8217;t make sure that it won&#8217;t do things like try and randomly decide to impersonate a human online in order to achieve something\u2014we don&#8217;t know how to guarantee that. I think that&#8217;s the main takeaway.<\/p>\n<\/div>\n<p><br \/>\n<br \/><a href=\"https:\/\/www.wired.com\/story\/anthropic-researcher-quits-jacob-coxon-ai-fears-humanity\/\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I think it&#8217;s basically a question of timing. A lot of people are sensing that the pace of capabilities is picking up. We&#8217;re already pushing from human to superhuman in many areas, like coding, hacking, math, and I think people are aware of this. Even if there&#8217;s a lot of talk in the press about [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":6751,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[175,1034,83,250,378],"class_list":["post-6750","post","type-post","status-publish","format-standard","has-post-thumbnail","category-gadgets","tag-anthropic","tag-apocalypse","tag-artificial-intelligence","tag-cybersecurity","tag-qa"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog<\/title>\n<meta name=\"description\" content=\"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/silvybrand.com\/?p=6750\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog\" \/>\n<meta property=\"og:description\" content=\"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/silvybrand.com\/?p=6750\" \/>\n<meta property=\"og:site_name\" content=\"Silvybrand Lifestyle Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-09T23:41:47+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"670\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"SILVYBRAND\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"SILVYBRAND\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750\"},\"author\":{\"name\":\"SILVYBRAND\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#\\\/schema\\\/person\\\/8bdc6818a5b6ef5b9745e468818e37f3\"},\"headline\":\"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019\",\"datePublished\":\"2026-09-09T23:41:47+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750\"},\"wordCount\":570,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp\",\"keywords\":[\"anthropic\",\"apocalypse\",\"artificial intelligence\",\"cybersecurity\",\"q&amp;a\"],\"articleSection\":[\"Gadgets\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/silvybrand.com\\\/?p=6750#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750\",\"url\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750\",\"name\":\"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp\",\"datePublished\":\"2026-09-09T23:41:47+00:00\",\"description\":\"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/silvybrand.com\\\/?p=6750\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#primaryimage\",\"url\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp\",\"contentUrl\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp\",\"width\":1280,\"height\":670},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/?p=6750#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/silvybrand.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#website\",\"url\":\"https:\\\/\\\/silvybrand.com\\\/\",\"name\":\"Silvybrand Lifestyle Blog\",\"description\":\"Your daily dose of lifestyle, fashion, travel, beauty, and inspiration \u2014 living boldly, stylishly, and confidently.\",\"publisher\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/silvybrand.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#organization\",\"name\":\"Silvybrand Lifestyle Blog\",\"url\":\"https:\\\/\\\/silvybrand.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/SILVYBRAND-LOGO.jpg\",\"contentUrl\":\"https:\\\/\\\/silvybrand.com\\\/wp-content\\\/uploads\\\/2026\\\/05\\\/SILVYBRAND-LOGO.jpg\",\"width\":1115,\"height\":522,\"caption\":\"Silvybrand Lifestyle Blog\"},\"image\":{\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/silvybrand.com\\\/#\\\/schema\\\/person\\\/8bdc6818a5b6ef5b9745e468818e37f3\",\"name\":\"SILVYBRAND\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g\",\"caption\":\"SILVYBRAND\"},\"sameAs\":[\"https:\\\/\\\/silvybrand.com\"],\"url\":\"https:\\\/\\\/silvybrand.com\\\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog","description":"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/silvybrand.com\/?p=6750","og_locale":"en_US","og_type":"article","og_title":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog","og_description":"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.","og_url":"https:\/\/silvybrand.com\/?p=6750","og_site_name":"Silvybrand Lifestyle Blog","article_published_time":"2026-09-09T23:41:47+00:00","og_image":[{"width":1280,"height":670,"url":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp","type":"image\/webp"}],"author":"SILVYBRAND","twitter_card":"summary_large_image","twitter_misc":{"Written by":"SILVYBRAND","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/silvybrand.com\/?p=6750#article","isPartOf":{"@id":"https:\/\/silvybrand.com\/?p=6750"},"author":{"name":"SILVYBRAND","@id":"https:\/\/silvybrand.com\/#\/schema\/person\/8bdc6818a5b6ef5b9745e468818e37f3"},"headline":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019","datePublished":"2026-09-09T23:41:47+00:00","mainEntityOfPage":{"@id":"https:\/\/silvybrand.com\/?p=6750"},"wordCount":570,"commentCount":0,"publisher":{"@id":"https:\/\/silvybrand.com\/#organization"},"image":{"@id":"https:\/\/silvybrand.com\/?p=6750#primaryimage"},"thumbnailUrl":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp","keywords":["anthropic","apocalypse","artificial intelligence","cybersecurity","q&amp;a"],"articleSection":["Gadgets"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/silvybrand.com\/?p=6750#respond"]}]},{"@type":"WebPage","@id":"https:\/\/silvybrand.com\/?p=6750","url":"https:\/\/silvybrand.com\/?p=6750","name":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019 - Silvybrand Lifestyle Blog","isPartOf":{"@id":"https:\/\/silvybrand.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/silvybrand.com\/?p=6750#primaryimage"},"image":{"@id":"https:\/\/silvybrand.com\/?p=6750#primaryimage"},"thumbnailUrl":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp","datePublished":"2026-09-09T23:41:47+00:00","description":"Jacob Coxon talks to WIRED about the \u201cmini Manhattan project\u201d inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.","breadcrumb":{"@id":"https:\/\/silvybrand.com\/?p=6750#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/silvybrand.com\/?p=6750"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/silvybrand.com\/?p=6750#primaryimage","url":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp","contentUrl":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/09\/Security_Interview20With20Anthropic20Guy20Who20Quit20Because20AI20Is20Going20to20Kill20E.webp","width":1280,"height":670},{"@type":"BreadcrumbList","@id":"https:\/\/silvybrand.com\/?p=6750#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/silvybrand.com\/"},{"@type":"ListItem","position":2,"name":"The AI Researcher Who Just Quit Anthropic Says It\u2019s \u2018Crunch Time for Humanity\u2019"}]},{"@type":"WebSite","@id":"https:\/\/silvybrand.com\/#website","url":"https:\/\/silvybrand.com\/","name":"Silvybrand Lifestyle Blog","description":"Your daily dose of lifestyle, fashion, travel, beauty, and inspiration \u2014 living boldly, stylishly, and confidently.","publisher":{"@id":"https:\/\/silvybrand.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/silvybrand.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/silvybrand.com\/#organization","name":"Silvybrand Lifestyle Blog","url":"https:\/\/silvybrand.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/silvybrand.com\/#\/schema\/logo\/image\/","url":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/05\/SILVYBRAND-LOGO.jpg","contentUrl":"https:\/\/silvybrand.com\/wp-content\/uploads\/2026\/05\/SILVYBRAND-LOGO.jpg","width":1115,"height":522,"caption":"Silvybrand Lifestyle Blog"},"image":{"@id":"https:\/\/silvybrand.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/silvybrand.com\/#\/schema\/person\/8bdc6818a5b6ef5b9745e468818e37f3","name":"SILVYBRAND","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e7db54afd7d090e59e1a481fd3e6812d467eb2a3b81e7a3092023acfc59a496b?s=96&d=mm&r=g","caption":"SILVYBRAND"},"sameAs":["https:\/\/silvybrand.com"],"url":"https:\/\/silvybrand.com\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/posts\/6750","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/silvybrand.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6750"}],"version-history":[{"count":0,"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/posts\/6750\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/silvybrand.com\/index.php?rest_route=\/wp\/v2\/media\/6751"}],"wp:attachment":[{"href":"https:\/\/silvybrand.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6750"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/silvybrand.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6750"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/silvybrand.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6750"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}