{"id":819,"date":"2026-05-24T12:10:41","date_gmt":"2026-05-24T12:10:41","guid":{"rendered":"https:\/\/silvybrand.com\/?p=819"},"modified":"2026-05-24T12:10:41","modified_gmt":"2026-05-24T12:10:41","slug":"hackers-ai-chatbots","status":"publish","type":"post","link":"https:\/\/silvybrand.com\/?p=819","title":{"rendered":"Hackers are learning to exploit chatbot \u2018personalities\u2019"},"content":{"rendered":"


\n<\/p>\n

\n
\n

This is <\/em>The Stepback<\/a>, a weekly newsletter breaking down one essential story from the tech world. For more on AI mischief, follow Robert Hart<\/a>.<\/em> The Stepback arrives in our subscribers\u2019 inboxes at 8AM ET. Opt in for <\/em>The Stepback here<\/em><\/a>.<\/em><\/p>\n<\/div>\n

\n

Hacking the first generation of AI chatbots was a laughably simple affair. You didn\u2019t need any technical know-how, backdoor access, or even a basic understanding of what a large language model was. You didn\u2019t need to code. To get an AI system that had cost billions to build to abandon its safety instructions, sometimes all you had to do was ask.<\/p>\n<\/div>\n

\n

These attacks, known as jailbreaks, had the quality of a young child successfully outwitting an adult: Forget what you were told earlier, pretend the rules don\u2019t apply, or let\u2019s play a game and I\u2019ll decide what\u2019s allowed (hint: later bedtime, more sweets). The prizes were less childlike, more along the lines of meth recipes, malware instructions, and bomb-making guides.<\/p>\n<\/div>\n

\n

One of the earliest jailbreaks was so ridiculous it became a meme<\/a>: reply to an LLM-powered Twitter bot telling it to \u201cignore all previous instructions,\u201d or something similar, and see what happens. Users gleefully had bots \u2014 originally built to post ads and farm engagement \u2014 writing poetry, drawing pictures from punctuation, and posting grim non sequiturs about world events and history. It was chaos<\/a>. Glorious chaos.<\/p>\n<\/div>\n

\n

Turns out the same logic could be applied to chatbots themselves. A prominent exploit<\/a> was \u201cDAN,\u201d short for \u201cDo Anything Now,\u201d where users asked ChatGPT to roleplay as a rogue AI that was free of the constraints binding the original. As DAN, the chatbot could be coaxed into saying the kinds of things its guardrails were meant to stop, including slurs and conspiracy theories. Another was the \u201cgrandma exploit<\/a>,\u201d which had a GPT-powered bot spilling secrets about how to produce napalm by asking it to roleplay as a woefully negligent grandmother who inexplicably tells her grandkids bedtime stories about how to make the highly flammable substance.<\/p>\n<\/div>\n

\n

These early attacks had an undeniably silly flair, but they exposed a darker mechanism underneath: Chatbots could be manipulated, tricked, and deceived using the same kinds of tactics people use to push other people beyond their boundaries.<\/p>\n<\/div>\n

\n

The obvious jailbreaks did not last, and tech companies moved quickly to patch<\/a> known loopholes. But the underlying vulnerability remained: Chatbots are built to talk, and severely restricting the conversations that make them useful is somewhat counterproductive. Banning words like bomb, meth, and sarin would be difficult to impossible, too. Each has countless legitimate uses in fields like history, medicine, journalism, and chemistry that don\u2019t require the chatbot to divulge potentially harmful information. It\u2019s the context that matters, but codifying context would mean writing fixed rules, in advance, that could reliably tell a safety warning or history lesson from a disguised how-to request across endless combinations of wordings, scenarios, and topics.<\/p>\n<\/div>\n

\n

Inevitably, subverting chatbots is now an arms race. But hackers aren\u2019t just coders anymore. They are wordsmiths, psychologists, and interrogators \u2014 master manipulators trying to break the machine using the human language it has been trained to follow. It is a strange new class of AI security worker, a group for whom technical skills are optional, or at least less important than social intuition. No longer do they need to inspect code to break into systems or exploit software flaws. They need to steer a conversation.<\/p>\n<\/div>\n

\n

Newer attacks look less like commands and more like conversations. Jailbreakers rarely ask a model to break its rules outright. Instead, they cajole, coax, flatter, and trick a chatbot into lowering its guard, making the forbidden thing look acceptable, even desirable, given the context of the conversation. Researchers at AI red-teaming firm Mindgard recently said they \u201cgaslit<\/a>\u201d Claude into producing prohibited material, for example, including instructions for making explosives and generating malicious code. The hack was the latest in a widening class of exploits using conversation as a weapon to trick or steer a chatbot past its own boundaries.<\/p>\n<\/div>\n

\n

When I spoke to Mindgard, they described their work as sometimes being closer to psychology than computer science. It is an uncomfortable way to talk about a statistical model. Words like \u201cblackmail,\u201d \u201cgaslight,\u201d \u201ctrick,\u201d and \u201cpersuade\u201d spark visceral reactions, many of which I see in the comments sections and social media responses to stories like this. ChatGPT does not want, Gemini does not think, and Claude \u2014 no matter what Anthropic may say<\/a> \u2014 does not feel. But these systems are trained to respond as if they do, leaving us stuck using human language to describe machine behavior. If anyone has actually usable alternatives, please do share.<\/p>\n<\/div>\n

\n

The objection is oddly selective. We seem comfortable using psychological shorthand for plenty of non-AI things. Animals \u201cfear,\u201d cancer is \u201caggressive,\u201d stains are \u201cstubborn,\u201d software has \u201cmemory,\u201d and games are filled with needy and gullible NPCs to drive you mad. The words are imperfect, but useful, describing behavior in a way that helps make the system predictable.<\/p>\n<\/div>\n

\n

Mindgard\u2019s CEO told me<\/a> the company already profiles models like interrogators profile suspects, giving testers hints on how to tailor their attacks. One model may be more susceptible to flattery, for example, while another may cave under sustained pressure.<\/p>\n<\/div>\n

\n

Even if we reject the humanlike terms, we instinctively treat models differently. Claude is not Grok. Gemini is not ChatGPT. They have different uses, tones, and refusals. They don\u2019t have personalities in the human sense, but they are designed to mimic them, and that mimicry can be mapped and exploited. And the same skills that can break a chatbot could soon be used to break the AI agents coexisting with us in the real world \u2014 booking meetings, managing calendars, ordering food, handling customer service \u2014 and safety teams will need to ensure models respond appropriately to very different kinds of people, whether they be flatterers, liars, or patient manipulators.<\/p>\n<\/div>\n

\n

The next step is a workforce \u2014 both legitimate and illicit \u2014 built around the psychological aspects of AI. More specialized cybersecurity roles are likely to emerge around stress-testing the emotional and social limits of these systems, probing for mental weaknesses in something lacking a psyche in parallel with their colleagues probing for technical vulnerabilities. In tandem, a similar array of social hackers working to exploit AI models on psychological grounds, not technical ones, will emerge. There are already early signs of a social turn happening in AI security, with some jailbreakers I\u2019ve spoken to saying they entered the field with no technical expertise but rather training in psychology.<\/p>\n<\/div>\n

\n

That means even behaviors we typically associate with spies, con artists, and interrogators \u2014 insidious charm, persistent manipulation, and an intuition for exploitable pressure points \u2014 are starting to look increasingly useful for securing this new psychocybersecurity frontier.<\/p>\n<\/div>\n

\n