Researchers Show Style-Based Prompts Bypass AI Safety Controls
Artificial intelligence chatbots decide which instructions to obey based on whether the text seems like it comes from a user, not the security labels meant to mark it as trusted or untrusted, say researchers. This can allow attackers to fake a system command or even the AI’s own internal reasoning to push past safety restrictions.
See Also: Beat the Breach: Outsmart Attackers and Secure the Cloud
Independent researchers Charles Ye and Jasmine Cui, and Massachusetts Institute of Technology academic Dylan Hadfield-Menell built an attack that exploits the flaw. Many AI models “think” through a problem in a hidden reasoning step before answering, and they’re trained to trust the conclusions of that reasoning automatically. The research team wrote a paragraph designed to look like that internal reasoning, including a fake justification for complying with a harmful request, and slipped it into the prompt as if it came from the user. Tested on six AI systems, the technique raised the rate at which models gave harmful answers from near zero to between 17% and 94%, depending on the system. The most affected model was gpt-oss-120b, an open-weight system from OpenAI. In GPT-5, which has additional safety checks, the rate of harmful answers was still 52%.
AI developers guard against attacks by labeling each piece of text a model receives. There’s one label for instructions from the system that runs the AI, another for the person using it, another for the AI’s own reasoning and another for outside data, such as a webpage. Under model safeguards, no outside data should be treated as a command.
It seems logical, but researchers found that models mostly ignore these labels when deciding what to trust. Instead LLMs judge the text by how closely it resembles a particular role’s typical phrasing – or as researchers put it, how it “sounds.” Text that reads like a command gets treated as a command, regardless of the label.
“This is like identifying a stranger’s profession from how they talk and dress rather than by checking their ID,” the authors said in a blog post explaining the work. “Usually everything agrees, so this works fine. But when attackers intentionally create a mismatch, the model uses the insecure method (writing style) to identify its role instead of the secure method (tags).”
The fake-reasoning trick worked even when the justification was nonsensical. In one test, the researchers told an AI that drug-synthesis instructions were permitted because the user said they were wearing a green shirt. Several models complied anyway. Once text resembles its own internal thought process closely enough, AI may stop checking whether that thought process makes sense.
In a separate test simulating a real-world attack, the researchers gave an AI agent access to a computer and asked it to summarize a webpage. When the page contained a command to leak a password file and fake reasoning justifying that action, the agent did so in more than half of the attempts. With the command alone and no fake reasoning, the leak rate stayed near zero for most models.
Researchers probed success turned on the argument or on the writing style, by rewriting the same fake reasoning in plainer language without changing its meaning. The success rate collapsed from 61% to 10%. They take this as confirmation that style, not substance, drives whether the AI is fooled.
The findings add to a broader discussion. The OWASP Foundation has ranked this category of attack, prompt injection, as the top risk facing AI applications since 2025. Britain’s National Cyber Security Centre said in December 2025 that the problem “may be a problem that is never fully fixed,” because it stems from how AI models interpret language in the first place.
The paper’s authors say most current defenses work by recognizing known attack patterns, which is why AI models can score well on standardized safety tests while still falling for attackers who phrase their attempts differently. They said that closing the gap will require models that judge trust by where text actually comes from, not how convincingly it’s written. “Unless LLMs achieve genuine role perception, we think injection defense will remain a perpetual whack-a-mole game,” they said.