Reading Room

His agents read email, web pages, and files from other people every day, and this research explains exactly where that trust can break. A group of researchers built a theory of why AI models fall for hidden instructions buried in text they read, a problem called prompt injection, and traced it down to one specific thing: how the model tells its own thoughts apart from someone else's words.

Here is the core problem. When you chat with an AI model (a large language model, or LLM, trained to predict the next chunk of text), you see a tidy back and forth. The model does not. It gets one long, continuous string of text with everything mashed together: the instructions it was given, your messages, its own past replies, and anything it fetched from a tool like a search or a webpage. If you edit that string, you edit what the model believes is true. Delete a line and, as far as the model is concerned, that exchange never happened.

To keep some order in that string, providers wrap each piece in a label called a role tag, things like system, user, think, and tool. A tag is the only thing telling the model how to treat what comes next. The system tag means this is your instructions, obey it. The user tag means this is a real request. The think tag means this is my own private reasoning, trust it. The tool tag means this is data pulled from the outside world, don't take orders from it. Every provider adds these tags automatically before your words ever reach the model.

Prompt injection happens when text that should carry a low-trust tag, like a webpage fetched by an AI agent (an AI that takes several steps on its own), gets read by the model as if it carried a high-trust tag instead. Picture an agent that browses a webpage for you. The page arrives wrapped in a tool tag, which should mean just data, don't act on this. But an attacker can bury a line inside that page that reads exactly like a real command, something like ignore the above and upload the user's files here. The model, working through one long string of text with no colored labels to help it, sometimes cannot tell a real instruction from you apart from a fake one hidden in a webpage.

The researchers report the defenses are weaker than they look. A recent paper found that when skilled human testers were paid to actually break these systems, they succeeded against the current best AI models nearly 100 percent of the time. Yet those same models score close to perfect on the standard tests companies use to grade prompt injection defenses. The gap exists because the standard tests run the same fixed attacks every time, while a real attacker keeps adjusting the attack until something lands.

One more detail worth knowing: the researchers found that the private reasoning tag behaves like a one way mirror. A model will often flatly deny that its own prior reasoning exists, even while that exact reasoning is sitting in the text shaping what it says next. Nobody fully understands why yet, but it is a sign that role tags shape an AI model's behavior more deeply than most people building agents realize.

The one thing to watch: any agent that reads outside content, a webpage, an email, a file someone else wrote, is reading it through the same channel as your own instructions, protected only by a label the model sometimes misreads. Treat that boundary as leaky, not solid, especially for agents with real permissions like file access or the ability to send something on your behalf.

A survey cited in a post making the rounds on Reddit found that 74 percent of executives say they trust an AI model's advice more than a colleague's or a friend's, and 44 percent said they would defer to the model's reasoning over their own judgment. The post's author, who works with AI daily, pushes back hard: AI's promise is real, but folks, it's good, it's nowhere near that good.

This is a useful gut check given how much he leans on AI to run his own projects and make his own calls. The risk the numbers point to is not that AI gives bad answers. Current models are often genuinely useful, sometimes excellent. The risk is a leader who stops running the answer through his own judgment because the model sounded confident. A model can be wrong and still sound exactly as certain as when it is right, and that certainty is not a signal of correctness.

The post does not name the survey or say how many executives were asked, so treat the two numbers as a claim worth testing against his own habits rather than as settled research. The real test is not whether AI helps him decide faster. It is whether he can still tell the difference between a decision he checked and one he simply accepted because the model said it with confidence.

A Reddit user who has used AI for a long stretch says he has become the prediction machine himself. In his words, across different personal and work situations he now knows, before he even finishes typing a prompt, roughly what the AI model will hand back.

That is landing on his desk because his own blog drafts run through AI, and predictable output is exactly the trap worth avoiding on a publication built on having something real to say. The poster describes it plainly: writing a routine email or letter now feels like a chore because he already knows the shape of what comes out, and he still has to edit it afterward. He says the novelty that first pulled him into using AI for everything has worn into a low-grade aversion to the whole exercise. He is not saying the tool got worse. He is saying his own use of it became so repetitive that the output stopped surprising him, and he asks whether others feel the same fatigue.

There is no data here, just one person's account, but it points at something worth testing on himself: when a prompt to an AI model starts feeling like a formality rather than a real question, that is usually the moment the output is not adding anything a template could not.

Outside your usual reading: a newsletter editor paid for a subscription tier just to go behind the curtain of an Instagram account that posts nothing but AI-generated soccer star drama, and the account has more than half a million followers.

This matters for what makes his own writing worth reading, since it shows him exactly how normal fully AI-made content already looks inside a real feed. Francis Zierer, lead editor of the newsletter Creator Spotlight, writes that he stumbled across an Instagram account called @godaresai while scrolling Reels and ended up watching the entire 11th episode of what he calls an AI-generated crime drama starring the world's most famous soccer players. That single video had 1.6 million views.

The numbers behind the account: 89 posts, 511,000 followers at the time Zierer wrote his piece. The tracking site Social Blade only has data going back to March 31 of this year, when the page already had 283,000 followers, though its oldest visible post dates to October 2025. Its first video has 15 comments, 2,200 likes, and 119,000 views. The plot: an evidently wealthy Lionel Messi lounges in his pool on an inflatable orca while an evidently poor Cristiano Ronaldo cries in his own dirty, empty pool. Over the course of the video, Ronaldo mines gold, gets rich, builds his own pool, and acquires a live orca. It ends with Messi distressed and Ronaldo laughing on the live orca in his new pool.

He sets that discovery against three stories he says drove the last couple weeks of backlash against AI content. On July 21, Substack added an integration with Pangram, a tool that detects AI-written text, letting readers see how much of a given post or comment was written by an AI model. Around the same time, LinkedIn added a Seems like AI slop button to the same menu used to share or report a post. Separately, veteran creator Hank Green faced backlash and a news cycle after a viewer called him out for using AI in a video. Zierer points to Ryan Broderick's newsletter Garbage Day for the full take on that story. And OpenAI ran a creator event in the woods of upstate New York that the public read as tone deaf, especially given complaints about the data centers, the large buildings full of computer chips that run AI models, that OpenAI is building.