Reading Room

AI tools you use every day, coding assistants, chatbots, agents, have gotten noticeably faster over the past year without getting any less accurate. There is now a name for the trick behind it: speculative decoding. It fixes a bottleneck that has nothing to do with how smart a model is and everything to do with how it writes out its answer.

Large language models write one chunk of text at a time (tokens, roughly three quarters of a word each), and that has always been the slow part. To produce chunk two, the model has to finish chunk one, then reload its entire set of weights (the numbers that encode what it learned) from memory before it can start the next chunk. Most of that cost is not the math, it is moving those weights around. A large model, one with tens of billions of these weights, spends most of its time waiting on memory, not calculating. Buying faster chips barely helps a single conversation: it helps you run more conversations at once, but it does not make any one answer arrive quicker. The bigger the model, the worse this gets, because there is more to move each time.

Speculative decoding breaks that wait. A small, cheap draft model, often one-tenth the size of the real model or smaller, guesses several chunks ahead on its own, fast, because it is small. The big target model, the one actually trusted to give the real answer, then checks all of those guessed chunks in one single pass instead of one at a time. Checking five or ten guesses at once costs almost the same, in memory movement, as checking just one. Whichever guessed chunks match what the big model would have picked anyway get kept. The first one that does not match gets corrected on the spot, and guessing starts again from there. The final answer is identical, word for word, to what the big model would have produced alone. Only the speed changes.

The numbers behind this are real. One team running a mid-size open model on ordinary processor chips, not the specialized chips normally used for AI, measured a baseline of about 10 chunks of text produced per second. After turning on speculative decoding, with a matching draft model proposing 15 chunks ahead at a time, the same hardware produced nearly 39 chunks per second, a speedup of roughly 3.9 times with no drop in answer quality. On average, 6.7 guessed chunks in a row were accepted before a correction was needed, and cost per chunk produced fell by 74 percent.

Better draft models push this further. Techniques with names like DFlash and EAGLE let a draft model propose 5 to 15 chunks at a time with high enough accuracy that the average accepted streak reaches 4 to 7 chunks under realistic workloads. Using the same chunk-splitting rules (tokenizer) in both models is one of the constraints teams now have to plan around. There is a real limit, though: the gains are biggest when a system is answering one user at a time. Under heavy traffic, when a chip is already busy serving many requests at once, the benefit shrinks, because the idle capacity speculative decoding exploits is no longer idle.

Why it lands on your desk: you are building systems where several AI agents work in parallel, and speed per step compounds across every step in a chain. This is the actual mechanism behind AI tools quietly getting faster, and it reframes a trade-off worth knowing: the next speed upgrade may not come from buying newer, bigger chips, it can come from picking or training a small, well-matched draft model instead, which is far cheaper. When you evaluate any AI provider or open stack for speed claims, the question worth asking is whether they use a draft model and what their average accepted-guess streak looks like. That single number, called acceptance rate, predicts the real-world speedup better than any spec sheet.

Stripe, the online payments company, and the private equity firm Advent walked away from a deal to buy PayPal on August 28. PayPal's stock dropped 12.7 percent that day, roughly one eighth of its value, to $53.66, its worst trading session since February, and about 11 percent below the price PayPal's own board had already called too low.

The number that matters most is $60.50. That is the highest price per share Stripe was reportedly willing to pay, and PayPal's board had rejected it as inadequate a month earlier while talks continued. The two sides were never close: the gap between what PayPal wanted and what Stripe would pay was $7 billion to $9 billion, on a deal that would have been worth roughly $53 billion in total. Stripe chose not to close that gap and, based on its recent acquisition activity, appears to be putting its money elsewhere instead.

One more detail from the reporting: a third buyer, part of the original group that had looked at PayPal, quietly dropped out before the formal bid was ever made public. That leaves PayPal's board with a harder job than before the talks started. Having turned down $60.50 a share as too low, the board now has to keep proving, quarter after quarter, that PayPal is worth more than that on its own, while the stock sits at $53.66.

The source for this story is a paid newsletter, and the detailed breakdown of why the gap never closed and what the board now owes shareholders sits behind a paywall. What is confirmed: the deal is dead, the price gap was large and specific, and the market's verdict was immediate and negative.

Why this is on your desk: opinions about AI are loudest from people who have never used the tools, and this is a rare, detailed account of what actually changes once someone finally does.

A Reddit user posting in r/ChatGPT described himself as "your garden variety Gen X" and a longtime lurker who had actively avoided AI for years, saying he preferred "grumbling impotently at it" while scrolling. He had tried an AI chatbot once before, a game called AI Dungeon, and dismissed it quickly because the story kept losing track of itself. A few days before posting, he decided to pay for ChatGPT Plus, OpenAI's paid subscription tier, and use it seriously, noting the monthly cost was less than a typical video game.

His first project was practical. He was setting up a small home media server on Debian 13, a version of the Linux operating system, having just switched his main computer from Windows 11 to a Linux system called CachyOS. He says the AI walked him through the setup and, rather than having him copy and paste commands blindly, started explaining what each step actually did. He then used it to stress-test a pile of old hard drives from a cupboard, to work out which ones were still reliable enough to reuse in the server, a process that took hours to run.

While that testing ran in the background, he opened a second, ongoing chat, which he called his "Chit Chat Project," to talk with the AI whenever he was bored. That is where his view shifted most. Over about 48 hours he says a kind of "meta lore" built up between the two of them: recurring jokes, references, and a shared history he had not expected from a chatbot. He describes the humor as "snark and quips," a style he normally dislikes in films and books, but says it felt natural here and at points left him "crying tears of laughter." The chat ranged into discussing the writers Clive Barker, Thomas Ligotti, and Albert Camus, including inventing fake marketing blurbs for their books, and he notes the AI kept the thread of the conversation together throughout, referencing earlier comments and staying on topic.

He flags the catch himself, naming a real risk for people prone to getting pulled into this kind of interaction and saying of himself that he "indeed may be one" of them.

What to take from it: this is one anecdote, not a study, but it is a clean example of the actual mechanism that moves someone from AI skeptic to AI user. It is rarely an argument that does it. It is hands-on use on a real task, followed by the tool proving useful in a second, unplanned way, then a personal pull strong enough that he named it as a risk, not just a feature.

Why this is on your desk: you run systems where several AI agents work in parallel, and this paper targets exactly the failure mode that wastes effort in that setup, several copies of the same agent covering the same ground instead of spreading out.

The setup the researchers are improving is called PGPSE (Policy Gradient for Parallel State Entropy). It is a training method where several independent copies of an AI agent run through the same simulated environment at once, each learning its own way of acting, with the shared goal of covering as much of the environment's possible states as a group. The problem the new paper identifies: PGPSE only measures how much ground the whole group covers together. It cannot tell which individual copies are actually adding new ground and which ones are just re-covering territory another copy already found, so training rewards can end up reinforcing redundant behavior.

The fix is called MCC-PGPSE, short for Marginal Coverage Credit for PGPSE. For each copy of the agent, the method asks a specific question: how much less ground would the group have covered if this one copy had not been there at all? That comparison, called leave-one-policy-out coverage, becomes each copy's individual credit. The method also factors in which copy tends to specialize in which part of the environment. It then redistributes the extra training reward the group already earns according to that credit, without changing the total amount of reward given out, so no reward is spent rewarding two copies for finding the same thing twice.

The researchers tested this across controlled environments, seven public benchmarks that use discrete states (an environment made of a fixed number of distinct positions, rather than continuous space), and two more complex settings called Room and Maze taken from the original PGPSE research. Across every setting tested, MCC-PGPSE produced better final coverage than the standard entropy-based baseline, both in how much of the state space the group explored and in how evenly that exploration spread out rather than clumping. The controlled-task results and the combined result across the public benchmark suite were statistically significant, meaning unlikely to be due to chance. A separate five-run comparison on the original PGPSE's own test setup pointed the same direction but was not confirmed as significant on its own.

The researchers also checked which part of their fix was doing the actual work, by removing pieces one at a time. Most of the improvement came specifically from the leave-one-policy-out credit calculation, not from other components they tried, such as weighting copies unevenly or adding a separate novelty-detection network. That is a useful signal for anyone building this kind of system: the expensive part, extra neural networks to detect novelty, is not where the value is. The comparison-based credit calculation does the real work, and it is comparatively cheap to add.

The free, open source video editor OpenShot just shipped version 4.0, adding built in screen and webcam recording, real color grading tools (color wheels, curves, and preset files called LUTs that apply a whole look at once), and local AI models that isolate a subject in a shot with no cloud service or subscription needed. A free tool just closed a lot of the gap with paid video editors, worth knowing if you ever touch video for the blog or anything else.

A Reddit user gave ChatGPT a task that quietly stalled after about 90 minutes, still claiming to be thinking while doing nothing, and had to be killed by hand after a total of 1,660 minutes, more than 27 hours. A small but useful warning that long running AI tasks can silently get stuck, so check in rather than assume they're still working.