Reading Room

Outside your usual reading: this one has nothing to do with AI, but it is a clean example of engineering discipline paying off in real, measurable savings. Cloudflare's engineers cut the memory used by their DNS cache in half. 1.1.1.1 is Cloudflare's public DNS service, the system that turns a web address like example.com into the numeric address a computer actually needs to reach it, and it runs on a platform Cloudflare calls Big Pineapple. That platform holds over 250 billion cached DNS answers at any given moment. At that scale, wasting even one byte on every entry costs more than 250 gigabytes of memory across Cloudflare's whole fleet of servers.

Five successive changes to how each cache entry is stored cut the memory used per entry by more than half. Across the fleet that freed roughly 100 terabytes of memory, equal to the RAM in 130 of Cloudflare's newest servers. Normally shrinking data this much slows a system down, because more compact storage usually means more work to read and write. Here the opposite happened: the rate at which new entries can be added rose 43 percent, and the time to look up an entry dropped 19 percent, because fewer memory allocations and better-organized data meant less overhead, not more.

Cloudflare tested this by filling a benchmark cache with entries matching real traffic: 56 percent A records (the basic address lookup), 25 percent AAAA (the newer address format), and 19 percent TXT (a free-text record type). Each entry also carries metadata: when it was cached, how long it stays valid (called the TTL, or time-to-live), and how many times it has been reused. Cache size is not fixed either. It grows wherever Cloudflare uses a feature called EDNS Client Subnet, which returns a different, more localized answer depending on which network a request comes from, meaning several versions of the same query get cached at once.

The biggest single fix targeted a data structure called Vec, a growable list in Rust, the programming language Cloudflare uses here. A Vec reserves a pointer to its data, a count of how many items it holds, and extra room for future growth. Once a DNS answer is cached, though, Cloudflare never changes it again, so that reserved-growth space is pure waste. Switching to a fixed-size container that cannot grow, and so needs no reserved room, was one of several changes made to the eight Vec and String fields each cache entry stores. Combined, these changes added up to more than 15 terabytes in savings across the fleet's 250 billion entries. A second fix combined three separate lists inside each entry into one, avoiding the overhead of separate pointers for each section. A third fix noticed that a DNS record's owner (the domain it belongs to) is usually identical to the domain that was actually queried, so storing it twice was redundant almost every time, except when a CNAME record, which points one domain at another, was involved.

None of this needed new hardware or a bigger budget. It needed someone to look hard at a structure that exists in memory 250 billion times over, and ask what was actually necessary in each copy. That is the kind of unglamorous engineering work that rarely gets a headline and consistently pays for itself, in both cost and speed at the same time.

This lands directly on a problem you have hit with Claude Code on big refactors: the assistant loses track of a large codebase partway through, or forgets what it just learned once the conversation compacts, meaning it compresses old history to save space. A new open-source coding tool called Benzi, posted to GitHub, is built specifically to avoid that failure.

Most AI coding agents today work one of two ways. Either they pull matching snippets of code out of multiple files and hand them to the model, or they turn the code into embeddings, meaning text converted into numbers so a computer can compare meaning, to build an approximate map of the codebase and hand that map over. Both approaches blow up the number of tokens, chunks of text roughly three quarters of a word each, that the model has to read. That slows things down, costs more, and eats into the context window, how much text the model can hold in mind at once. Worse, the model spends its own thinking effort rediscovering the structure of the program each time, then forgets most of that work the moment Claude Code compacts the conversation, or loses all of it on a multi-file refactor because every line number shifts and has to be re-searched from scratch.

Benzi's approach is to never hand the model raw source code in the first place. It answers structural questions directly through tool calls: when the model is about to change a function, it can ask Benzi what functions feed into this one and get a precise answer instead of searching the codebase itself. Benzi's compiler also proactively reports the blast radius, everything a change could affect, before and after an edit, backed by static analysis, which means checking the code for problems without actually running it. Because a static analyzer can be wrong, Benzi tags every fact with one of three confidence levels: RESOLVED, meaning proven by the analysis, CANDIDATE, meaning the analysis could not fully resolve it, and OBSERVED, meaning what genuinely happened when the code ran. For anyone running Claude Code across a large project, that rediscovery complaint will sound familiar: Benzi's bet is that structural facts should be worked out once, by a deterministic tool, and looked up cheaply after that, rather than re-derived by the model every single time.

The post backs the claim with a direct comparison of how many lines of source code each tool had to read to finish the same tasks: Benzi running on Sonnet read 9,125 lines, Claude Code on Sonnet read 20,704, a tool built by DeepSeek read 43,598, and OpenCode read more than 65,000 and was disqualified for failing repeatedly. Benzi reports 78.2 percent on SWE-bench Verified, a standard test that measures how well a coding agent fixes real, verified software bugs. It currently supports Python, JavaScript, TypeScript, Java, C#, C++, C, Go, Rust, and Ruby, and can resolve HTML, CSS, and JavaScript layout conflicts deterministically rather than by guessing.

This is one developer's GitHub project and benchmark, not an independent audit, so treat the numbers as a claim to check rather than settled fact. But the diagnosis, that today's coding agents drown in their own retrieved code and then forget it, matches what you have seen on your own large refactors. Worth a real trial on the monorepo before trusting it on anything that matters.

Outside your usual reading: Germany's Sovereign Tech Agency is putting 508,640 euros into Flatpak, the packaging system that lets most Linux desktop apps run boxed off from the rest of the system, over a two-year project. The money funds named contractors to close gaps in areas like microphone versus speaker permissions and VPN support, the kind of unglamorous plumbing work that rarely attracts private funding on its own.