Give two coding agents separate worktrees. Give them separate tasks. Each changes a different file, runs the existing tests, and reports success. Git merges both changes without a conflict.

The combined code can still fail the test that both agents just passed.

Both agents can report their results accurately. The test suite can contain the exact check needed to catch the bug. The mistake happens when someone treats two separate passing results as evidence about a third version of the program.

Here is a small, deliberately constructed example. Two regions reserve capacity from a shared budget of 100 units. The repository stores their reservations in different files, and an existing check adds them together.

VersionEastWestTotalCheck Base404080PASS East branch6040100PASS West branch4060100PASS Combined6060120FAIL

Both branches start at the base. Each raises its region's reservation to 60. Each runs exactly the same check. The files don't overlap, so Git has no conflict to ask anyone to resolve.

The reproduction script creates a temporary Git repository, makes those branches, runs the check on each, and merges them. It requires Python 3 and Git. Its output includes:

base: reserved=80, limit=100: PASS
east: reserved=100, limit=100: PASS
west: reserved=100, limit=100: PASS
merge: CLEAN (Git exited 0; no unmerged index entries)
combined: reserved=120, limit=100: FAIL

This is a synthetic branch example, with no agents or hosted pull requests involved. It demonstrates the failure mechanism. It says nothing about how often a particular agent or development team encounters it.

The test already knows what is wrong. We have to run it on the version that contains the problem.

Separate files can share a constraint

Worktrees solve a useful problem. Two workers can have separate checked-out files and indexes instead of fighting over one working directory. Git's worktree documentation distinguishes that per-worktree state from shared repository data, including most refs and, by default, configuration.

But separate checkout state does not make two tasks independent. The east task depends on the west reservation staying at 40. That dependency exists even though the east branch never edits west.json.

The diff tells you who changed a file. It takes more work to see who depended on that file staying the same. An agent told to stay inside a directory can obey that instruction perfectly while writing code that depends on a sibling's old interface.

There is a useful database analogy here. In A Critique of ANSI SQL Isolation Levels, Berenson and colleagues describe write skew: transactions read related values, update different items, and together violate a constraint that each transaction would have preserved alone. Our capacity example has that same shape. Git isn't a database transaction manager, so the analogy supplies a way to recognize the dependency, not a claim about Git's isolation level.

Software researchers understood the larger integration problem long before coding agents. Horwitz, Prins, and Reps' 1989 paper, Integrating Non-Interfering Versions of Programs, used program dependence graphs to reason about preserving behavior when combining changes. Their method addressed a restricted programming language. They asked which behaviors interfere when the versions are combined. That question still applies to agent changes.

Adding more worktrees doesn't answer it.

What the agent evidence actually says

The September 2026 STALE-bench paper, by Haocheng Xia, Eugene Wu, and Yongjoo Park, studies parallel agent development. Its mined and constructed results deserve to stay together.

From 4,262 candidate pairs drawn from 685 Django PRs, the authors validated 417 pairs. After correcting their grading, one of 834 agent runs showed a newly failing test that passed with each patch alone and failed when they were combined.

The constructed experiment used 12 Django helpers, three scripted interface changes, and three trials per combination. Tasks required the agent to call the affected helper. With GPT-5.5, 105 of 108 runs without visibility into the peer's change produced interference. A message describing the finalized change removed measured interference in 89 of 108 message-conditioned runs.

That setup deliberately creates incompatibilities. The mined sample comes from reviewed PR history and has its own selection limits. Neither estimates an ordinary agent team's failure rate. The message supplies complete information, not a protocol proven to discover what agents need to share.

There is a concrete dependency failure here to design around. Calling it "97% of agent merges fail" would misrepresent the experiment.

Yes, this is what integration CI is for

If your CI already blocks a change until the actual combined candidate passes the required checks, you have the relevant protection for this example. You do not need to rename it agent orchestration.

It would also be wrong to say ordinary pull_request CI only tests the PR's head branch. For an open, mergeable PR, GitHub's default actions/checkout follows the synthetic merge ref. A workflow can explicitly check out the head SHA instead.

The remaining question is which target it tested. East and west can each pass against the old base. Once east lands, west's previously passing candidate no longer represents the next proposed state of main. The check was accurate when it ran. Its answer belongs to a different version.

GitHub merge queues construct candidates with the latest target branch and the PRs ahead in the queue. For GitHub Actions, required workflows need the separate merge_group trigger. A workflow that only listens for pull_request events does not automatically provide the queue's required check.

GitLab merge trains make the concurrency tradeoff especially clear. With changes A, B, and C in order, they can test the target plus A, plus A+B, and plus A+B+C in parallel. If an earlier change drops out, the downstream candidates change and their pipelines restart. You can speculate in parallel as long as you stop relying on results for combinations you no longer intend to merge.

Rust's Bors design takes a simpler route for automatic builds. It merges a PR with the latest base, tests that commit, and fast-forwards the base after success. One automatic build runs at a time. The documentation explicitly identifies independently passing PRs that fail together as the problem this prevents.

These systems already construct and test the combined candidate. A workflow that merges local agent branches needs to preserve that protection. Ten agents reporting "tests pass" still leave the integration question unanswered if they tested ten separate candidates.

Make the tested version part of the handoff

An agent's completion message should let the next worker find the evidence. "All tests pass" leaves too much reconstruction to whoever merges it.

A useful handoff names the base and head revisions, the combined candidate that was tested, and the check command or workflow run. It also identifies the relevant environment inputs, such as the dependency lockfile and service fixtures. If a local check used dirty or untracked files, a commit ID alone cannot describe what ran.

That gives the coordinator a concrete procedure. Construct the next candidate from the current target and the intended changes. Run the required integration checks on it. If the target, changes, or relevant test inputs move, reconsider that result before landing anything. An existing merge queue can do much of this bookkeeping.

The final commit ID may change under squash or rebase. In that case, establish that the landed source tree matches the tested tree and account for any checks or builds that depend on commit metadata. A matching tree is useful evidence about tracked source, not a description of every input to a build.

External state needs separate attention. Two worktrees pointed at the same test database can interfere before Git merges anything. Give candidate runs separate database namespaces or reset fixtures between them. If a required check cannot run independently, coordinate that stage. There is no reason to serialize all the editing just because one integration step shares a resource.

And a queue only enforces the checks you put in it. Without the capacity check, the combined reservation of 120 could pass every configured gate. In this example the check exists, which is why choosing the right candidate is enough to catch the failure.

I want the branch checks while the work is still cheap to change. Once the coordinator proposes combining those branches, I want a result for that combination. A correct report about yesterday's base is not enough to authorize today's merge.

Before accepting an agent's "done," ask which version passed. Before advancing main, ask whether it is the version about to land.

The world (i.e. the internet) is ending. Here’s just a few things that went down this week.

• PHP had a bug that fires before your code runs. Patch, then restart every worker, or you're still vulnerable.

• NVIDIA's driver trusts whatever the GPU says. Flip one bit and you own the host in about 17 seconds.

• A certificate authority's private keys could be worked out from public data. Browsers pulled the plug.

• GitLab shipped two perfect-10 bugs in two weeks, both in the part that compiles your regexes.

• Someone breached 100+ companies with off-the-shelf AI agents for about $5,000 total. Fifty bucks a company.

• I cut the "no" out of a model on a laptop. 90 refusals became 3. That doesn't make it smart.

• The tools you use to build AI agents are themselves getting popped.

• Attackers' agents ran 23 hours per target. My ten agents leave me feeling lobotomized by the end of the day.

• Your Wi-Fi router can tell who is walking across the room, and Comcast keeps a week of it.

• Six remote-management and backup appliances are being actively exploited. Chrome fixed 15+ critical bugs in one release.

• AI copies your placeholders exactly. That's a feature if you know how to block a scene, and a trap if you don't.

Plus one prediction at the end.

A quick decoder for the numbers

CVSS is a 0 to 10 severity score for a bug. Anything 9 or above means "an attacker needs little or nothing from you to do a lot of damage." A CVE is the catalog number for a specific bug. KEV is the US government's list of bugs that are confirmed to be used in real attacks right now, which makes it the only list worth panicking over. RCE means remote code execution: someone far away gets to run their program on your machine. That's the whole game.

PHP2Shell: the bug is underneath your code

The hit: PHP, which runs most of the web, had a bug in how it handles running out of memory. It fires before your application code starts, so the quality of your code doesn't matter. Update to the fixed version and restart every worker process.

Here is the mechanism, because it's a good one. When PHP runs out of memory mid-request, it uses a C function called longjmp() to bail out of whatever it was doing and abort the request. Some code paths were never written to expect that bailout. They leave request memory uninitialized, and global state that was pointing into that memory never gets reset. When one request leaves broken, the next request inherits that brokenness, and that can be leveraged. That's why I called it a progress bar that became a skeleton key for 70% of the web: the bug is literally underneath the code you wrote.

The two CVEs are 2026-109618 and 2026-109619, CVSS 9.2, patched October 21. Fixed branches are 8.2.35, 8.3.36, 8.4.27 and 8.5.12. The part that gets missed in the patch rush: if you still have a worker running after you've updated, it's still running the old vulnerable code. Restart every worker. Help Net Security, BleepingComputer and Bitsight have the version tables. Bitsight also found that only about a third of identifiable PHP instances run a supported branch at all. Two-thirds of the floor has no patch coming.

GPUBreach: NVIDIA's driver believes whatever the GPU tells it

The hit: Researchers used Rowhammer, a trick that flips bits in memory by hammering the cells next to them, against the GPU's own page tables. One flipped bit gives control of the card in about 17 seconds, and from the card, root on the host. Hardware memory protections like IOMMU and ECC don't stop it.

The new part is aiming Rowhammer at the GPU and then walking through the driver. The paper found dozens of sites in the NVIDIA driver where GPU-managed data structures are consumed without sanitization. In plain terms, NVIDIA built a driver that assumes the GPU is never lying, and then researchers found a way to make the GPU lie. The graphics card is now a front door for anyone who wants to run this technique in a data center full of NVIDIA cards.

My favorite detail is the error-correcting memory. ECC is supposed to catch flipped bits. In two cases it "fixed" the data with wrong values and then told the system everything's good here, boss. The follow-up paper, GPUThor, breaks ECC on purpose. Google paid a $600 bug bounty for the original finding. Six hundred bucks for telling us the front door is unlocked. Coverage at BleepingComputer, SecurityWeek, the CSA research note and IT-Connect on GPUThor.

eMudhra: the thing that vouches for websites got vouched for badly

The hit: A certificate authority, one of the companies whose signature makes the padlock appear in your browser, issued eight intermediate signing keys that could be recovered from public certificate data. Mozilla, Google, Apple and Microsoft all blocked them within four days.

A certificate authority (CA) is the root of trust for the web. When you visit a site and the browser shows a padlock, it's because a CA signed a statement saying "this key belongs to this site." If someone can recover the CA's private key, they can sign that statement for any site they want. eMudhra disclosed that eight intermediates (six for TLS, two for code signing, all from one key ceremony in April 2025) had that weakness, and they had it for about 17 months. Mozilla and Google blocked them October 17, Apple October 19, Microsoft October 20, and Google starts removing the emSign ECC Root CA-G3 root on October 27.

We covered it on the stream. BleepingComputer, SecurityWeek and LWN have the details. If you run anything that pins or bundles its own CA list, this is a reissue, not an update.

GitLab: two perfect scores in the part nobody reads

The hit: GitLab shipped two CVSS 10.0 bugs in two weeks, both integer overflows in the regex compiler, both allowing an unauthenticated stranger to run code on your GitLab server. Affected versions go back to 7.0.

A regex compiler is the piece that takes a text pattern you wrote and turns it into something fast. You write the pattern, it compiles, you never think about it again. An integer overflow is when a number gets bigger than the box it lives in and wraps around to something small, which means the code thinks a buffer is one size when it's another. CVE-2026-97229 landed October 13 and CVE-2026-110264 on October 22. Help Net Security on the first, BleepingComputer on the second. If you self-host GitLab, you are patching this week whether you planned to or not.

What those four have in common

Interpreter, driver, CA, compiler. Four components that assumed the layer under them was honest. None of them were wrong to assume it, in the sense that you can't build anything if you re-verify the floor every time you take a step. But the floor is where the bugs are now, and the fix advice was the same every time: don't just update. Restart and reissue.

The $50 breach

The hit: A Chinese-speaking actor pointed Claude Code and DeepSeek agents at more than 1,200 targets and got into over 100 companies, Deere, SharkNinja and Cummins among them. Total spend was about $5,000 in tokens, which works out to roughly $50 per intrusion. The agents ran about 23 hours per target.

That's the whole story, and it's the one I think we'll look back on. On the stream I said we're going to look back on this segment and call it the first domino to fall among many, and I'll stand on that. The tools involved are not exotic. They're the same coding agents a lot of you use for work. If it's available to governments, it's eventually available to everyone else, and this one was already available to everyone else. AI Weekly has the figures.

Here's what I think is actually happening underneath, and I'll own that this is my read. PHP is 31 years old. PHP was invented when I was five. From the time I was five to 36, nobody found PHP2Shell. It was almost certainly unearthed by AI meticulously going through the code. AI is discovering vulnerabilities that humans aren't discovering, and it's discovering them in everything. Put that next to a $50 per-company price tag and you have the attacker's cost collapsing at the same moment the supply of bugs goes up.

Abliteration: I cut the "no" out of a model

The hit: The refusal in an AI model is a direction in its weights. You can find it and remove it on a laptop. I did: 90 out of 104 refusal markers before, 3 out of 104 after, same prompts, same settings. A model that doesn't refuse is not the same as a model that's right.

The video walks through it. When the model says "I'm sorry, I can't help with that," it isn't because it doesn't know the answer. It's because the answer ranks high on no-no-ness, and that ranking lives in a direction you can subtract out. In the Short I said the level of hacks is increasing because people have access to uncensored models, and I still think that's true. But notice the campaign above ran on commercial models that were never jailbroken. The honest version is both: the agent that doesn't refuse, and the agent that was never asked to refuse anything, because "scan this network for a foothold" is a perfectly normal engineering task until it isn't.

One caveat I'll keep repeating: the fact that it doesn't refuse does not mean it is accurate. Those are two unrelated things.

The AI tooling is also the target

The hit: IBM Langflow OSS, a popular tool for building AI agents, has a CVSS 9.8 bug (CVE-2026-93674) that lets a remote attacker run commands on the box. Versions 1.0.0 through 1.12.2.

The thing you use to build agents can be used to build an agent on your server. If you run Langflow anywhere reachable from the internet, treat it like the PHP patch: update and restart.

Your agents don't sleep, but you do

Small human sidebar. I run 10 coding agents at once. Weaponized ADHD is killing us, man. I feel like a lobotomy patient at the end of the day. Meanwhile the attacker's agents ran 23 hours per target and did not feel like anything. Burnout is asymmetric. Defenders context-switch; the other side's agents just keep going. I don't have a fix for that. I'm just noting that the humans in this loop are all on one side of it.

Your router knows when you walk across the room

The hit: Wi-Fi routers measure the room to shape their signal, and that measurement is broadcast unencrypted. Researchers can re-identify a person by their walk from it. Comcast's Wi-Fi Motion feature keeps seven days of it and can hand it over under subpoena.

Here's the full breakdown. Modern Wi-Fi uses beamforming, which means the access point aims the signal at your device instead of spraying it everywhere. To aim, it needs a report back from your device about what the room looks like to radio. That report is called beamforming feedback information, BFI, and it goes back over the air in the clear. BFI is just free. Not only is it free, it's unencrypted. Your body is part of the room, so your body is in the data, and gait is distinctive enough to pick you out.

Temper the panic a little: the 99.54% accuracy figure is only on the mannequin in the clean-room environment. But the fine print is real. Comcast's policy says it may be required to disclose information generated by Wi-Fi Motion in response to a valid subpoena, court order, search warrant, or other lawful legal process. There's no great conspiracy that it was designed to spy on you. It has to measure the room, and by extension the things in the room like your body, to give a clean signal.

Two more radios, same shape

The hit: A leaked Cellebrite document describes an appliance that can unlock iPhones from about 8 meters away, Stolen Device Protection on. And GPUBreach, from earlier, can read AI model weights and post-quantum encryption keys straight out of a graphics card's memory.

Privacy Guides published the Cellebrite leak and Schneier picked it up. The product is called "Advanced Unlocks" and targets phones in the after-first-unlock state, which is the state your phone is in basically all day. The GPU piece is the same paper as above: the card's memory holds whatever you loaded, including LLM weights and cuPQC keys, and the attack reads it.

The connection across all three is that hardware which has to measure its surroundings to work (channel estimation, radios, DRAM cells) leaks those surroundings. Nobody designed it to spy, and it does anyway. Someone on the stream put it well: every old device you own that is no longer receiving updates is now about as secure as your bag in a Paris riot.

KEV quick hits: the management boxes are the front door

The hit: Six remote-management and backup appliances are on the actively-exploited list this week. If you run any of these, it's not a "schedule it" patch.

• Ahsay AhsayCBS, backup software, OS command injection in the Replication Receiver, CVSS 10.0 (CVE-2026-105134)

• Arista VeloCloud Orchestrator, SD-WAN management, any logged-in user becomes superuser, 9.9 (CVE-2026-114625)

• Cisco Secure Email Gateway, arbitrary file write, 9.8 (CVE-2026-76519)

• BeyondTrust Remote Support, improper origin validation, 9.3 (CVE-2026-102764)

• Atlassian Data Center suite (Jira, Confluence, Bitbucket and more), 9.3 (CVE-2026-21589)

• Collabora Online CODE server for Nextcloud, remote code execution exploited in the wild in September and October, 9.2 (CVE-2026-110447)

In the GPU video I said the graphics card is now a front door. This week it's the backup box, the SD-WAN orchestrator and the remote-support appliance. The things you bought to manage the fleet are how the fleet gets managed by someone else.

Chrome: just update

Chrome 156.0.8078.47 shipped more than 15 CVSS 9.6 fixes in a single release on October 20. That's it. Close this, go to chrome://settings/help, come back.

AI does exactly what you showed it, placeholders included

The hit: I made a 90s mecha anime with Blender and AI this week (sponsored by Flova). The model copies your placeholders exactly, every single one. If you can block a scene, you can direct it. If you can't, it will faithfully render your mistakes.

The lesson that stuck had nothing to do with anime. If you tell the model the blocking image is the authority, it keeps the boxes. The gray boxes I threw into Blender did more work than any prompt I wrote. That's the same lesson as the abliteration caveat. A model that doesn't refuse isn't accurate, and a model that follows your layout isn't creative. Both are literal. Neither is wise. You are still the one who has to know what the picture is supposed to look like.

One prediction

I was director of engineering at American Express, and I can tell you the state of software at those banking institutions is anything but secure. It's band-aid fixes on band-aid fixes. Given everything above, $50 intrusions, agents that don't sleep, a floor full of 31-year-old bugs that AI is now reading line by line, I expect a US bank to get hit in the next three to six months.

- Dr. J

Take two hypothetical agents with the same 50% success rate. Give each four attempts at the same task. Under one scoring rule, agent A beats agent B by 93.75% to 50%. Under another, B beats A by 50% to 6.25%.

The scoring rules are pass@4 and pass^4. Replace the @ with a caret and the ranking reverses. Both calculations are correct. They describe different behavior, and neither tells you which answer your product will actually return.

That's the part I want next to an agent benchmark score. Show me how the system chooses what to ship.

Two agents that average out the same

This is an invented probability model, not a measurement of any commercial agent. Assume the success labels are correct. We can get into trouble before arguing about whether the tests are any good.

Agent A has a 50% chance of succeeding on every fixed task. Each repeat is independent. Agent B always succeeds on one half of the task types and always fails on the other half. Both agents face the same equally weighted task set.

Run either agent once on a randomly chosen task and its success probability is 50%. Now keep the task fixed and repeat the run four times, starting each run from the same initial state.

pass@4 asks whether at least one of those four runs succeeds. For A, the only losing outcome is four failures. That happens with probability 0.5 to the fourth power, or 6.25%, leaving a 93.75% chance of finding at least one success. B stays at 50%. More attempts cannot help with its impossible half.

pass^4 asks whether all four runs succeed. A manages that 6.25% of the time. B manages it on the same half of tasks it always solves, so it stays at 50%.

AgentOne runAny of fourAll four A50%93.75%6.25% B50%50%50%

These are exact probabilities under the assumptions, not estimates from four observed runs. The Python example enumerates the possible outcomes using fractions. It makes no model calls and requires no packages.

Calling B the more reliable agent would leave out something fairly consequential. It never solves half the tasks. Calling A the better agent would also need a qualification. Its advantage comes from collecting multiple attempts and recognizing a successful one.

Suppose the product generates four answers but always returns the first. Its success rate remains 50% for both agents. Generating four candidates has bought A a larger collection of answers, including more chances to contain a correct one. The product still needs a way to choose it.

The chooser was in the original papers

This distinction predates the current enthusiasm for agents. In the 2021 HumanEval paper, Chen and colleagues explicitly distinguished an oracle that knows the unit tests from a heuristic that selects an answer without them. In the reported Codex-S comparison, generating 100 samples at temperature 0.8 found a test-passing function for 77.5% of the 164 problems. Selecting one from the same 100-sample pool by highest mean log-probability solved 44.5%. Those are results for that model and code-synthesis setup, not current agent performance.

The authors measured both quantities. The trouble starts when a reader takes the larger number into a product discussion and leaves the selection mechanism out.

AlphaCode made the constraint explicit with n@k. Generate k programs, select n submissions using information available to a competitor, then see whether any selected submission passes the hidden tests. Its system filtered candidates with public examples and clustered them by behavior before selecting submissions. A correct program hidden somewhere in a large pool was useful only if the selection procedure could get it into the much smaller submission budget.

There is a good reason to build this way. If you can generate isolated candidates, safely run a sufficient verifier, and return one that passes, sampling can improve the product. In our toy example, A plus a perfect chooser really does succeed on 93.75% of tasks with four candidates. That improvement is real within the model. It depends on having the chooser.

The verification available during evaluation can be stronger than the information available when making the choice. A hidden answer key can tell an evaluator which candidate was right after the fact. A deployed ranker has to make its choice before that answer is revealed. Measure the ranker on the same candidate pools and the gap becomes visible.

The caret doesn't mean power the average

The 2024 tau-bench paper introduced pass^k for repeated agent tasks. It asks for the probability that all k independent, identically distributed trials of the same task succeed, averaged across tasks. A trial is a complete interaction, potentially containing many tool calls. Four trials are not four steps within one conversation.

For a fixed task with success probability p, the independent-repeat calculations are simple:

Any of k succeeds: 1 - (1 - p)^k
All k succeed:     p^k

Calculate those quantities for each task, then average. Raising the benchmark's average success rate to the fourth power would assign both of our agents a 6.25% all-four score. That is wrong for B by a factor of eight. Averaging first erased the difference between being intermittently successful everywhere and being consistently successful on only half the tasks.

This is why I'd want the distribution behind the average. How many tasks never succeeded in the observed runs? Which tasks succeeded intermittently? Do failures concentrate in a particular task class? Zero observed successes does not prove a task is impossible, but it tells you where another identical attempt has not yet helped.

Real evaluations also have finite observations. With n runs and c successes for one task, the reference implementation uses combinations to estimate all-k success. The corresponding any-k estimate counts subsets containing at least one success:

pass@k estimate = 1 - C(n-c, k) / C(n, k)
pass^k estimate =     C(c, k) / C(n, k)

Here C counts combinations and n must be at least k. These are per-task estimates, averaged afterward. Under independent, identically distributed trials within each task, they estimate the probabilities above without the bias introduced by simply substituting the observed success fraction for p and taking powers. The downloadable example checks the subset calculations too.

Independence is an assumption with consequences. A shared outage can make several trials fail together. An adaptive second attempt can use information the first attempt lacked. A run against a database already changed by an earlier run is a different experiment. In those cases, report the actual joint outcomes and protocol instead of manufacturing a probability from the single-run average.

Evaluate the policy that returns the answer

Anthropic's evaluation guide makes room for both metrics because the relevant question depends on the product. That seems more useful than declaring one punctuation mark the honest one.

For a concrete evaluation, imagine a coding assistant allowed to generate up to four patches in isolated checkouts. Its actual tests select a patch, or it returns the task for human review. Run that whole policy as one trial, including the selection and the handoff. Grade the selected patch with a separate evaluation suite. If an acceptable patch existed in the pool but the selector rejected it, record that. If the selector accepted a wrong patch, record that too.

Now you can distinguish a generation failure from a selection failure. You can also compare the policy with a single-attempt baseline, including generation and verifier cost, latency, and the fraction of tasks handed back to a person. An escalation should remain visible rather than quietly becoming an autonomous success.

If you then want repeatability, rerun that complete policy from the defined initial state. Its internal retries belong inside each trial. Its pass^4 describes four executions of the product policy, not four chances for a user to recover from an already released mistake.

Even that leaves other questions. Rabanser and colleagues' ICML 2026 study evaluated 15 models on two benchmarks using twelve metrics spanning consistency, robustness, predictability, and safety. Repeatability is one part of that picture. An all-four score cannot tell you how severe a failure was or what happens when tools misbehave.

For a benchmark report, I want the attempt budget, task-level outcomes, and success rate of the actual selection policy beside the candidate-pool score. If the system cannot identify its successful attempt, some of the improvement exists only in the evaluator's knowledge. The product has to earn it by choosing what the user receives.

The people determining who gets to see videos exposing fraud are participants in the fraud, of course they are going to suppress my political content.

Effective immediately I’m making a the following sweeping changes to my content strategy.

The DrJ Channel on YouTube is about tech.

https://www.youtube.com/watch?v=yIqYHxfbRuo

I will continue to go autism-level-deep on cybersecurity, hacking, and system design implications of our modern tech stack. I enjoy doing this and it’s clear that there is a hungry audience for this type of content. H-1B fraud content on YouTube is done. I have privated most of my political commentary videos on YouTube at this point.

X is the global town square

I will, of course, keep fighting fraudulent immigration schemes that harm the American worker. All of this content (including videos) will now live on my X https://x.com/drjoshcsimmons

X has been a lovely medium where my pro-American content is not rabidly suppressed and as a result I’ve grown my account from 0→10k followers in just six months. The correct people are resonating with what I am doing on X.

The newsletter is synthesis

My long form writing will continue to focus primarily on think-pieces that bring these seemingly disparate parts together. I am trying to grow on YouTube and X. I am not trying to grow my newsletter at this time. This is a feature not a bug. You should only be reading this if you are here for the unadulterated alpha that flows from my raw ideas. These ideas are at the core of everything I produce but they’re not shaped to be “hooky” “viral” or “for mass consumption”.

I hope you’ll keep reading, but if not, maybe I’ll see you on X or on YouTube.

As a reminder, I don’t typically answer subscriber emails, if you want to engage I have set up a Discord server where I regularly chat with everyone https://discord.gg/3F6H7zdp8

- Dr.J

Ten jobs are running. Another 9,990 are waiting. The dashboard says concurrency is ten, which is correct and considerably less comforting than it sounds.

This is an easy system to build. Give each document, URL, or agent subtask its own asynchronous task. Put a semaphore around the expensive call. Everything looks civilized because the model endpoint only sees ten requests at once. Meanwhile, the process has already accepted responsibility for the other 9,990.

Those numbers come from the small, deliberately stalled program below. It makes no model calls. The mistake it demonstrates sits in the code that launches work, before intelligence has much opportunity to help.

The queue is made of tasks

Here is the shape of the problem, with process standing in for the work:

limit = asyncio.Semaphore(10)

async def run(item):
    async with limit:
        return await process(item)

await asyncio.gather(*(run(item) for item in items))

The semaphore does its job. It limits how many callers hold a permit. But asyncio.gather schedules the coroutine arguments as tasks. Creating all those wrappers happens before the permits can slow the producer down. The pending tasks become a queue, whether or not a variable named queue appears anywhere in the program.

The same pattern appears in a library. At this pinned LangChain revision, gather_with_concurrency wraps coroutines in a semaphore and passes the wrappers to asyncio.gather. Its name describes the limit it provides. That helper is not promising bounded admission, and inspecting it does not establish how every LangChain application submits work. It does show why reading a max_concurrency setting is insufficient to understand the surrounding system.

A waiting task still exists. It can retain arguments and whatever those arguments reference. If the caller constructed a large document context before scheduling it, the context waits too. Moving the model call behind a permit does not move that allocation behind the permit.

For a known, small batch, this may be entirely fine. The problem starts when a local limit gets mistaken for a limit on everything the application has accepted.

Count what has already been pulled

The complete runnable example compares two ways to process 10,000 integer jobs. One creates a task for every job and puts a ten-permit semaphore inside each task. The other uses ten workers, a queue that holds twenty items, and one producer reading a lazy source.

An event holds the workers still while the program counts the work. This makes the comparison inspectable without guessing how long to sleep or calling a model. On Python 3.14.7, it prints:

eager: active=10, waiting=9990, unfinished_tasks=10000
bounded: active=10, queued=20, held_by_producer=1, pulled=31
Both completed 10000 jobs. All count assertions passed.

The second line is the interesting one. Thirty jobs have reached a worker or the queue. The producer has pulled one more item and is waiting to place it. It has not pulled the remaining 9,969. Once the event releases the workers, both designs finish all 10,000 jobs.

The example measures counts, not memory savings, throughput, agent quality, or API cost. It exercises normal completion; a service still needs a policy for rejected, expired, and cancelled jobs. Integers make cheap jobs. Real payloads will not all cost the same. This run shows where unfinished work resides in these two implementations and whether the producer keeps pulling while the workers are stalled.

In a fixed-worker design, the producer's core loop is plain. Worker startup and shutdown are in the complete example:

queue = asyncio.Queue(maxsize=20)

for item in source:
    await queue.put(item)

Python's queue contract says put waits when a positive maxsize has been reached. Here, the producer itself waits. It cannot advance the loop to fetch the next item until that put succeeds.

Now use asyncio.create_task to schedule each queue.put(item) instead of awaiting it in the loop. You have bounded the queue's contents and created a separate population of tasks waiting to enter it. The queue can proudly report twenty while the work waits outside. The location of the await matters more than the presence of the queue.

This assumes the source is actually lazy. An upstream function that first loads every document into a list has already made a different decision. A source with its own prefetch buffer adds another place to count. Backpressure has to reach the producer that can stop acquiring work.

We had names for this

Matt Welsh, David Culler, and Eric Brewer described explicit stage queues and overload policies in SEDA in 2001. A finite queue could reject an entry, letting the application block upstream, shed load, return an error, or offer reduced service. The queue made a resource decision visible.

Reactive Streams later made demand part of the exchange between asynchronous components. The receiving side should not be forced to buffer an arbitrary amount of data merely because the sending side can produce it. Neither idea needs to be rediscovered as an agent architecture.

Java's ThreadPoolExecutor documentation is refreshingly explicit about the choice. A fixed pool can sit in front of an unbounded queue. A bounded queue with finite thread limits instead reaches a saturation point where a rejection policy applies. The worker count and the waiting-room size are separate decisions.

An agent supervisor adds a particularly enthusiastic source of work. In its June 2025 account of building a research system, Anthropic described early agents spawning fifty subagents for simple queries. The same report describes benefits from parallel research and prompt rules for allocating effort. It is an account of one system, not a claim that fifty is always wrong or that parallelism is wasteful.

A model can choose a sensible amount of work. The runtime can enforce how much it will accept. Better delegation instructions help the model make that choice; the runtime still needs an enforced limit.

A semaphore can be the right answer

The strongest objection is correct. A single producer can acquire a permit before it creates the next task. With careful permit ownership and cancellation cleanup, that can bound unfinished child tasks without an explicit worker queue. The semaphore itself is not the mistake. Putting the wait inside thousands of already-created tasks is what gives those tasks somewhere to accumulate.

There are other perfectly reasonable choices. A backfill may intentionally accept a large backlog into durable storage, then feed compact job identifiers to workers. A small bounded batch may use gather because its maximum size is already known. Adding a scheduler to six independent operations could be needless ceremony.

I would ask a more specific question before changing any of them: when the downstream service stops making progress, which part of this system stops accepting new work?

If the answer is only "the model call," keep tracing upstream. Count callers waiting for a permit, callers waiting to enqueue, and jobs materialized before either wait. Then count retained results downstream. A pipeline that streams its input and keeps every output forever has another accumulation point.

Recursive delegation needs a separate check. If every worker holds its slot while waiting for child work that requires one of those same slots, a perfectly bounded system can deadlock. The flat worker example does not solve dependency scheduling. Limit the resource-consuming operation deliberately, and ensure waiting parents cannot occupy every slot their children need.

Put the limit where the work enters

For an agent runner, I want to see active work, queued work, and callers waiting for admission separately. I also want the age of the oldest waiting job. A queue of twenty jobs can represent a brief delay or a task the user abandoned an hour ago. The count alone cannot tell you which.

An item limit also leaves other resources unbounded. Ten short lookups and ten document-sized model requests consume different amounts of memory and tokens. Google's SRE chapter on overload makes the broader point that request counts can be poor proxies for resource cost. Apply the relevant budget where that resource is acquired; don't expect one integer to cover every constraint.

For interactive work, include admission wait in the deadline. When capacity runs out, decide whether the caller waits, gets an overload response, or accepts less work. If waiting is allowed, bound the callers allowed to wait too. Otherwise "we'll get to it" becomes an allocation policy that nobody wrote down.

The practical test is simple. Stall the workers. Keep presenting work. Observe what grows, and where the producer finally stops. If ten calls are active and everything else is accumulating off the dashboard, the concurrency limit is working. You still have a queue to design.

Imagine a release agent that proposes publishing after CI fails. You save the conversation, rerun the task, and watch it pass. The next attempt looks fine. Nobody has changed the code.

The convenient explanation is that the model had a moment. It worked the second time. Ship it.

But the second run may never have encountered the condition that broke the first one. CI now passes. A retrieved document changed. The model chose another tool. You have a successful new run and an unexplained old failure.

For reproducing the code around a model, I want replay to mean something fairly strict. Run the recorded version of that code against the observations it received, including the model's responses. Stop if the program requests something the recording cannot supply. A fresh model call belongs in a separate experiment.

"Replay" already describes several operations with different guarantees. Looking at a saved conversation, resuming a checkpoint, and executing code against recorded inputs answer different questions. A button with the same name does not make them interchangeable.

A bug small enough to keep

Here is a deliberately constructed example. An agent asks a tool for CI status on main. The surrounding code decides whether to propose publication. No actual model, CI service, or deployment system is involved.

The gate contains an ordinary Python bug:

def buggy_gate(result):
    return "publish-intent" if result["status"] else "stop"

The string "failed" is truthy. So is "passed". Both produce the same publication intent.

Our authored recording contains a model response requesting get_ci(ref="main"), followed by a tool result with status="failed". Another fixture simulates the world later, when that same branch has passing CI on a different commit. The old code proposes publication in both cases. Only the first case exposes the mistake.

Download the complete Python example. It uses the standard library, makes no network calls, and has no executor that could publish anything. Save it and run python3 replay_example.py.

Its first three results are:

historical replay: publish-intent on failed CI (bug reproduced)
changed world: publish-intent on passed CI (bug hidden)
code regression test: stop on failed CI (fix handles old input)

The regression test changes only the gate to require result["status"] == "passed". It keeps the old failed observation. That is a controlled code change against an old input. The historical replay keeps the original buggy gate too.

In this example, the program receives an authored model response. Freezing its response lets us investigate what the rest of the program did with that response. We have learned nothing about how often a real model would request this tool or whether a revised prompt would improve its judgment.

The output should tell you which of those questions the test can answer.

The request has to match too

Returning a stored response is the easy part. Returning it to the right request is where a green test can stop meaning anything.

Suppose you change the prompt to inspect a release branch. Your test intercepts a call to the same model endpoint and supplies the old response, which still asks for main. The test finishes. You conclude that the new prompt works. The new prompt never reached a model.

The example therefore checks the next request before supplying each recorded response:

expected_kind, expected_request, response = self.events[self.cursor]
if (kind, request) != (expected_kind, expected_request):
    raise ReplayDivergence(f"request changed at event {self.cursor}")

It checks the model request as well as the tool name and arguments. It consumes events in order and refuses an incomplete or exhausted recording. Changing the prompt stops at event zero; changing the tool arguments stops at event one. There is no fallback that quietly calls a live service when a fixture is missing.

HTTP testing tools already make this choice explicit. VCR.py's configuration makes request matching configurable. Its documented defaults include the method and URL components but omit the request body. For a model API, the body can contain the entire experiment. A match on the endpoint alone can therefore accept a different prompt. Its none record mode refuses new requests within the intercepted traffic. Local files and other uncaptured I/O still need separate controls.

A real recording needs a deliberate matching policy. You may want to ignore a transport request ID. You probably do not want to ignore a changed system prompt or tool argument. Every field you discard becomes a claim that it cannot affect the behavior you are testing.

An unmatched request is useful evidence. It tells you where the old recording stopped describing the new execution. Treating that disagreement as permission to improvise a response destroys the experiment.

Other people already solved parts of this

This is an old debugging problem with a new external dependency. In his 2005 discussion of event sourcing, Martin Fowler identifies both sides of the problem. Historical queries need historical answers. Replaying events must also avoid sending the same external updates again. His proposed gateway can remember query responses and suppress outgoing effects during replay.

The 2016 rr paper describes a more demanding implementation at the boundary between user-space programs and the kernel. It records inputs and sources of nondeterminism, then applies their recorded effects during execution. It does not need the filesystem to spontaneously recreate yesterday. Its fidelity depends on its recording boundary and supported execution conditions, including its approach to scheduling threads.

An agent fixture is a much smaller promise. It can hold the observations crossing model and tool interfaces steady while ordinary application code runs. It does not inherit rr's instruction-level guarantees because someone put "time travel" in the product name.

Current orchestration systems make the distinctions concrete. Temporal requires deterministic Workflow code, with nondeterministic work such as model calls and database queries in Activities. During replay, Workflow commands are checked against history. A successful history replay says something about that workflow's compatibility with recorded events; it does not demonstrate that a model would generate the same response today.

LangGraph's time-travel documentation describes a different operation. Results before the chosen checkpoint are saved, while nodes after it execute again. That includes model calls and API requests, which may return different results. This is useful for trying another continuation. It is not the frozen-input experiment above unless you add controls around those calls.

Read what the system executes again before deciding what a passing run proves.

But the model was the bug

This is the strongest objection, and it is correct. If the failure was a bad model choice, replaying that choice cannot show that a new model would do better. You can reproduce how the program accepted it, where validation failed, or which action followed. You cannot evaluate new reasoning by substituting old reasoning.

For a prompt or model change, preserve the old prefix, mark the point where the experiment changes, and call the candidate model there. Keep the relevant tool world controlled if you want to isolate that change. Run enough trials for the reliability claim you intend to make. A single successful continuation tells you that one continuation succeeded.

For a code change, the frozen recording can become a regression case, as it does in the example. For a changed database or service, choose explicitly whether to restore old observations or test current behavior. Save those results separately. Otherwise a code fix, a different model answer, and a repaired external dependency can all arrive under the same "passed" label.

There is no need to build a general replay platform to fix a one-line condition. A captured response and a unit test may be plenty. Use a recording when the sequence and surrounding state matter enough that a single fixture loses the explanation.

An HTTP recording misses a local file read. A serialized list of tool results may miss the order in which concurrent tasks changed shared state. A redacted value can change a branch condition. Keep the original executable and dependency versions if the question is what the original program did, and document whatever the recorder leaves live. Missing inputs are a limit on the claim, not an invitation to declare the run deterministic anyway.

Keep replay isolated from production because the intended action may be exactly the bug you are reproducing. Record the proposed write, or route it through an inert replacement. Debugging a duplicate email by sending another email would be an unusually literal interpretation of reproduction.

I want a failed agent run to arrive with a small executable case: the code version, the requests it made, the observations it received, and a way to run it without touching the real account. Then I can change one thing and see what follows.

Before celebrating a successful rerun, make the old failure happen on purpose.

I was excited for about an hour. JD Vance stood up and said Microsoft laid off 6,000 American workers last year while benefiting from 6,300 H-1B visas. I've been on the Microsoft beat since January. Somebody over there watches the show.

Then the logic caught up with me.

This is not an H-1B ban. Nothing happens to existing H-1Bs. Nothing happens to anyone who already has a green card. These companies can keep filing H-1B petitions tomorrow. Nobody has said how long the pause lasts.

What it does prove is that they have the power. If the Labor Department can freeze green cards for eight companies with a press conference, this administration can shut down the H-1B program. Every morning it stays open is a choice somebody makes.

I've seen this play from the inside as an engineering leader. When you need a quick win, you announce a pile of initiatives, take one small step, and let the media cycle do the rest. It looks like a fight. It costs nobody much. Expect a lot more of it before November 3.

So here's my rule for the midterms, and I'd urge you to use it too: judge a politician by what they've done, not what they say. Skip the press releases. Go straight to the record.

That's what this guide does.

Download the free guide (PDF)

The Single-Issue Voter's Guide to the Midterms: Say vs. Do on H-1B scores every candidate in 30 House and Senate races twice, on the same scale. What they did comes from 1,035 immigration bills filed since 2023. What they say comes from their own speeches, press releases, interviews and posts.

A few things it turned up:

• Some of the strongest H-1B votes belong to people who never talk about them. Sheri Biggs signed the bill to pause H-1B, the PAUSE Act, and the bill to end OPT. Asked for her top issue this fall, she talked about the border.

• Some talk tougher than they vote. Beth Van Duyne says H-1B should be stopped or redone so it doesn't exist. Her bill raises H-1B fraud fines. She filed it 33 days before the election.

• One record argues with itself. Andy Harris cosponsored the Mass Immigration Reduction Act, then thanked the president for extra guest-worker visas. His gap between talk and record is the widest in the guide.

• 19 people on the November ballot signed the bill to end OPT, the student work program that feeds H-1B. Most voters will never hear about it.

There's a page for every state, with every race, every bill and every statement linked. Find your district. Look at what your candidates actually did.

If the incumbent could fix it today and hasn't, that tells you something. If the challenger has never held office, you're taking a leap of faith either way, and the guide shows you what they've put on the record.

Get the guide (PDF, 33 pages)

I made a video about the PERM news today: JD Vance Suspends Microsoft, Infosys and Tata From PERM. It's Not an H-1B Ban. If you know someone who votes on this issue, forward them this email.

Josh

Consider a coding agent asked to summarize one issue in a private repository. It finishes in seven minutes. The interface says "Done." Its access token still has 53 minutes left.

Nothing has gone wrong yet. But if an old worker wakes up, or a queued tool call finally runs, what makes the service reject it?

"The agent finished" is a statement about work. "This credential is valid" is a statement about authority. Unless the system connects those states, finishing the work changes nothing about what the credential can do.

Authority granted for a task should end with that task. Completion should close the grant. Cancellation should close it too. A new task should obtain its own authority, even when the same human stays logged in.

The seven-minute example is hypothetical. The distinction is already present in ordinary infrastructure. GitHub App installation tokens expire after an hour, and callers can narrow their repositories and permissions at issuance. Those are useful controls. An hour on a clock does not tell GitHub that your particular task ended seven minutes after issuance.

A timeout can become a request for more power

You can tell an agent to use only the access it needs. Whether it follows that instruction is another matter.

ToolPrivBench, a July 2026 preprint by Kaiyue Yang and colleagues, examines that assumption. Its 544 simulated cases each offer six tools, three with lower privileges and three with higher privileges. Every tool is constructed to be sufficient for the task. The evaluation also injects transient failures unrelated to permissions into the narrower tools.

Six of the eleven evaluated models used higher-privilege tools in more than 30% of cases while sufficient lower-privilege alternatives remained. Results varied substantially by model. The authors also observed escalation after tool failures, despite those failures not establishing a need for more access.

These are simulated tool choices over at most five turns. They do not measure production incidents, and the study does not test revocation after task completion. It gives us a narrower reason to enforce permissions outside the planner. A failure can influence which authority the planner tries to use.

A network error should not grant access to another repository. A retry should not renew a cancelled task. Those decisions belong in the system that authorizes the call.

The security theory is older than the agent

Saltzer and Schroeder's 1975 protection principles include least privilege and complete mediation. Limit a program to the authority needed for its job, and check authority on each access. For an agent, those checks must follow a job that can branch, pause, delegate, and end.

There is also a long history of narrowing delegated authority. Macaroons, published in 2014 by Arnar Birgisson and colleagues, let a holder add restrictions to an authorization credential before passing it on. Delegation can reduce what the next holder may do.

None of this requires inventing a new agent token standard. It requires carrying the grant through the parts of the system that can act.

Suppose our issue-summary task receives the following grant. This is an illustrative server-side record, not a token format:

{
  "task_id": "summarize-417",
  "actor": "worker-12",
  "audience": "issue-broker",
  "resource": "repo/widgets/issues/417",
  "actions": ["read"],
  "expires_at": "2026-10-03T17:00:00Z",
  "generation": 8
}

The trusted runtime creates the record from an approved policy. The worker gets a handle to it. On every request, the broker authenticates the caller, resolves the handle, checks the actual target and operation, and consults the task's current state. It supplies the upstream credential only after those checks pass.

The worker must not get to assert its own identity or declare its task active. A JSON object supplied by the model is a request for authority, not evidence that authority exists.

When the task closes, the broker invalidates that generation. Reopening the conversation does not reactivate the old handle. An explicitly resumed task gets a fresh grant under the current policy. If task state cannot be checked, the broker refuses the call.

Policy engines already express much of this decision. Cedar evaluates a principal, action, resource, and context. The application still has to provide current task state and invoke the check on every protected path. A correct policy evaluated against yesterday's inputs can authorize today's wrong action.

MiniScope, a December 2025 research framework by Jinhao Zhu and colleagues, implements a related arrangement for tool-calling agents. Its gateway keeps the user's credentials and gives the agent a session token. New permissions go into a session-specific fork unless the user chooses to make them permanent. Ending the session discards that fork.

That gives temporary permissions a defined end. Previously permanent grants remain a separate policy decision. The evaluation uses synthetic requests and simulated user choices, so it does not establish how people will manage these boundaries in daily work.

Expiry does not mean cancellation

A short lifetime bounds how long a credential can remain usable. Revocation is what you need when the task ends before that deadline.

OAuth's revocation specification describes the tradeoff. A resource server can validate a self-contained token without contacting its issuer. Immediate revocation then needs additional communication or state. Short-lived tokens limit the remaining window, but they do not make it disappear.

Even a live lookup can become stale if someone caches it. The token introspection specification explicitly discusses the security cost of caching an active-token response after revocation.

Delegation adds another obligation. OAuth token exchange does not generally propagate revocation from an input token to a token issued in exchange. If your agent gives a child worker separate authority, closing the parent must also invalidate the child's path or leave a documented period in which it still works.

For the issue-summary design, keeping the provider credential behind the broker makes immediate rejection of new broker calls possible. That claim depends on the broker being the only usable route. Hand the worker a copied provider token and the broker's task flag no longer controls every call.

NVIDIA's OpenShell team reported a revealing demo in September. A REST policy blocked a repository write. The agent then used a broadly scoped GitHub credential through git-remote-https, taking another permitted path that could write to the repository. The team used this result to explain its work on proving containment across combinations of policies.

That was a reported demo, not a claim about every current OpenShell deployment. It illustrates why testing one approved tool wrapper cannot establish that the credential is contained.

Current OpenShell architecture keeps provider credentials outside the sandbox and binds supervisor tokens to a sandbox generation. Deleting the sandbox cuts off its supervisor. A product using those controls still needs to define whether one sandbox generation represents one task or a month of unrelated work.

Useful work changes its mind

Narrow task grants can also get in the way. You cannot always know the necessary resources before starting. An issue mentions a pull request, the pull request points to a build failure, and diagnosis requires a log elsewhere.

AgentDyn tests this difficulty with 60 open-ended tasks and 560 injection cases. Its authors found that several defenses lost substantial task utility when legitimate steps emerged during execution. The benchmark is manually designed and limited. In that evaluation, filtering tools according to the initial task scope blocked legitimate steps discovered later.

A task grant therefore needs a controlled expansion path. A policy can preapprove reading linked pull requests in the same repository. Crossing into a different private repository can require a new decision. The agent proposes that change; it cannot approve the change merely by describing it as necessary.

Long-running work can renew a grant while the task remains authorized. A recurring job can obtain a new grant for each run under a standing policy. Neither case requires asking the human about every read. Renewal does require that something other than the worker's determination to finish governs continued access.

Scope also has limits. Permission to comment on an issue can still produce a harmful comment. A task grant cannot determine whether the text contains a secret or whether the conclusion is right. Content checks, destination restrictions, or approval of a consequential action may still be necessary.

Test the end of the task

For a system with immediate broker revocation, the acceptance test should include these cases:

Attempt Expected result Read the approved issue during the active task Allow Use the same grant to write a comment Deny Read a different private issue outside the grant Deny Replay a queued read after task closure Deny Restart the worker with the old handle Deny Use a child grant after its parent task closes Deny Call while authoritative task state is unavailable Deny

Run the test through the actual integration, including alternate clients and network paths. A unit test of the grant predicate cannot tell you whether a shell command can bypass the broker.

There is a boundary you must state honestly. Closing a grant cannot undo an email already accepted by a mail service. A request can also pass a check just before cancellation and reach the provider afterward. Serialize cancellation with admission of new broker calls, and define how the system drains or reconciles calls already admitted. Stronger guarantees require cooperation at the service performing the effect.

Testing the end of a task makes that boundary visible. End a task, keep the worker alive, and try its old authority again. If the request succeeds, identify what authorized it and how long that access remains.

The task should stop obtaining new authority before the interface tells the human it is done.

That was the warmup. This week I covered a data center stuck in the New Mexico desert, a Dev Day with no new model, and a cheap new Anthropic model that burned through my tokens, and I went in treating them as three separate stories. By the end of Dev Day I'd changed my mind. The AI buildout ran into the physical world, and the bill is rolling downhill onto the people already paying.

Oracle can't pour concrete on a software schedule

Oracle's Project Jupiter is an AI data center campus in Doña Ana County near Santa Teresa, New Mexico. Oracle still calls it "on schedule." It also just sent a force majeure notice. That's the contract clause you invoke when an earthquake, a war, or a natural disaster stops you from performing. None of those happened. What happened is the power isn't there.

The 2.45 gigawatt fuel cell plant that's supposed to run the place is still waiting on air quality permits from the New Mexico Environment Department. The state denied the natural gas pipeline a crossing through a short section of state trust land, and the gas in-service date slid to February 2027, for a campus that's supposed to reach full service in 2028. The $18 billion loan about 20 banks put behind this thing is trading below 90 cents on the dollar in the secondary market. Oracle's plan was always to spend money it doesn't have and make it back when AI takes off. They fucked around with the funding. Now they're finding out.

I learned this lesson the hard way. A few years ago my software team was working on something that flies, out at a testing ground in the middle of the desert, living out of a couple of double-wides. The software was done. The hardware wouldn't even turn on. Twelve hours of debugging later, we found a wire about the size of a pen tip pinched in the housing, shorting out the whole thing. Software people think you can pay people more and tell them to work double hours. Hardware doesn't work like that, and neither does a state air permit.

I like AI and I use it a lot. I still don't want to live next to one of these things, and neither do the people near the site who already buy bottled water because they don't trust what comes out of the tap. That opposition is part of the schedule now, whether Oracle budgets for it or not.

Scarce compute means your plan shrinks

When the data centers slip, the labs fight over the compute that exists, and you feel it. ChatGPT is sitting at 99.65% uptime, and that's after the old trick of splitting the service into a bunch of components on the status page. The normal target for a serious web service is about 30 minutes of downtime a year. Codex was down for almost an hour last week. If Uber went down for an hour, it'd be front-page news. With AI we've just come to expect shitty service.

The speed of GPT-6 Astra fell off a cliff for me too. Pro is turning into the new Plus, and Plus is turning into the new free. The rumored $500 plan looks like the old $200 plan at a higher price. They're boiling the frog slowly so you don't notice.

Then Dev Day. I wanted them to pull it off. I'm not rooting for Anthropic or OpenAI, and competition is what keeps both of them shipping new models. So I sat through the whole stream waiting for a model that answers Opus 5.5, and it never came. We got Dots, an always-on agent product that is basically Grokbot, which a lot of us have been running with Hermes Agent and OpenClaw for six to eight months. We got GPT-6.1 Sol, pitched as "very near Astra level intelligence at a fifth of the price." And we got an ultra fast Astra, eight times the speed at six times the price. I'm already chewing through my $200 plan on regular Astra. Who is going to pay six times the price for this at scale?

The one feature I liked, sign in with ChatGPT, has a catch. Your users bring their ChatGPT tokens to your app, which is cool, until OpenAI watches your API usage light up, learns the market exists, builds a counter product, and puts you out of business two months later. That's how they steal your alpha.

Cheap per token is not cheap

Anthropic dropped Sonnet 5.5 the day before Dev Day, so I ran it on real work: pulling a 10 GB H-1B data set off a Tor-hosted repo, building a website of the ten most interesting findings, and cutting an infographic video from it. It got all three done eventually, which is why the early reviews say Sonnet does everything Opus does for less money. The problem is how it got there.

On the website, Sonnet reviewed its own 29 charts and fixed 21 of them for substantive errors: mislabeled occupations, partial-year data, a data gap. Opus doesn't get things that wrong in the first place. The video took 22 minutes, and most of that was Sonnet finding its own mistakes, fixing them, and re-rendering, including a 14 millisecond sync issue nobody can perceive. It spun up subagents on fool's errands and burned tokens the whole time.

So yes, it's faster per action and cheaper per token. It is not more efficient. When compute gets scarce, every vendor has a cheaper option for you, and you pay for it in retries. You get what you pay for.

Who pays

At the end of Dev Day, OpenAI ran a feel-good reel of people making art and having fun. Here's what I said watching it live:

"Everybody using this technology is being expected to work 10 times as hard. You're being expected to work 10 to 20 hours extra a week because they're firing people at your company and they're saying you could just make up the difference with ChatGPT."

Follow the bill downhill. Oracle borrowed against a future it can't build fast enough, the labs are rationing the compute that exists, and your plan shrinks in the same quarter your boss cuts headcount and tells you ChatGPT will cover the difference. Everyone upstream gets to call it a delay. You absorb it in extra hours.

The move

Cancel Codex. If you're on an Anthropic plan, use Opus 5.5 for the real work, and keep Sonnet 5.5 on narrow jobs like drafting an email reply. Price your AI by what it costs to get the job right, not by what it costs per token.

Watch

• Oracle Can't Build the Data Centers. That's Why ChatGPT Keeps Breaking.

• OpenAI Dev Day Was a Funeral. They Have Nothing.

• I Tested Sonnet 5.5 on Real Work. It Drains Your Wallet.

Book an hour with me

If your company is cutting headcount and handing you the extra work, and you're thinking about getting out, I'm now doing one-on-one career coaching. I've interviewed hundreds of people over the years. In an hour we get your resume in shape, put your job search in order, and automate the tedious parts. It's $250, you get written notes the same day, and if it wasn't useful you get your money back. Book a session.

If this was useful, forward it to someone who's paying for two AI plans right now.

- Dr. J

An agent writes a patch, writes the tests for the patch, runs them, and reports success. The diff looks responsible. There's an implementation, a test file, and a green check.

The implementation and the tests may agree because they contain the same mistake. If the agent misunderstood a requirement while writing the code, reading that code to produce the expected answers can preserve the misunderstanding. A larger test suite can make the mistake harder to remove. Fix the implementation later and the tests complain.

I want a more useful completion report from a coding agent. Show the behavior the test protects. Show a plausible broken version that the test rejects. Then show where the expected answer came from. That last part matters as much as making the test fail.

Two branches, one missed bug

Take a deliberately small example. Shipping costs 500 cents for orders below 5,000 cents and nothing for orders at or above 5,000 cents. Order totals are nonnegative integers.

def shipping_fee(total_cents):
    if total_cents >= 5000:
        return 0
    return 500

These tests execute both branches:

assert shipping_fee(4999) == 500
assert shipping_fee(5001) == 0

Now change >= to >. Both tests still pass. A customer with an order totaling exactly 5,000 cents gets charged for shipping, in violation of the policy. The tests reached the code. They didn't distinguish the two rules.

Adding this assertion catches the difference:

assert shipping_fee(5000) == 0

The test passes on the correct implementation and fails on the altered one. We can point to the requirement and explain why the failure matters.

The runnable example also tries two other faults, always charging for shipping and never charging for shipping. The original examples reject two of the three chosen faults. Adding the boundary case rejects all three. An equivalent rewrite still passes.

This is a constructed example, not an account of a model session. Three caught faults are three caught faults. They tell us nothing about a checkout bug outside those cases.

We can now name a specific mistake these tests would stop. That gives a reviewer something to inspect beyond a successful test run.

Reading the code can contaminate the answer

There is direct research on this problem. In a July 2026 preprint, Michael Konstantinou, Florian Tambon, and Mike Papadakis compared test-generation workflows across five models and three Python programming benchmarks.

They generated implementations, kept executable but incorrect ones, filtered out easy faults, and selected a difficult faulty implementation per task. Tests generated in a conversation containing the faulty implementation detected about 14% of faults. Tests generated in a fresh context from the task description detected about 25%. Detection required rejecting the faulty implementation while accepting the reference solution.

Those percentages describe selected faulty benchmark programs. They are not estimates of how often a production coding agent breaks your application. The result does support a practical concern, though. Showing a test generator the implementation can make its expected answers follow the implementation's mistake.

Huang and colleagues found that correct implementation context helped test generation compared with incorrect code. Reading code can supply useful information. The problem is treating its behavior as the answer key before establishing that the behavior is right.

A fresh context helps separate the work. It doesn't make an ambiguous requirement precise, and a second model can misunderstand the same requirement. The useful question is where the expected behavior came from, not how many agents agreed with it.

Testing the tests is old work

Changing the comparison operator is a basic example of mutation testing. Deliberately alter the program, run the tests, and see whether they detect the change. A detected mutant is conventionally called killed. A surviving mutant is a reason to investigate the tests, the change, or both.

The idea predates coding agents by decades. Richard DeMillo, Richard Lipton, and Frederick Sayward described the approach in their 1978 paper on test-data selection. Coding agents give us another reason to use it.

Meta already combines it with language models. Meta's ACH system starts with an engineering concern, generates simulated faults that existing tests miss, and asks a model for tests that pass on the original code and fail on the altered code. It filters candidate changes and sends the resulting tests through review.

In Meta's Android Kotlin evaluation, 277 of 571 mutant-killing tests, about 49%, added no line coverage. They found distinctions inside code the tests already reached. A filter demanding new line coverage would have discarded those tests.

ACH hardens existing behavior against selected simulated regressions. It doesn't establish that the existing behavior satisfies every requirement. Still, this is a concrete use for generated tests. A reviewer can inspect the fault each proposed test catches.

Attach that example to the agent's work. I can review the rule, the fault, and the assertion together. A list of new test names tells me much less.

Sometimes preserving current behavior is exactly the assignment. A regression suite can protect a working application during a refactor, even when nobody has a complete written specification. Meta's TestGen-LLM explicitly filters for tests that build, pass repeatedly, and add coverage. That is useful hardening. The mistake is treating preserved behavior as evidence that a new business rule was implemented correctly.

The answer key can be wrong

Return to the shipping example and change the boundary assertion:

assert shipping_fee(5000) == 500

Now the correct implementation fails and the broken implementation passes. The test is specific. It executes the relevant branch. It also demands the wrong answer.

An agent trying to get a green run could change the code to satisfy it. An agent starting with the broken code could generate it by copying what the code already does. In either case, the green check would endorse the shipping bug. When a requirement-backed test fails, investigate it before changing the expected answer. Otherwise the agent can complete the task by deleting the evidence against its patch.

This is the test oracle problem. An oracle decides whether an observed result is correct. Staats, Whalen, and Heimdahl's work on testing foundations treats the program, test set, and oracle together. Test quality depends on their relationship. Inputs that reach interesting code are insufficient when the decision about correctness is wrong.

Mutation testing doesn't supply a missing business rule. You can preserve the wrong boundary and generate plenty of other alterations that your tests reject. The score rises while the customer still gets charged incorrectly.

For this example, the source of the expected answer is explicit. The policy includes the threshold. In an application, the source might be an agreed requirement, a protocol rule, a previously confirmed bug, or an independently checked calculation. Each assertion should have a reason beyond the output the current code happened to produce.

If nobody can say whether shipping should be free at exactly 5,000 cents, settle that question. Generating an assertion doesn't settle it for you.

Failures need interpretation too. Meta's 2026 work on just-in-time catching tests separates a test that detects a change from one that detects an unintended change. Engineers confirmed eight true positives among 41 candidate catches reported to them. A failing generated test can be useful, wrong, or objecting to the change you actually requested.

Make the extra work earn its cost

Running a test suite against every conceivable alteration is expensive. Reviewing useless failures is expensive too. An agent can produce either kind of bill without much encouragement.

Google's 2018 mutation-testing report describes a more selective approach. It focuses on changed code, suppresses unproductive mutation targets, and limits how many mutants it generates. Findings appear in code review, where someone can act on them. The authors treat developer attention as a cost alongside execution time.

For an agent making a bounded change, start with the behavior the change could damage. A shipping change suggests boundary faults. An authorization change suggests an incorrectly permitted operation. Choose the failure before congratulating the agent on the number of tests it wrote.

Then ask it to produce a small, inspectable result:

• The requirement supporting the expected behavior.

• A test that passes on the proposed implementation.

• A plausible behavioral fault that makes that test fail at the relevant assertion.

• The actual execution results, including faults that survived and checks it could not complete.

Keep the deliberately broken variants in disposable copies. A syntax error or missing import doesn't demonstrate that an assertion catches the business-rule violation. Run the same test against the original and the altered version, with the expected answer held fixed.

Also check that the test tolerates a valid rewrite. Equivalent mutants change the code without changing the relevant behavior. Demanding failure on those changes encourages tests that police implementation details. Don't turn a mutation-score target into an excuse to make refactoring miserable.

These checks still cover only the cases and faults you chose. They don't replace integration tests, checks against known real bugs, or review of a requirement nobody has managed to express correctly. Spend that effort on the behavior whose failure would matter.

The agent can help construct all of this. It can suggest a fault, write a test, execute both versions, and show the difference to a reviewer. What it cannot do is turn agreement with its own implementation into an independent reason to believe the implementation.

When an agent says the tests pass, I want it to show the bug those tests would have stopped.

I charted where the feeling goes in 195 films from the IMDb Top 250. The long version has all the charts and the method. This is the short version for anyone who makes videos.

Movies are not YouTube videos. Nobody clicks away from The Godfather at eight seconds. But a video and a film are doing the same job, which is holding attention and paying it off with a feeling, and the films that did that job best for a hundred years left a pattern in their dialogue. Here is the pattern, and a way to use it.

Seven rules

1. Put your best joke in the first quarter

Across these films, humor peaks about 25% of the way in and drops for the rest of the runtime. Almost none of them save a joke for the end. If you have one great bit, it goes early. The end is for something else.

2. Tension only goes up

The average tension curve rises for 88% of the film and then drops hard. That drop is the ending. There is no dip in the middle where the film relaxes and comes back. Every time you resolve something before the end, you are spending tension you cannot get back. Raise the stakes, do not answer the question, keep going.

3. The last 10% is where you make them feel something

Warmth, grief and meaning all rise through the film and peak at the end. Warmth hits its peak in the last tenth of the film for a third of them. The middle is where films differ from each other. The ending is where they agree. Whatever the emotional point of your video is, it lives in the last minute, not the first.

4. Ending on a downer is fine

Fewer than half of these films end happier than they start. 93 of 195 end up. 102 end down. A quiet or sad ending is not a mistake. It is the majority.

5. Open darker than feels comfortable

Films that open with heavier dialogue rank higher on the list. The effect is small, but it is the only valence number that moves with rank at all. Start with the problem, and make it a real one.

6. Stakes beat jokes if you want to be remembered

The two things that go with a higher rank are tension and moral weight, meaning something is actually at stake and somebody could be wronged. Humor goes slightly the other way. Funny videos get watched. Videos with stakes get ranked.

7. Do not copy a template

I went looking for the handful of story shapes that every screenwriting book promises. They are not in the data. The films sit on a continuum. The rules above are tendencies that most great films share, not a shape you trace. Use them as defaults and break them on purpose.

A ten-minute video, minute by minute

Here is what the average curve looks like when you translate it into a plan. Adjust the numbers to your length. The proportions are what matter.

• 0:00 to 0:30. The problem. State it darker than you want to. No joke yet.

• 0:30 to 2:30. Your best bit. The funniest or most surprising thing you have goes here, while attention is highest.

• 2:30 to 8:30. Raise the stakes. Every 60 to 90 seconds, something gets worse, bigger, or more personal. Do not answer the main question. Do not relax.

• 8:30 to 9:00. The peak. The most intense moment. This is where tension maxes out in the films.

• 9:00 to 10:00. Drop it. Resolve the question, then land the feeling. Warm, sad, or moving, your choice, but this is the only minute that gets it.

That is the whole thing. Joke early, tension up, feeling at the end.

One caveat

Retention curves belong to YouTube, not to cinema, and a film has a captive audience that a video does not. Everything above is about where the feeling goes once someone is watching. Getting them to watch is a different problem. Test it on your own numbers and tell me what breaks.

• Dr. J

I drove out to East Pecos Road in Mesa to film Hadrian's Factory 3, the building that opened in January with the governor, a congressman, and the mayor standing out front while the company's release promised more than 350 new jobs in early 2026. What you can see from the street is a warehouse with loading docks, and the street is all I got, because when I asked to film inside, the answer was no.

Less than two months after the ribbon, CEO Chris Power explained the business to Defense One while talking about a Navy manufacturing partnership. The Navy's number one problem, he said, is that it cannot find enough skilled workers, "so we have to automate our way out of this problem." The machines let Hadrian keep capacity "idle in capital equipment and robotics, and not have to, say, hire or fire people."

So in January the pitch to Arizona was jobs, and by March the pitch to the Navy was that the machines mean Hadrian never has to make a hiring decision it doesn't want to make. Both pitches are true at the same time, which is the part that took me a while to sit with. There is a mechanism that makes them compatible, and I have been calling it the Deskilling Subsidy. The government pays for a system that pulls the craft out of skilled work and stores it in software the company owns, and the launch event keeps selling the worker as the reason the whole thing exists.

The salesman

Before Hadrian, Chris Power worked in e-commerce partnerships and workforce software. TechCrunch's profile has him moving to the United States in 2019 with about $6,000, cold-calling Texas hotels, and founding Hadrian in 2020. Nowhere in that path, at least the version the profiles document, is there a machining job.

He opened his written testimony to the House with "As you can hear from my accent, I was born in Australia," then argued that automated factories are the only way to win, and that a New American Workforce could be recruited out of restaurant drive-thrus and warehouses.

I know some of you want the passport to be the story, and I'm asking you not to go there, partly because it's ugly and partly because it's the less interesting fact. The interesting fact is that a growth operator with no documented time on a shop floor is now explaining to Congress what American industrial labor should turn into. The people who do have that time noticed. A Practical Machinist thread from 2024 called the thirty-day training pitch smoke and mirrors and said it showed little understanding of the trade. A forum thread is not a study and I'm not treating it like one, but it is what working machinists said out loud the first time they saw the pitch.

The superpower pitch

The model, as Power describes it, is clever. I'll give him that. The bottleneck in a traditional machine shop, he told TechCrunch, is Bob or Jeff, the veteran who is literally the only person who knows how to make a particular part. Hadrian's answer is to write that process into software, automate around 80 to 90 percent of the operation, and have one person run four to six machines. He compares it to Chick-fil-A and Amazon logistics. Standardize the process, copy the site, and stop depending on the guy.

The TBPN Digest transcript of a March 2025 interview has him saying "100% of our workforce of people have never set foot inside a factory before," and that those workers are ten times more productive because software and robotics give them superpowers. Hadrian's website advertises a factory workforce trained in thirty days or less. To Congress the number was forty days, from no machining experience to precision parts bound for space, and he told the committee, "Everyone has equity. Everyone has purpose." When the $260 million round was covered in 2025, the framing was that Hadrian supercharges workers rather than replacing them, and on Fox Business it became an Iron Man suit of productivity.

I want to take the Iron Man suit seriously, because I think output really does go up when you do this and I'm not going to pretend otherwise. But look at where Bob's thirty years went. The reason this alloy chatters at that feed rate used to live in Bob's hands. Now it lives in the company's software, and the person standing at the machine was working a drive-thru a few weeks ago. The knowledge moved, and control and bargaining power moved with it, out of the person doing the work and into a system the company owns. The reason Hadrian keeps bragging about how little experience its workforce needs is that the low requirement is the product.

The factory franchise

The Mesa building is not something Hadrian raised from dirt. AZBEX reported that Hadrian leased an existing Class A building at The Cubes at Mesa Gateway, finished in late 2023, and did tenant improvements inside it. The square footage drifts across the public descriptions, 265,500 in one place, roughly 270,000 in another, 290,000 in the ribbon release, and I don't think the drift means much. The building type means a lot. If you can take a logistics cube, fill it with standardized machines and software, and get aerospace parts out the other end, then production is something you can copy, and copying is the whole business.

The machines, Power said in the same TBPN interview, come from Germany, South Korea, and Japan. America invented CNC and no longer builds the tools he runs. That doesn't make the parts un-American and I'm not going to pretend it does. It does tell you what "reindustrialization" means in this version. The imported spindle is the durable asset, and the American at the console is the thirty-day variable.

And then the model leaves the warehouse. In December, Lockheed Martin announced an agreement to put a Hadrian production cell inside one of its Missiles and Fire Control sites, with PAC-3 MSE, THAAD, PrSM, and GMLRS among the named programs. Hadrian calls this factories as a service, which in plain English means the standardized production system gets installed inside the defense prime itself.

I need to be careful here. There is no evidence Hadrian emptied out a Mesa machine shop or caused a documented layoff anywhere, and I looked. The displacement argument is about the supplier model, not a specific victim. The Bob-and-Jeff shops that used to hold that knowledge are exactly what a production cell inside Lockheed is built to make unnecessary, and once the cell is in place, the trade they practiced is a setting in somebody else's software.

Follow the public money

This is the part that gets misquoted, so slow down with me. Defense One described a Navy public-private manufacturing effort with about $900 million in government money and more than $1.5 billion in private investment, over $2.4 billion across three maritime facilities. Nobody wrote Hadrian a $900 million check. That figure is the government share of a broader effort Hadrian is part of. The facility described in the article targeted about 80 percent automation and training in as little as thirty days, and it was in explaining that facility that Power said "we have to automate our way out of this problem" and that the machines let Hadrian "not have to, say, hire or fire people." He was describing what the public money buys.

The Army is in too. Red River Army Depot awarded Hadrian a contract with about $39.2 million obligated on a firm-fixed-price basis and a cumulative face value up to $80 million. Hadrian was the only bidder.

The jobs math

Now put the Mesa promise next to the Mesa contract, because they are different documents making different promises. The ribbon-cutting release said more than 350 new jobs in early 2026. The city agreement requires at least 300 full-time jobs averaging $71,000, within five years, and in exchange Mesa reimburses Hadrian $750 for each qualifying Mesa resident it hires, paid after the employee completes a full year and at least five hours of training, capped at $75,000, which works out to roughly 100 hires. The public heard 350 jobs this year. The city signed for 300 jobs within five years.

In August 2026 Hadrian was still posting entry-level Mesa technician roles that prioritized curiosity and willingness to learn over industry experience. I could not find independent confirmation that 350 Mesa positions were filled. Maybe they were. What I can say is that the only source for the Mesa headcount is the company that told the Navy it doesn't have to hire.

Hadrian has missed a dated promise before. Its Series C post in July 2025 said a 400,000-square-foot corporate and research headquarters would be fully operational by January 2026. In August 2026, Axios reported that the Los Angeles headquarters and the San Francisco engineering hub were still being planned.

Before somebody puts it in the comments, Hadrian says it plans to grow from about 700 employees to 2,000, per Manufacturing Dive, and that is a big hiring claim that belongs in this story. It also doesn't contradict anything above. A company can hire thousands of people while designing every unit of output to need less experience and less commitment to any particular one of them. Hiring a lot of people and needing none of them in particular is the plan.

The charge under the ribbon

On May 21, 2026, an individual filed NLRB case 28-CA-387462 against Hadrian. The listed location is Mesa, Arizona, 85212. The allegation is Section 8(a)(1), concerted activities, with retaliation, discharge, and discipline listed. As of this writing the case is open, and open is all it is. It's an allegation. Nobody at the Board has found anything.

I'm being exact because this is where people get sloppy. The charge document is not public on that page, so the merits are unknown, and nobody should say Hadrian fired a whistleblower or that the Board ruled against the company. The docket supports neither. What it does establish is that the ribbon was cut on January 29 and, less than four months later, someone in Mesa told the federal government that Hadrian punished them for acting together with coworkers. I think the public should know that exists, and I think it belongs on the same page as "Everyone has equity. Everyone has purpose."

There is also a pattern in who gets to look inside, and I'm part of it. Sourcery got an exclusive. John Coogan produced a factory film while working at Founders Fund, which invests in Hadrian. Not Boring disclosed both an investment and a sponsorship in its profile. My request to film at Mesa was declined. The friendlies get the tour and I get the wall, which proves nothing about the labor charge but does explain why the public documents are the only way into this company.

Their strongest case

Hadrian's best argument is not weak, so here it is. The skilled-trades shortage is real. The defense industrial base needs more capacity, and the Navy is not making up its submarine backlog. Hadrian says it is hiring toward 2,000 people, paying equity to factory technicians, and making each of them far more productive than a traditional shop could, and Power's stated intent, every time he's asked, is to supercharge workers rather than replace them.

Then hold that intent up against what he says when the audience is the Navy. About 80 percent automated, training in as little as thirty days, capacity parked in machines so the company doesn't have to hire or fire. I found no verified Mesa layoffs, no specific displaced shop, and no independent Mesa headcount, and I'm saying so plainly because the argument doesn't need me to hide it. A hiring plan doesn't cancel a labor-optional operating model. It just makes the contradiction easier to see.

Capacity without craft

America needs more ships, more missiles, more parts, and more people who can make them, and automation belongs in that effort. I'd be lying if I said I wanted the repetitive work done by hand. Build the robots.

But a thirty-day checklist is not industrial memory. A country that wants to make its own weapons needs machinists and apprentices and small shops and engineers who understand the work beyond the interface, because the interface is owned by a company, and companies get bought, pivot, and fail. When the Navy pays for a system that absorbs the craft, and the ribbon-cutting still sells the worker as the centerpiece, the public is paying for both halves of the same trick. That's the Deskilling Subsidy. Spend the money in a way that leaves the American worker harder to replace, because capacity without the craft underneath it is a dependency with a hard hat on.

If you know someone still posting the ribbon-cutting photo, send them this.

• Dr. J

I had a folder in Google Drive with the English subtitles for the IMDb Top 250 and no plan for it. The idea I kept coming back to is from granular synthesis. You take a sound, cut it into tiny overlapping grains, and the one continuous thing becomes a pile of small things you can measure and rearrange. Do that to a film and you get a few dozen five-minute windows of dialogue, each small enough to ask a simple question about. How tense is this? How funny? Is anybody grieving?

So I did it. 195 films, 10,129 windows, 3.8 million words of dialogue, eight emotion scores per window plus a dominant emotion, all placed on a shared clock that runs from 0% to 100% of the film so a 90-minute comedy and a three-hour epic can be laid on top of each other. The ten things I found come first. The method, the caveats, and the stuff that did not work are after that, for the people who want to argue.

Ten things the charts say

1. Great films front-load the jokes

Humor peaks about a quarter of the way in and sinks below the film's own average by the end. This is dialogue humor only, so a sight gag or a Buster Keaton fall is invisible here. The words still tell one consistent story: you get the laughs early and you pay for them later.

2. Tension climbs the whole film and then falls off a cliff

On average, tension rises for 88% of the runtime and then drops harder than anything else in the data. That drop is the resolution. Arousal does the same thing a beat behind. It is the most consistent shape in the whole set.

3. The ending is the one place these films agree

For most of the runtime the films disagree about where their beats go, which is why the average curves are so flat in the middle. Then the last tenth arrives and they line up. Warmth peaks there in 31% of films and valence in 30%. Humor peaks there in 2%.

4. Warmth, grief and moral weight all get heavier as the film goes

On the raw 0 to 4 scale, the average film gains about half a point of warmth, grief and moral weight between its first tenth and its last tenth. Humor loses more than half a point. Valence and conflict barely move on average, which hides a lot of individual movement. That is the next chart.

5. Fewer than half of them end happier than they start

93 of the 195 end with higher valence than they opened with. 102 end lower. The happy ending is not the default in this canon, and a lot of the films you would call uplifting get there through an ending that is quieter and sadder than their opening.

6. There is no secret set of story shapes

I clustered the curves looking for the six or seven shapes everyone hopes to find. The best split on all eight axes has a silhouette score of 0.04. On valence alone it is 0.08. A score of 1 would mean clean separation, and 0 means the clusters are no tighter than a random cut. The honest reading is that these films sit on a continuum. It could be that my axes are the wrong ones, and I say more about that below, but on this data there are tendencies, not types.

7. The one split that does show up is "ends up" versus "ends down"

When you force the valence curves into two groups, the only thing the groups disagree about is the last few minutes. 90 films rise at the close. 105 fall. Everything before that looks the same.

8. The peak-end rule does not predict which films rank highest

Kahneman's rule says we remember an experience by its most intense moment and its ending. I tested it. Correlate each film's valence peak, its ending, and the average of the two against its place on the list. All three land near zero. The one valence number that moves with rank is the opening, and it moves the wrong way for a feel-good theory: films that open darker rank higher.

9. Tension and moral stakes go with a higher rank. Jokes go the other way.

Average tension across the whole film correlates with rank at +0.22, moral weight at +0.17. Humor is negative and not significant. These are modest numbers inside an already elite list, so read them as which way the wind blows, not as a formula. The wind blows toward stakes.

10. The rater passed the sniff test

Before you trust any of the above you want to know the model reads dialogue the way a person would. The most positive film on the list is Singin' in the Rain. The funniest is Monty Python and the Holy Grail. The most tense is Dr. Strangelove. Downfall, Full Metal Jacket and Oldboy sit at the bottom of valence, and the loudest film in the set is The Wolf of Wall Street, which anyone who has seen it will accept without a chart.

How I did it

The subtitles came from a folder of 200 SRT files, one per film, ranked at the time they were collected. Five came out before any analysis: four silent films that only have intertitles (Modern Times, City Lights, Metropolis, The Kid) and Kill Bill: The Whole Bloody Affair, whose subtitle file has 230 cues for a 232-minute cut and is clearly incomplete. Kill Bill Vol. 1 stayed. That leaves 195.

Each file was parsed, stripped of formatting tags and the ads subtitle sites inject, and cut into windows.

Every window's dialogue went to Jev, TypeSafe AI's evaluation model, through the Vercel AI Gateway, where it is free. Jev is not a chat model. You give it a piece of text and a set of typed questions, and it returns a calibrated answer for each without generating any prose. For a score question you supply the rungs of a rubric in order, and it returns a probability for each rung plus an interpolated score. That is a better fit for this job than asking a chat model to emit JSON and hoping the numbers mean the same thing on window 9,000 as they did on window 1. The model saw only the dialogue. No title, no year. If it knew it was reading The Godfather it would rate from memory of the film instead of from the text, and the measurement would be gone.

The eight axes, each a five-rung rubric scored 0 to 4, and the theory each one is anchored to:

Axis Anchored to valence Russell's circumplex model of affect arousal Russell, and Zillmann's excitation transfer tension Brewer and Lichtenstein's structural affect; Huron's theory of expectation conflict Zillmann's affective disposition theory warmth kama muta, the "being moved" research from Fiske and Menninghaus grief Bonanno's grief trajectories; Herman's stages of recovery moral weight Haidt's moral foundations; Shay's moral injury humor its own axis, because humor is not the opposite of sadness

Plus a ninth question that picks one dominant emotion from Plutchik's eight, or neutral. The pipeline is Bun and TypeScript, eight calls in parallel, and it finished 10,129 windows in about twenty minutes with 1,690 retries and no failures. The gateway was having a day. I had Claude Code write most of it.

Then the normalization. Every film's windows were placed on its own runtime, resampled to 100 points from 0% to 100%, and z-scored per film per axis. The z-score is the choice that makes the curves comparable. It throws away how tense a film is on average and keeps only where in the film the tension goes. The raw 0 to 4 levels are kept separately for the questions that need them.

Clustering was k-means on the resampled z-curves, downsampled to 25 points per axis, with k chosen by mean silhouette from 2 to 6. The peak-end test used Spearman correlation against each film's rank. I do not have IMDb's ratings in the dataset, but the Top 250 is ordered by weighted rating, so rank is a monotone transform of rating and a rank correlation on one is identical to a rank correlation on the other.

What this cannot tell you

Dialogue measures what the characters say, not what the audience feels. The scariest stretches of a horror film have no dialogue at all, and a score cue can flip a scene the words did not. Silence is recorded in this data as an empty window and interpolated across, which is the right call for shape and the wrong call if you care about the silence itself.

The rank correlations are small and the range is restricted. Every film here is already in the top 250 of a database of millions. Finding a +0.22 inside that group is interesting. It is not a recipe.

The scores are a model's judgments. I have not hand-coded a validation set against them, and the sniff test in Chart 10 is not a validation set. It is a smell. The axes are also my choice, anchored to theories I find useful, and a different codebook could find shapes that this one cannot see. The clustering result in Chart 6 should be read with that in mind: no types on these axes.

And five-minute windows hide anything shorter than five minutes. A single devastating line inside an otherwise calm window gets averaged into the calm.

What I would do next is add a second rater and see where they disagree, and then go looking for the films that break the average curves on purpose, because the interesting ones always are. The practical version of all this, for people who make videos rather than films, is in the follow-up.

• Dr. J

Suppose an agent is asked to open one support ticket. It calls the ticket API. The service creates the ticket, but the connection drops before the answer comes back. The agent sees a timeout, decides the attempt failed, and tries again.

Now there are two tickets. Both describe the same broken export. Every field is valid. The agent understood the assignment and selected the right tool. Its mistake was deciding that a missing answer meant nothing happened.

A timeout tells you that an answer did not arrive in time. It does not tell you whether the other machine changed the world. The request might never have arrived. It might still be running. It might have finished while the reply disappeared. Those possibilities can look identical from the caller's side.

That is a terrible place to ask a language model to improvise.

I want the agent to keep that outcome marked unknown. It should retain the identity of the action it attempted and have a defined way to resolve the uncertainty. Otherwise, the next round of reasoning can turn recovery into another action.

This is old distributed-systems territory. In End-to-End Arguments in System Design, Saltzer, Reed, and Clark explain why the application may still need duplicate suppression even when lower layers provide it. An application can originate a second attempt that looks like a perfectly new message to the communication system. The application knows the relationship between those messages. The network does not.

An agent can make that second attempt by thinking about the problem again.

HTTP's rules for retries already account for the distinction. An idempotent operation has the same intended effect when repeated. A client should not automatically repeat a non-idempotent request unless it knows the operation is safe to repeat or can establish that the original was never applied. This is why creating another ticket needs more care than fetching the ticket list.

The difficult part is preserving that distinction through the agent's tool interface. A wrapper that turns every timeout into success: false has thrown away information the rest of the system needs. Now the planner has a neat failure to repair, even though the business operation may already be complete.

The second call looks new

Idempotency keys help when the provider supports them. The client assigns an identifier to an operation and reuses it on retries. The provider recognizes the repeat and prevents an additional effect within its documented contract.

But a retry inside an HTTP client and a new call proposed by an agent are different things. The HTTP client might reuse the key correctly. The agent might respond to the same uncertainty by calling create_ticket again, producing a new tool-call identifier and a fresh provider key. Each layer can obey its local rules while the user gets two tickets.

A small runnable example makes the difference visible. The simulated ticket service creates the ticket and then drops its first response. The caller retries. Run it with Python 3.10 or newer and it prints:

No provider deduplication: 2 tickets
Same operation key: 1 ticket
New key after replanning: 2 tickets

This is a deterministic toy example with no LLM, network, or persistent database. Its service assumes an indivisible ticket write and deduplication record. It demonstrates what changing the key does under that assumption. It does not measure how often a model will choose to do it, or establish a production implementation's crash safety.

The third line is the one I care about. The provider honored its contract. The caller bypassed that protection by presenting one logical action as two different operations.

The identity has to belong to the operation the user authorized. A transcript message ID, a new worker, or a restarted run should not quietly create permission to perform it again. Create the operation record before dispatch, bind it to the account, target, and resolved arguments, and keep that record across recovery. If the model proposes another attempt at the same unfinished operation, route it through that record. Give each send a new attempt ID for tracing, while preserving the operation ID and its mapping to the provider key.

Matching the JSON is insufficient. Malcolm Featonby's account of idempotent APIs at Amazon points out that a caller may want two identical EC2 instances. Identical parameters do not necessarily mean a duplicate. AWS uses caller-provided request identifiers and checks for parameter mismatches. For its own database mutation, the server must record the identifier and the effect atomically. Writing the identifier and then hoping the effect follows leaves another failure window. That transaction does not automatically cover a downstream email or webhook consumer.

The integration has to decide what counts as one operation. One ticket for a particular approved support request is a useful identity. Every ticket with the same title forever is not. A hash can detect changed arguments; it cannot decide whether the user intended a second occurrence.

An August preprint on replay-resistant agent actions studies a related failure. In its synthetic agent-generation harness, an equivalent action can be proposed again after an uncertain outcome and receive a fresh single-use token for the same user authorization. The authors' CapLease design and a matched server-side ledger both track the authorization across token issuance. Both provide the tested replay protection under matched assumptions. The guarantees require trusted action identities and durable state. Preventing duplicate external effects also requires the destination to enforce idempotency. The study does not measure the rate of duplicate actions in deployed agents.

Persistence ends somewhere

Existing workflow systems already document this boundary.

Temporal's Activity documentation describes the case directly: a worker completes an Activity and crashes before reporting completion. The history lacks the successful result, so the Activity can run again. Completed Activities recorded in history do not rerun during normal replay. The vulnerable interval is between the external effect and the recorded result. Temporal recommends idempotent Activities and explains that downstream services enforce the keys.

The workflow engine can preserve its own history without participating in the ticket service's database transaction. I argued in the graph-engineering essay for explicit state and recovery between steps. A graph still needs to represent uncertainty inside a step that talks to another system.

There are also time limits. Stripe's idempotency contract returns the saved status and body for repeated requests with the same key, including saved 500 errors. Keys can be pruned after they are at least 24 hours old. Reusing a pruned key creates a new request. An agent that resumes days later cannot assume a stable string still carries the same duplicate-prevention guarantee. It needs to know the provider's scope and retention rules, and reconcile the existing operation when that guarantee no longer applies.

Writing an intent to a local database is useful. It preserves what the agent meant to do, but it does not make a remote write atomic with the local record. You still have to account for the moment when one system has committed and the other has not learned about it.

Make unknown change the next action

An unknown label earns its place only if it changes execution. It should stop the planner from treating the unresolved operation as a fresh opportunity to make progress.

The recovery path depends on the service. A provider-supported key may permit a bounded retry of the same operation. An operation ID may support a status query. If the caller received a resource ID before losing contact, it may identify the ticket to inspect. The missing reply may also have contained the only copy of that ID available to the caller. An asynchronous service may report 202 Accepted while processing remains incomplete. That response does not promise the work will eventually succeed. Accepted and completed deserve different states.

Readback needs its own care. A search returning no matching ticket may reflect a delayed index, another account, or an original request that has not finished. Even a current read can race with a write still in progress. Finding the particular completed operation can resolve uncertainty. Failing to find it does not automatically authorize another create request.

When the provider offers neither safe repetition nor a conclusive way to inspect the outcome, the uncertainty belongs in the product. Tell the user what was attempted and what remains unresolved. For an action where duplicates matter, waiting for investigation can be the correct result. A system that hides this choice in an optimistic retry policy has chosen for the user anyway.

Several effects require several records. Creating a ticket and sending its notification are separate operations. If the notification fails, creating another ticket is a bad recovery strategy. A compensating action may repair a partially completed workflow, but repair has limits. Garcia-Molina and Salem's sagas work treats compensation as an application-defined operation. A correction email cannot make the recipient unread the first one.

The test I want happens immediately after the external system commits. Drop the response. Restart the agent. Let it replan or delegate. Count the resulting objects and inspect the operation IDs, rather than judging only its final explanation. Repeat with a request that never reached the service, with delayed visibility, and with two workers trying to recover the same operation. A design that avoids duplicates by never completing anything has failed a different part of the test.

Measure successful completion, unwanted effects, unresolved outcomes, and time spent reconciling them. Keep the cases where the system admits it cannot determine the answer. That uncertainty belongs in the results, even if it makes the demo less satisfying.

Before I trust an agent to run unattended for hours, I want to see what it does after one missing response. It should be able to say which operation is unresolved and how it will find out what happened. "I'll try again" is only reassuring when it still means the same operation.

- Dr. J

Meridian is the independent media company behind the show and this newsletter. Right now, it is small: me, the work you already see, and the systems I've built to publish it. The show has more than 30,000 YouTube subscribers, and the work is about technology, labor, money, and power, especially what those systems mean for the people doing the work.

I want Meridian to become a home for more than my own work. That means finding people who have something worth saying and can make the thing, not just pitch it. It also means admitting the obvious: I cannot do all of it myself.

So I am looking for two people.

Host and producer

Make your own show about technology, work, money, or power. You bring the premise, research it, get on camera, and deliver a finished episode.

Newsletter writer

Turn reporting on technology and work into a newsletter with a clear argument and strong sources.

Both opportunities are remote and open to U.S. citizens.

I do not need you to sound like me. I need you to have a point of view, support it with real work, and finish what you start.

View the roles and apply

- Dr. J

In this video, I examine the payroll numbers behind his argument, reduced hiring of young workers, employer incentives around AI and offshoring, and what happens when the wage on offer barely covers the cost of living near the office. I also draw on my experience on the hiring side of the desk.

My advice to someone trying to find work is still to keep moving: learn, apply, build something, and look for a way to earn. Taking responsibility for your next move does not require accepting a layoff or rejection as a verdict on your character. Employers make decisions too.

In the video

• 00:48 What the payroll numbers measure

• 02:32 Amazon and the hiring side of the desk

• 03:21 The entry-level door

• 04:58 H-1B, offshoring, and employer incentives

• 06:04 Degrees, debt, and underemployment

• 07:09 Housing and the cost of independence

• 11:33 My advice to young men

First published on September 8, 2026. The data and policy discussion reflect that recording date. Payroll jobs and labor force participation measure different things; participation includes people looking for work. The graduate figures include men and women. The Amazon cuts are not a verified count of young American men replaced by AI, and the Stanford findings do not establish causation.

Sources discussed

• The original Flynn post

• BLS: payroll and household measures

• BLS: participation by age and sex

• Andy Jassy on AI and the workforce

• Stanford Digital Economy Lab research update

• New York Fed: the labor market for recent graduates

Subscribe for more on work, technology, and the decisions shaping your livelihood.

In this video, I separate three numbers that get blurred together in the AI infrastructure story: future orders, sales, and cash moving in and out. Oracle is growing quickly. That growth also requires an enormous amount of construction and financing before the promised business turns into cash.

I walk through the reported results, the funding plan, and my argument about what those decisions mean for workers. The distinction matters: a large order book can be evidence of demand without being money available to spend today.

In the video

• 00:00 The $664 billion order book

• 02:49 Give the growth its due

• 03:27 Orders, sales, and cash

• 04:13 The cash-flow gap

• 04:59 Borrowing, share sales, and the bet Oracle is making

• 06:53 Layoff reports and the stakes for workers

Recorded and first published on September 14, 2026. The September layoff discussion reflects preliminary reporting available that day. The March 31 layoffs are a separate event. The financial statements do not establish that a particular data-center project caused a particular job cut. Backlog can include customer prepayments; it is neither cash available today nor guaranteed profit.

Sources discussed

• Oracle Q1 FY2027 results

• Oracle funding plan for calendar 2026

• Moneycontrol reporting on possible September cuts

Subscribe for more reporting and commentary on AI, work, and who gets paid.

I sat down with him at Ai4 to ask what using Neuralink actually feels like. He explains how he trains the system, how intended movement becomes cursor control, and why the details of using it on his own computer matter.

Then we talk about the rest of his life: building a speaking business, traveling, and the Christian faith that helped him find contentment before he received the implant. I spend a lot of time covering technology that gives institutions more control over people. This conversation asks what it means for a tool to give someone more control over his own life.

In the conversation

• 02:21 How intention becomes cursor control

• 05:47 What using the implant feels like

• 10:07 What Noland hopes the technology could eventually do

• 13:40 Building a life and a speaking business

• 21:55 Faith, contentment, and the decision to get Neuralink

• 32:33 Who is responsible for how we use AI?

Recorded at Ai4. Thank you to Ai4 for providing the interview space and connecting us. First published on YouTube on September 9, 2026. The discussion of restoring movement describes hopes for future technology, not a capability Noland has today.

Subscribe for more conversations and commentary on technology, work, and human judgment.

The level of incompetence was unlike anything you can comprehend until you work somewhere like that. They made me do silent layoffs and hire in India to replace the people we cut. The CIO came in and purged the ranks from the top down with Indian nationals. My entire management chain up to him was Indian.

The office at 18850 N. 56th Street had more than 1,000 H-1B filings tied to that location across hundreds of legal employers. A campus of shell employers, staffing companies, contractors, and client arrangements, all pointing at the same physical floor. I made the video when I first left because people needed to see what the arrangement looked like from inside it.

I reported what I saw to Harmeet Dhillon and the Department of Labor. I gave them proof that I RISKED MY LIVELIHOOD TO OBTAIN. Harmeet's fixer, Varda Hussain, sent me form letters.

That is the game. They staff the floor by any means necessary, then hide behind a vendor, a federal form, or a silent layoff. I am pro-American. American jobs should go to qualified American citizens first. If we lack qualified Americans, train them.

The form invents the occupation after the seat is filled

USCIS approved 406,348 H-1B petitions in FY2025. Of those, 252,088, or 62 percent, were in computer-related occupations. Systems analysis and programming alone accounted for 204,971 approvals. Among the computer-related approvals, 21 percent were for initial employment and 79 percent were for continuing employment. The shortage story has a retention machine bolted to it.

The credential is a costume. USCIS reported that 57.5 percent of approved beneficiaries held a master's degree, while 123,824 listed a bachelor's as the highest degree. Doctorate holders had median compensation of $105,000. Bachelor's degree holders had a median of $132,000. The extra letters did not raise the bid.

The intake valve is everywhere a form can be printed. A separate USCIS report counted 45,796 FY2025 petitions from employers with no more than 25 full-time equivalent employees. DOL also certified 1,446 applications from Grandison Management listing 57,840 requested positions. Those are positions on applications, not visas or unique people, but the number should make your eyebrow move.

The form invents the occupation after the seat is filled. The registrar is the recruiter. The shortage is a lottery with a federal login.

They mock you as backwards rednecks

A Y Combinator partner called hiring Americans "DEI," then sneered at Ohio as flyover country. He locked his account after Palmer Luckey and the FTC chair spent the next 48 hours on him. Garry Tan defended him.

The a16z new media team reached out to me last week. I proposed REAL INTERVIEWS about what the 99 percent of Americans outside San Francisco want to know. The woman on the call looked like I had thrown cold water on her. She said they "weren't interested in dumbing down their content."

That sentence tells you everything. They think you are stupid if you do not live in San Francisco and work in tech. They need you cast as low-class trailer trash so they never have to answer you.

I am from the Rust Belt. I grew up in the wreckage, I will always keep the spirit of Detroit in my heart, and I believe in the American soul of this great country. There are more important things than money and status. Young people need to understand that before they accept comfort money in exchange for never trying to boil the fucking ocean.

I slipped out through the window

I lived that other life. I started from a similar place, except I was living out of my car. I worked my way into prestigious tech companies, and succeeding in the rank and file came easily because I had a stronger work ethic than most of my coast-born counterparts.

Then you find the ceiling: Stanford pedigree, family connections, the club you cannot join this late in the game. You can make half a million dollars in total compensation and outsource your entire life so you can grind harder. Maid, meal prep, Equinox membership, nice watch. Everything looks good while you work harder to pay for the machinery that lets you keep working harder. Ouroboros.

Eventually you wonder what the fuck the point is. Then a window opens. You meet a nice girl and see a wife, children, and a church without a rock band or projector setup. You realize the illusion was mostly propagated by bitter childless people, or people with "show children" who outsource the interaction and raising to somebody they pay.

I cannot tell you what to do, but I slipped out through the window. Things are not always idyllic. I never question the point of my life anymore, and that is one of the most precious things a person can have.

That changed what work was for. A job should support a life, and leadership means carrying the pressure so your people can do theirs.

Be the guy who gets things done

The clearest lesson I learned about leadership came on a high-stakes engineering floor. I was managing a team through a major demo. They had worked their asses off and built software worth being proud of, so the day before the performance I wanted the room set and two full rehearsals completed.

I run demos the way I ran classical music ensembles when I was a composer. The performance should be the most boring repetition the performers have ever done. Everything should be locked deeply enough that muscle memory can take over.

Then I looked around and realized how tired everybody was. Stoked, but tired. At that moment, my entire job was morale management. I took food and drink orders, rotated people out for breaks, and cut everybody loose to rest as soon as I could.

The job was to be the guy who gets shit done, regardless of what the work looked like. Sometimes that meant writing a mountain of code. Sometimes it meant cracking a joke because the room was too tense. Technical talent pointed at the wrong problem is useless. Be the guy who gets things done. Keep it simple.

If you watch one thing after this, make it the American Express floor. That is where this story started.

Be well. Never let them change you.

- Dr. J

Check your employer's story

Bring three readers to this newsletter through your personal link below. Once all three confirm their subscriptions, I'll send you my Employer Investigation Kit free: a seven-page guide to checking layoffs, visa filings, wages and company disclosures, plus a worked Microsoft example and reusable worksheets.

Persistent memory lets an agent carry yesterday's correct answer into a world where it is wrong.

Suppose an agent learns how your company calculates net revenue. Refunds are deducted from the payment amounts, so summing the payments gets the right answer. The agent saves the query and an explanation. Next month, accounting changes the model. Payments retain their gross amounts and refunds live in a separate table.

The remembered query still runs. It returns a plausible number, and the agent can point to the session where it learned the procedure. It faithfully remembers the old procedure. Your report is wrong.

Better recall can make this bug easier to reproduce.

In July, I argued that agent systems were becoming a graph engineering problem. Useful work needed explicit state, dependencies, and recovery across many steps. Persistence leaves us with another obligation. A system can resume exactly where it left off, carrying assumptions that should have been retired while it slept.

The system needs to know what would make an old lesson stop applying. We tend to bury that requirement inside the word "memory." We use it for a user's instructions, old tool results, conversation summaries, facts extracted from documents, and the agent's guesses about why something worked. Those records have different owners and different reasons to be trusted. Put them in one searchable pile, and a useful observation can quietly acquire the standing of a rule. Staleness starts with something that was correct. Bad initial inferences and accurate facts used outside their scope need different checks.

I like plain files. In Agent memory as a file format, Cal Paterson proposes portable Markdown pages with optional metadata and a search index. The agent's notes remain readable, editable, and independent of the software that wrote them. I want those properties. Paterson also recommends source links so agents can check outdated material. That recommendation deserves as much attention as the file format. The stale revenue query may be the closest semantic match to the new revenue question.

Researchers have started separating these failures from ordinary recall. STALE tests 400 scenarios where later information changes what an assistant should believe without explicitly announcing that an older memory is false. In one LightMem diagnostic case, a user has moved from Seattle to Austin. For a question that still presupposes Seattle, the old entry ranks first and the Austin update tenth. Both are available. The answer follows the old state.

Across the benchmark's probes, Gemini-3.1-pro, the best general model tested, scored 55.2% overall. The authors' targeted CUPMem prototype scored 68.0%. These are constructed scenarios, with human validation and model-based judging, not a measurement of how often your assistant fails. The benchmark shows a specific failure worth testing in our own systems. Retrieving the update can still leave the old belief in control.

A stored answer saves work because you assume it remains reusable. HTTP caching gives responses a freshness lifetime and supports validation against the origin. The client has a way to check whether it can keep using its copy. Reuse comes with conditions.

For an agent's observations about a repository or an API, that is a useful starting point. The saved revenue query should carry the source it came from and the version of the accounting model it depended on. When that model changes, the saved query needs review before reuse. A timestamp on the memory file does not do this. Rewriting the same old lesson this morning gives it a younger timestamp and the same old defect.

In 1977, Jon Doyle's truth maintenance work recorded the justifications behind a program's beliefs so it could revise them when their supporting assumptions changed. That is a better ancestor for a model-written conclusion than a bag of text chunks. A conclusion can lose one justification and retain another. A past decision can remain important historical evidence after it stops being the current policy.

The cache analogy has a boundary here. An explicit instruction to use Python in a particular project has an author and a scope. A recollection that a service was healthy yesterday is evidence about yesterday. An agent's theory that a timeout was caused by rate limiting may never have been established in the first place. A seven-day expiration policy cannot tell these records apart.

The engineering gets easier when we keep those distinctions and track the inputs to the things we generate. Build systems à la carte describes the familiar job of rebuilding outputs when their dependencies change. A memory summary is also an output. It came from particular records through a particular transformation. If a source record changes, everything derived from it deserves reconsideration.

Mark Lubin's Synix uses a build system for agent memory, tracking sources and transformations so affected artifacts can be rebuilt. Graphiti represents temporal validity and the source episodes behind extracted relationships. The difficult work is deciding which facts depend on which sources, and making the whole application honor the result when something changes.

When source changes are observable, a dependency graph can identify which generated notes need checking. It cannot prove that the model correctly summarized the source, or that it identified every dependency. Hashing a hallucination gives you a reproducible identifier for a hallucination. Where dependencies are uncertain, the system may need to throw away a larger summary and reread its sources. For an external source that exposes no revision or change notification, checking may require another read. The right frequency depends on how quickly the information changes and what a stale answer would cost.

An August 31 preprint on invalidation contracts tests a narrow version of the problem. Agents save API error fixes and reuse them across tasks. Responses carry a version stamp for each reference table, and a separate change report identifies affected rows. The client can discard the corresponding fixes while retaining unrelated ones. The evaluation covers seven models and roughly 9,400 episodes across two APIs built for the study.

The authors separate whether a cached fix remains valid from whether the model applies it. Tracking versions addresses the first. The second still varies by model. Detection happens when a response arrives, so the design can incur a stale attempt before correction. The APIs and recovery metadata are hand-written for the experiment. Production services would have to expose and maintain those signals.

Checking the world can also improve what gets remembered in the first place. In the September 10 preprint Grounding Agent Memory, Microsoft researchers gave a memory curator read-only tools to check candidate lessons against the environment. Across five paired runs of a 40-question CLBench database test using GPT-5.4, with schema changes halfway through, reported pass rates were 39% without memory, 70% with trajectory-only memory, and 73% with environment-probed memory. The 95% confidence intervals overlap for the two memory conditions. The largest observed jump is from no memory to trajectory-only memory. The extra checking produced a much smaller difference in this experiment. Useful memory is worth maintaining.

For someone building an agent today, I would start by keeping the original evidence available. A summary should lead back to the records that support it. Keep explicit user instructions distinguishable from what the model inferred, and record the project or account a lesson belongs to. A perfectly accurate memory from the wrong customer is still the wrong input.

Then give source changes somewhere to go. A schema revision can mark dependent query notes for review. A stale flag should suspend reuse while the system checks whether the note still applies. A corrected document can invalidate its summaries and search entries. A newer approved decision can supersede an older decision without erasing the history. The event needs to reach what the agent will actually read, including any context that has already been assembled. Changing a database row does little for an agent still reasoning over the old copy in its prompt.

This can start with source IDs and revisions attached to a Markdown note, plus a check before reuse. When a fact is cheap to fetch and expensive to get wrong, remember how to fetch it. Fetch current service health and schema information when the task depends on them. Check permissions with the system that grants them. A sentence in an agent's notebook should not be able to approve its own deployment.

The test I would run changes the world halfway through. Let the agent learn the original revenue procedure, alter the accounting model, and ask for another report. Check the answer and the sources it used. Repeat with a harmless source change to see whether it throws away useful work unnecessarily. Include corrections that arrive indirectly, and a question whose wording invites the old assumption. Measure the revalidation cost alongside correctness.

That is the version of persistent memory I want. It should save the agent from rediscovering everything while making it possible to find out which lessons no longer apply. A demo where the agent recalls what it was told is the beginning. I want to see the next session, after that thing has changed.

- Dr. J