All posts

Verification Debt

AI

One of the talks at WeAreDevelopers World Congress in San Jose in September gave a name to something I suspect most people building with agents have felt. The speaker, from SonarSource, called it verification debt: the gap between the code that has been written and the code that has been verified.

The argument went like this. Agents now write code faster than humans can review it. Their bugs are rarer, but more convincing. And that combination produces a habit the speaker called "cognitive surrender": the output is convincing, so we check it less.

Rarer and More Convincing

That pairing is the uncomfortable part. If agents wrote obviously bad code, review would be easy. You would catch it. Rarer bugs mean most of what you review is fine, which trains you to expect fine. More convincing bugs mean the occasional bad one looks exactly like the good ones. Put those together and review gets faster at the same moment it gets less useful.

The speaker cited one study in which people followed advice from artificial intelligence (AI) 79.8% of the time even when the advice was wrong. I do not find that number surprising. I suspect it describes most of us on a tired evening. When the output reads well, runs, and handles the case you described, the effort to doubt it has to come from somewhere, and usually it comes from nowhere.

This is the failure I keep coming back to on this blog. AI fails plausibly, not loudly. A crash is cheap, because you can see it. The expensive failure is well formatted, confident, and wrong, and it is expensive precisely because everything downstream keeps working.

Why Debt Is the Right Word

Technical debt is code that works today and costs you later. Verification debt is code nobody has checked that everyone is treating as checked. It builds up silently. Every merge of something nobody really read adds to it. And like technical debt, you do not get to deal with it when it is convenient. You deal with it when something breaks, at the worst time, with the least context.

My take: the dangerous thing about verification debt is that it is invisible on every dashboard people look at. Velocity goes up. Pull requests close. The tests that exist pass. Nothing measures the gap between what was generated and what anyone actually confirmed, so nothing tells you it is growing. You find out how big it is all at once.

Two Kinds of Checking

The fix the speaker proposed was not "review harder". It was to combine two kinds of checks, because each catches what the other misses.

KindWhat it checksWhat it is good at
AlgorithmicStatic analysis, security, dependenciesExhaustive and repeatable: the same answer every time, on every line
AgenticIntent, business logicJudgment: does this do what was meant, and does it fit the rules of the domain

Algorithmic checks are tireless and blind. They will find every unsafe pattern they know about, on every line, every time, and they have no idea whether the code does what you asked. Agentic checks can read intent, which is exactly the thing a convincing bug gets wrong. Neither one alone closes the gap.

I think the pairing is right, and I would add one condition. An agentic check is only as good as the definition of intent it checks against. If nobody wrote down what the code was supposed to do, an agent reviewing it can only confirm that it does something reasonable. That is how a convincing bug gets a second convincing approval.

Guide, Verify, Solve

The talk framed the fix as the Agent Centric Development Cycle, in three steps:

  1. Guide. Set the context and the constraints before the agent writes anything.
  2. Verify. Run the algorithmic and agentic checks on what it produced.
  3. Solve. Fix what the checks found, and go around again.

The loop is wired into continuous integration (CI), the automated build and test run on every change, and it repeats until the build is green. Verification stops being something a tired person does at the end and becomes something the pipeline does every time.

The step I would underline is the first one. Guide is where verification debt is either prevented or guaranteed. It is the point I made in Evaluation Is the New Code Review: a check is only as good as the definition of right it checks against, and that definition has to exist before the code does. A green build against a vague definition is not verification. It is a faster way to approve the wrong thing.

Where I Would Start

If I had to reduce verification debt on a project tomorrow, I would not start with tooling. I would start with one question, asked before the agent writes a line: how will we know it is right?

If you can answer it, the answer becomes the Guide step, the agentic check has something real to check against, and the CI loop has a definition of green that means something. If you cannot answer it, stop. That is the finding. Code you cannot define as correct is code you cannot verify, and code you cannot verify is debt from the moment it is written.

The speed of agents is real, and I build with them every day. But speed of writing was never the constraint that mattered. The constraint is how fast you can know that what was written is right. Every hour an agent saves on writing is an hour of checking you now owe, and the only question is whether you settle it on purpose or by surprise.

Leave a comment