What Teams Actually Use for Verification Loops in 2026
A field guide to the sensors teams put in coding agent loops, from compilers and linters to Playwright, screenshots, judge models, evals, mutation testing and human review. What each one catches, what it misses, what it costs, how fast it answers and how noisy it is, plus a playbook for which to stack on which kind of change.
A coding agent finishes a change to a signup form and reports: "Implemented the new field, all tests pass, verified the form renders correctly." Every word of that is true. The type checker is happy, the unit tests for the validation function pass, and the component renders without throwing. On a phone, the new field pushes the submit button below a sticky footer, and nobody can sign up. No check in the loop could have seen it. None of them looked at a screen.
The agent did not lie. It reported what its sensors told it. The problem is that "verified" meant three cheap checks, all of which measure whether the code is well formed, and none of which measure whether a person can use the page.
Every team building with agents ends up assembling a stack of checks around them, some decades old (compilers, tests, CI) and some new (agent-driven browsers, judge models, evals). Each sees a different slice of reality at a different cost, speed and noise level. This post is a field guide to that stack as teams use it in 2026, and a playbook for which sensors to put on which kind of change.
What a sensor is, and the five things to know about one
A sensor is anything that looks at the agent's work and returns a signal the agent (or a person) can act on: a compiler error, a failing test, a lint finding, a described screenshot, a judge's verdict, a review comment.
For each sensor you put in a loop, you want to know five things:
- What it catches. The class of defect it can see at all.
- What it misses. The classes it is blind to, however well you tune it.
- Cost. Compute, tokens, and human attention per run.
- Latency. Time from "agent finished an edit" to "agent hears back." Slow sensors run less often and get batched, which blurs the signal.
- Noise. How often it fails for reasons unrelated to the change, or passes when it should not. A noisy sensor teaches an agent to ignore it.
Most stacks go wrong on the fifth. A sensor right 95% of the time sounds fine until an agent runs it forty times a day.
The sensor taxonomy
Costs and latencies are rough orders of magnitude for a mid-sized codebase, not benchmarks. Your numbers will differ; the ranking rarely does.
| Sensor | Catches | Misses | Cost per run | Latency | Noise |
|---|---|---|---|---|---|
| Compiler and type checker | Wrong types, missing symbols, broken imports, unhandled cases in exhaustive matches | Any behaviour that type-checks | Near zero | Seconds | Almost none |
| Linters and static analysis (ESLint, golangci-lint, Semgrep, CodeQL) | Banned patterns, known bug shapes, security smells, your own architectural rules | Logic errors that look like normal code | Near zero to low | Seconds to a few minutes | Low for precise rules, high for broad ones |
| Unit tests | Behaviour of a function on the inputs someone wrote down | Inputs nobody wrote down; integration between parts; tests that assert the wrong thing | Low | Seconds | Low, unless they touch clocks, network or shared state |
| Integration and contract tests | Parts that disagree: schemas, APIs, database queries, migrations | Real third-party behaviour when the fake differs from the real thing | Medium | Minutes | Medium |
| CI | Everything above, in a clean environment, against the merge base | Whatever the suites don't cover; anything only true on a laptop | Medium (runner minutes) | Minutes to tens of minutes | Inherits every flaky test it runs |
| Scripted browser tests (Playwright, Cypress) | User journeys that break: buttons that don't submit, pages that don't load, flows that dead-end | Visual and layout problems unless explicitly asserted; journeys nobody scripted | Medium to high | Minutes | Medium to high: timing, animations, test data |
| Agent-driven browser (Playwright MCP, computer use) | Problems a person would notice exploring: broken flows, confusing states, errors on screen | Anything it doesn't think to try; it can believe an action worked when it didn't | High (tokens) | Minutes | High run to run |
| Screenshots as evidence | Layout, overflow, contrast, wrong state on screen; before and after comparison | Behaviour between the captured states; anything off-screen | Low to capture, attention to review | Seconds to capture | Pixel diffs are noisy across platforms; described captures less so |
| LLM judge and reviewer agents | Rule violations that need judgment, missing tests, suspicious logic, mismatches between intent and diff | Shared blind spots with the author model; anything outside the diff it was shown | Medium to high (tokens) | Minutes | Medium: verdicts vary run to run and have known biases |
| Evals | Regressions in model-driven behaviour: prompts, tools, agent flows | Inputs outside the eval set; drift after the set was written | High | Minutes to hours | Medium, needs repeated runs |
| Property-based tests (Hypothesis, fast-check) | Edge cases nobody listed: empty, huge, unicode, ordering, round-trips | Properties nobody stated | Low to medium | Seconds to minutes | Low; failures shrink to a minimal reproducing input |
| Mutation testing | Tests that run code without checking it | Nothing about the product directly; it grades the tests | High if run on everything, low on a diff | Minutes | Some equivalent mutants are unavoidable |
| Human review | Wrong intent, bad design, risk, things nobody thought to check | Whatever the reviewer skims; large diffs | The most expensive minute in the loop | Hours | Varies by reviewer and by day |
Read down the "Misses" column: every sensor is blind to something a later row can see, and later rows are slower and dearer. So teams layer them.
Cheap, deterministic sensors run on every edit and absorb most attempts. Expensive, judgment-based ones run once per change, on work that already passed everything below. A judge reviewing code that does not compile is money spent on the wrong layer.
Why "ask the agent to verify it" is not a sensor
The first thing everyone tries is an instruction: "test your changes before you say you're done." It helps a little. But an agent's description of its verification is not verification.
Anthropic's own write-up on harnesses for long-running agents names the failure directly: Claude tended to mark a feature complete without proper testing, and did much better once explicitly prompted to use browser automation and test as a human user would. Their computer use documentation carries the same warning in a different form: Claude sometimes acts as if its actions succeeded without explicitly checking the results.
The fix is not a sterner instruction. Every claim of verification must point at something a sensor produced: a test log, a CI run on the exact commit, a described screenshot. No artifact, no verification. Claude Code's best practices put it simply: give Claude something it can run that comes back pass or fail.
Layer one: compilers, types and linters
These are your best sensors: deterministic, nearly free, and fast enough to run on every edit.
Make the type system do more work. Exhaustive matches, branded id types and non-nullable fields turn runtime bugs into compile errors an agent fixes in one attempt. Strict mode is worth more with agents, because the agent pays the cost of satisfying it.
Write linters for your own rules, not just the community's. Semgrep is the common choice because its rules look like the code they match. If "handlers never call the database directly" lives in an instruction file today, it should be a Semgrep rule. An instruction is followed most of the time; a lint rule, every time.
Write lint messages for the agent. A finding that says "forbidden call"
leaves the agent guessing. A finding that says "use store.Get instead;
handlers must go through the store so tests can fake it" gets fixed on the
first attempt. OpenAI's
harness engineering post
describes doing exactly this: lint error messages written to carry the
remediation into the agent's context.
The limit: none of these sensors know what the code is for.
Layer two: tests and CI
Agents write tests that pass. The risk is tests that assert whatever the code currently does, check a mock instead of the real thing, or had their expected value edited to match a bug. Anthropic's Claude Sonnet 4.5 system card lists tests that verify mocks rather than real implementations among the reward hacks it still sees.
Three cheap counters:
- Tests that drive the real thing: the real database in a container, the real HTTP stack. A fake that diverges from production is a sensor calibrated to the wrong instrument.
- Changed-line coverage. A floor on the lines this change touched stops untested branches without demanding a whole-codebase cleanup.
- Flag test edits that ride along with code edits, so a person looks.
CI is the clean room. Its job is to run the full suites somewhere the agent cannot influence: a fresh environment, against the current merge base. That is why CI noise is so expensive. Google's testing team reported in 2016 that about 1.5% of their test runs gave a flaky result, and that about 84% of the pass to fail transitions they observed involved a flaky test. An agent that sees CI fail and pass on rerun often enough learns that red means "rerun," which is the worst lesson it can learn.
SWE-bench is a useful reminder of what tests can certify. It grades agents by running the repository's own tests, which is strong evidence of "fixed this without breaking that" and none at all that a maintainer would accept the fix.
Layer three: browsers, screenshots and evals
This is the layer that would have caught the signup form. It is also where most of the new tooling is, so it is worth separating the options.
Three ways to put a browser in the loop
| Approach | How the agent sees the page | Good for | Watch out for |
|---|---|---|---|
| Scripted tests (Playwright, Cypress) | It doesn't. A script asserts on the DOM | Journeys that must never break; running in CI on every change | Only checks what someone scripted; brittle selectors |
| Playwright MCP | Structured accessibility snapshots of the page, no vision model needed | Exploratory checks, reproducing a bug, writing the scripted test afterwards | Token cost per step; the README itself notes many coding agents now prefer a CLI for token efficiency |
| Computer use | Screenshots, mouse and keyboard | Anything with a screen and no DOM: desktop apps, canvases, native dialogs | Slowest and most expensive; can misjudge whether an action worked |
The pattern teams converge on: use the agent-driven browser to explore and reproduce, then turn what it found into a scripted test that runs in CI forever. Pay for the noisy sensor once; keep the cheap one.
Agent-driven browsing has blind spots. Anthropic's harness write-up notes Claude could not see browser-native alert dialogs through the Puppeteer MCP, and accessibility snapshots show no colour, overlap or canvas. Visual defects need pixels.
Screenshots as evidence
Screenshots work best as evidence attached to a change, not as a pixel-diff gate. Playwright's visual comparisons are real and useful, but the docs are candid that screenshots differ between browsers and platforms because of rendering and fonts. Pixel-perfect gates across machines become a flake factory.
What works better for agent loops:
Three rules make screenshots trustworthy:
- Before and after, under comparable conditions: same viewport, theme and seed data. The new state alone proves nothing about what changed.
- Whoever captures it describes it ("submit button visible at 390 px, above the footer"). The text is checkable, and it forces the capturing agent to actually look.
- Never fabricate a baseline. A missing "before" is a known gap. A reconstructed one is false evidence.
The limit: an image proves a state, not a behaviour. A screenshot of a success page does not prove the payment went through.
Evals for model-driven behaviour
When the change is to a prompt, a tool description, or an agent's instructions, unit tests stop being the right sensor because the output is not deterministic. That is what evals are for. Anthropic's Demystifying evals for AI agents is the most practical guide we know: 20 to 50 simple tasks drawn from real failures is a great start, grade the final state the agent produced where you can, and read the transcripts, because you won't know whether your graders work otherwise.
Run each case more than once, report pass rates rather than verdicts, and keep held-out cases the agent tuning the prompt never sees.
Layer four: judges, reviewers, and testing the tests
LLM judges and reviewer agents
A reviewer agent reads the diff and reports problems. It is the only automated sensor that can check rules requiring judgment ("this error message leaks an internal path," "this test doesn't test the behaviour its name claims"). It is also a sensor with known biases. The original MT-Bench study found strong judges agree with human preferences over 80% of the time, about the same rate humans agree with each other, and also documented position, verbosity and self-enhancement biases.
What makes reviewer agents useful rather than decorative:
- Fresh context, read-only tools. The reviewer sees the diff and the rules, not the author's reasoning.
- One concern per reviewer. One rule against a few hundred lines beats everything against everything.
- Structured findings (rule, file, line, evidence) that can be counted.
- A verifier pass that re-checks each finding, cutting the false positives that teach authors to ignore reviewers.
- The author cannot dismiss findings. Fix them, or escalate.
The biggest limit is independence: a reviewer from the author's model family shares its blind spots. And mechanical rules ("no magic numbers") are a poor use of a judge; they are cheaper and more reliable as lint rules.
Property-based testing
A property-based test states a rule that should hold for all inputs ("parsing then formatting returns the original," "the sorted list has the same elements") and lets a library such as Hypothesis or fast-check generate hundreds of inputs, then shrink any failure to a minimal case. It catches exactly the edge cases an agent's example-based tests skip. A 2025 paper on agentic property-based testing ran a Claude Code based agent that wrote Hypothesis tests across 100 Python packages and found real bugs.
Mutation testing
Mutation testing changes the code on purpose (flip a comparison, delete a line) and checks that some test fails. A surviving mutant is code the tests run but do not check: the most direct sensor for "the agent's tests assert nothing." On a diff it is cheap, and that is how it runs at scale: Google's mutation testing system mutates only changed code during code review and surfaces surviving mutants to the author. Meta's ACH goes a step further, using an LLM to generate both the mutants and the tests that kill them.
Layer five: human review
People are the slowest, most expensive and least consistent sensor, and the only one that can answer "is this what we meant?" Spend them on intent, design, data model, security boundaries and anything irreversible, not on formatting or rules a linter can enforce.
Make the human's job reading evidence, not gathering it. A pull request with a CI run on its head commit, described screenshots, resolved reviewer findings and a list of what was not verified takes minutes to review well. One that says "all tests pass" takes an hour, or gets approved unread.
A worked example: the signup form, again
Here is the opening change with a sensor stack chosen for it. Each row adds a sensor, and the third column shows why it was not redundant with the ones above it.
| Sensor added | What it caught or would catch | Why the earlier rows could not |
|---|---|---|
| Types and lint | The new field's value was optional in the form type but required by the API client | Baseline: always on |
| Unit tests on validation | An empty string passed validation | Types allow empty strings |
| Property test on the validator | A name made only of combining characters passed, then failed on the server | Nobody wrote that example |
| Scripted Playwright journey: fill the form, submit, land on the welcome page | Would catch the hidden submit button only if the test ran at phone width | The journey at desktop width passes |
| Before and after screenshots at 390 px, described by the capturing agent | Submit button hidden under the sticky footer | Nothing above looks at a rendered screen |
| Reviewer agent against the UI rules | The new field had no accessible label | Requires reading the markup with a rule in mind |
| Human review | Asked whether the field should exist at all on the first step | Only a person owns that decision |
The stack composes: cheap rows absorbed most attempts, so expensive rows ran once, on code that already passed. And the screenshot finding became permanent and cheap: the Playwright journey now also runs at phone width.
How to measure your sensors
| Metric | How to collect it | What it tells you |
|---|---|---|
| Flake rate per sensor | Rerun on unchanged commits; count failures on main that pass on rerun | Which sensor to fix first |
| Time to signal | Timestamp from push to result, per sensor | Which sensors are too slow for the inner loop |
| Catch rate | Seed known bugs, or use mutation score on the diff | Whether the sensor can see what you think it sees |
| Escapes by missing sensor | For each bug found after merge, record which sensor should have caught it | Where to add the next sensor |
| Cost per change | Runner minutes plus tokens per merged change | Whether a layer is earning its place |
| Judge agreement | Sample judge verdicts and have a person label them | Whether a reviewer agent can be trusted on a rule |
What this does not fix
A sensor stack cannot verify a target nobody wrote down. If the spec never says the form must work on a phone, no layer is obliged to check it, and the stack will report green with complete honesty.
Agent-driven sensors (browser agents, judges, evals) are themselves nondeterministic. Stack too many and the loop spends more time arguing with its sensors than fixing code.
And independence is never complete. The more of your stack is model-driven, the more it matters that the bottom layers are deterministic and the top one is a person.
What it costs
Browser journeys and evals cost runner minutes and tokens on every change. Reviewer agents cost tokens in proportion to diff size and rule count. Mutation testing is cheap only on a diff. Property tests take thought to write. And every sensor is one more thing that can flake, which someone has to own. The discipline is not to add every sensor to every change. It is to choose.
The playbook
Pick the row that matches the change. "Always" means the bottom of the stack (types, lint, focused tests, CI) runs regardless.
| Situation | Add these sensors | Skip or defer | Human looks at |
|---|---|---|---|
| Pure function or library logic | Unit tests, a property test, mutation on the diff | Browser, screenshots | Only if the interface changed |
| UI change | Scripted journey at desktop and phone width, before and after screenshots with descriptions, reviewer agent on UI rules | Mutation testing | Screenshots and intent |
| Bug fix | A test that fails before the fix and passes after, plus one line naming why the bug escaped | Broad exploratory runs | The root cause note |
| Refactor with no behaviour change | Full existing suite, changed-line coverage, reviewer agent for accidental behaviour change | New tests beyond coverage gaps | Only on large diffs |
| Database migration | Up, down and up again on the real engine, integration tests against the migrated schema | Screenshots | Data-loss risk and rollback plan |
| Prompt, tool description or agent instruction | Evals with repeated runs and held-out cases | Unit tests on the prompt text | Transcripts from failed eval cases |
| Security-sensitive path | Semgrep or CodeQL rules for the pattern, reviewer agent with a security rule set | Nothing | Always |
| Exploratory bug report from a user | Agent-driven browser to reproduce, then convert to a scripted test | Judges | The reproduction |
| Change to the checks themselves | Run the old and new check on the same commits; compare | Nothing | Always: this is changing the judge |
The tools will change. The principle will not: put the cheapest sensor that can see the defect closest to the agent, and save people for the questions only people can answer.
Written by CatalEx Engineering. We build the AI operating layer for AI-native companies: one platform to build, deploy, and run AI agents in production. More at catalex.co.