Stop Reading AI-Generated Code. Start Verifying It.
There is a difference. It matters more than you think.
Somewhere in your codebase right now, there is code an AI agent wrote. Maybe you reviewed it carefully. Maybe you skimmed it. Maybe you approved the pull request because the tests were green and you had three other things open.
This is not a problem to be solved by reading faster. It is a problem to be solved differently.
The question worth asking is not “did I read this code?” The question is: “do I have sufficient evidence that this code is correct?” Reading is one way to gather that evidence. It is not the only way, and for AI-generated code at any meaningful scale, it cannot be the primary one.
The Distinction That Changes Everything#
Reviewing code means reading it. You trace the logic, spot the edge cases, ask whether this is the right approach. It is slow, expert-dependent, and does not scale.
Verifying code means confirming it is correct, by whatever means available. Review is one path to verification. Machine-enforceable constraints are another. At sufficient scale, the second path is the only viable one.
This reframing is not an excuse to be lazy. It is an invitation to be more rigorous: to replace the informal, inconsistent process of “a human skimmed this” with a formal, repeatable process of “this code passed a defined set of constraints.”
What Good Constraints Look Like#
The goal is to define a space of valid programs so precisely that anything outside it cannot pass, and anything inside it is almost certainly correct. Four types of constraints do most of the work.
Property-based tests#
Standard unit tests check specific cases: given input 15, expect “FizzBuzz.” This is useful but limited. A property-based test asks a harder question: does this property hold for all valid inputs?
You write the property. The testing library (Hypothesis for Python, fast-check for JavaScript, QuickCheck for the Haskell family) generates hundreds of inputs automatically, favoring edge cases: zero, negative numbers, very large values, boundary conditions. If the property holds across all of them, you have meaningful confidence that it holds in general.
This constrains the solution space toward correctness. It ensures the requirements are met.
Mutation testing#
Here the direction reverses. Instead of asking “does the code satisfy the tests,” you ask “do the tests actually test anything?”
Mutation testing tools make small, deliberate changes to your code: swap > for >=, flip a boolean, change a constant. Then they re-run your test suite. If the tests still pass after a change that should break something, the tests are not doing their job.
Used in the usual way, mutation testing helps you improve your test suite. But there is a second use: if you have a strong test suite and all mutations are killed, you can invert the logic. Any code that passes these tests must be doing exactly what the tests describe, nothing more. The mutation score becomes a measure of how constrained the valid solution space is.
Side-effect isolation#
A function that only transforms inputs into outputs is a function you can verify in isolation. A function that writes to a database, calls an API, or modifies global state is a function whose correctness depends on the state of the world at runtime.
Requiring pure functions where possible is not just good software design. It is a verification strategy. A pure function can be tested exhaustively. A side-effectful function cannot.
Static analysis#
Type checking, linting, and similar tools catch a category of errors before anything runs. In Python, mypy or pyright. In TypeScript, the compiler itself. These are table stakes, not the interesting part, but they eliminate a class of bugs that would otherwise require dynamic testing to surface.
Together, these four constraints define a small, well-lit region of possible programs. The agent must land inside it. Most invalid programs cannot.
Going Further: Validation at Scale#
The four constraints above work well for individual functions. When agents are generating entire services, the validation architecture needs to grow with them.
Formal contracts#
Contracts are preconditions and postconditions expressed in the code itself. A function with a contract says: if you call me with valid input, I will return valid output, and here is precisely what “valid” means for both.
Libraries like deal for Python, spec for Clojure, or the type system in Rust make this explicit. The agent cannot produce a function that violates its own declared contract. Contracts can be checked statically (before anything runs) or at runtime (as a continuous assertion). Either way, they narrow the space of valid programs more precisely than tests alone.
Sandboxed execution#
Side-effect isolation can be enforced at the test level. But for production agents generating code that runs automatically, you want structural enforcement: a sandbox where the generated code physically cannot reach the network, the filesystem, or external services.
Firecracker microVMs, WebAssembly runtimes, and seccomp-filtered containers all do this at the OS level. The question changes from “did the code try to make an external call?” to “the code cannot make an external call, so we don’t need to ask.” This is not just correctness verification: it is containment. For autonomous systems, containment matters as much as correctness.
Differential testing#
If you have an existing implementation of the thing the agent is replacing, you have something valuable: a reference. Run both implementations in parallel across a large sample of real or synthetic inputs. Compare the outputs. Where they agree, the new code is almost certainly correct. Where they diverge, you have a precise failure case to examine.
This approach scales well and requires no additional test-writing. The reference implementation does the work. The old code, the thing you were trying to improve or replace, becomes your verification oracle.
Schema validation at boundaries#
Any time generated code produces structured output (JSON responses, database records, message payloads), a schema validator at the boundary is a low-cost, high-value check. Not “is this valid JSON” but “does this JSON have the right shape”: the required fields, the expected types, the value ranges that downstream consumers depend on.
Pydantic, Zod, and JSON Schema are all mature options. An agent cannot silently change an API response shape if a schema validator is standing at the door. Regressions of this class, common and annoying, are caught automatically with almost no engineering cost.
Semantic testing for decision-making code#
When generated code makes judgements (classifies inputs, extracts structured information, routes requests) unit tests may not be the right tool. The correct output for a given input is not always a single deterministic value.
For these cases, a labeled evaluation set works better. You assemble a representative sample of inputs where you know the correct answer, run the generated code against it, and measure accuracy. A threshold below which the code fails. This is how machine learning models are evaluated, and it applies equally well to any code that sits at the fuzzy boundary between computation and judgement.
Agent chains as verification infrastructure#
When one agent generates code and another agent tests it, and a third reviews the test coverage, and a fourth checks for type errors, the chain is itself a verification structure. Each agent certifies a specific property. The code must pass every certification before it is accepted.
This is a natural architecture for teams already running multiple agents. The key discipline is making the certifications explicit: not “agent B approved this” but “agent B confirmed that all property-based tests pass and mutation score exceeds 90%.” Explicit certifications are auditable. Vague approvals are not.
What We Are Actually Giving Up#
There is something the old model of code review provided that this new model does not: shared understanding. When a human reads code, they build a mental model of what it does. They carry that model into future debugging sessions, architecture decisions, conversations about whether to change the code.
AI-generated code, verified by machine constraints, produces no such shared understanding. The team knows the code is correct. Nobody knows why it works, or how, or what it would mean to change it.
This is a real cost. It is worth acknowledging rather than pretending it is not there.
The honest answer is that it may be acceptable for certain categories of code: isolated utility functions, data transformations, well-bounded integrations. The same way teams accept that compiled code is a black box, we may need to accept that some generated code is a black box too, provided the box is well-sealed and well-tested.
For core business logic, architectural decisions, anything that needs to be understood to be maintained: human review remains the right tool. The goal is not to eliminate review but to reserve it for the code that actually needs it.
The Practical Path#
You do not need to implement all of this at once. A sensible progression:
Start with property-based tests. Pick your three most critical functions and write property-based tests for them. See how many edge cases they surface that your unit tests missed. The answer will be instructive.
Audit your test suite with mutation testing. Before trusting your tests to verify agent output, find out whether they are actually testing anything. A mutation score will tell you quickly. Fix the gaps.
Enforce purity where you can. Any function that could be pure, should be. Document the exceptions. Make side effects visible and intentional rather than incidental.
Put schema validators at every output boundary. Every API response, every database write, every message payload. This takes an afternoon to set up and pays dividends forever.
Build toward agent chains. As you add agents to your workflow, give each one a specific certification responsibility. Explicit certifications that accumulate into an audit trail.
The overhead is real. For a single short function, reading it is faster than building this infrastructure. The infrastructure makes sense when you are not looking at one function. You are looking at a thousand, generated by agents, arriving faster than any team can review them.
The question then is not how to read faster. It is how to know, without reading, that what arrived is correct.
Enfin.