← Field Notes // Field Notes / 06

Testing at Every Level Catches a Different Kind of AI Mistake

Jeff Hay, Portable Knowledge, LLC

"We have good test coverage" is a sentence that hides a lot. Coverage of what, at what level? A codebase can have thorough unit tests and still ship a system that's badly broken, because the failure was never inside any one function, it was in how two correctly-behaving functions interacted. Testing isn't one activity. It's several, at different levels, and each level exists because it catches a class of bug the others structurally can't.

The three levels, and what each one is actually for

Unit tests check a single function or module in isolation: given this input, is the output correct. They're fast, cheap to write, and they catch logic errors close to where they happen. They're also the level most AI coding assistants are naturally good at satisfying, because a unit test's contract is usually visible in the same context window as the code being generated.

Integration tests check that components behave correctly together: does this module's output actually match what the next module assumes it's receiving, does shared state stay consistent, does the ordering of operations hold under real conditions. This is where an AI-generated change is most likely to go wrong in a way nobody catches quickly, not because the code inside the changed function is wrong, but because the agent making the change didn't have, and often couldn't have had, full visibility into every other part of the system that depended on an assumption it just altered.

System, or end-to-end, tests check the whole thing against what it's actually supposed to do in the world: does the safety interlock still fire in time, does the system hold its real-time performance guarantee under load, does the pipeline still produce a correct result end to end. Individually correct components can still add up to a system that fails here, and this is usually the most expensive level to test and the first one teams skip under time pressure.

Two domains where "we can't mess this up" means very different things

I've worked at the highest-stakes end of two different failure modes, and understand that the discipline required in each case looks a little different.

FDA-regulated medical imaging software. The stakes were regulatory and clinical: a full-lifecycle requirements-to-test traceability process, where "we tested it and it seemed fine" isn't an acceptable standard and every test has to trace back to a documented requirement. The failure mode there is auditable and procedural, you can trace exactly what went wrong and why the process should have caught it.

Precision semiconductor manufacturing research instrumentation. The stakes were physical and irreversible: a control system moving sub-micron positioning equipment around high-value material, where a single mistake in real-time behavior could destroy an expensive wafer or an experimental run that took weeks to set up (no audit trail undoes that). The discipline that prevents it looks less like paperwork and more like hardware-in-the-loop validation and integration-level testing of the actual timing behavior under real conditions, because the bug that matters most is rarely inside one function, it's in how the control loop, the sensor feedback, and the positioning hardware interact in real time.

Different domains, different failure modes, same underlying answer: know which level of testing actually catches the failure you're afraid of, and don't assume unit-level coverage protects you from a system-level failure just because the word "tested" applies to both.

Why this matters more with AI in the loop

A human-paced team, working in a single codebase over years, tends to develop an intuition for the integration-level risks, the developer who remembers "oh, that module is fussy about ordering" and reviews accordingly. An AI coding assistant, working on one piece of a system inside a limited context window, doesn't inherit that intuition. The class of bug it's most likely to introduce is exactly the class that unit tests structurally cannot catch. And integration and system-level tests are the layers teams are most likely to have under-invested in, because they're the hardest and most expensive to build.

That's the actual argument for investing there now, not more testing everywhere uniformly, but testing at the level that matches the specific way your development process has changed.