← Field Notes // Field Notes / 09

The Blind Spot You Can't Ask the Same Model to Find

Jeff Hay, Portable Knowledge, LLC

Every AI coding assistant I've used gets things wrong in the same general shape: it produces an answer that's internally consistent with its own training, and that consistency is exactly why it doesn't notice the mistake. Ask it to check its own work and you're asking it to apply the same assumptions a second time. It usually comes back clean.

I don't think most people doing a "second pass" with AI are actually getting a second opinion. They're asking the same model, or a different-sized model from the same company, to re-read something it already decided was fine. That's not review. That's the same judgment running twice.

Why the provider matters, not just the model

Different labs make different calls about what a model should assume by default: how cautious to be, how to handle ambiguous instructions, what "done" looks like for a task with no explicit spec, which tradeoffs to make when speed and correctness pull against each other. Those calls get baked in during training, long before anyone's prompt enters the picture.

And they don't show up anywhere you'd normally look. Not in the code. Not in the comments. Not in anything you could point to and say "there, that's the assumption."

A model from a different provider was shaped by a different set of those calls. In my experience it's far more likely to flag something the first model walked right past, not because it's smarter, but because it isn't carrying the same blind spot into the review. Two models from the same provider, even very different sizes, tend to share the family resemblance more often than they catch each other's mistakes.

This isn't actually a new idea

Pair programming has worked for the same reason for decades. Not because two people happened to look at the same code, but because two people who think differently looked at it. Pair two engineers with identical training and identical instincts and you'd gain far less, because the mistakes they share get waved through twice. The value was always in the difference, not in having a second set of eyes for its own sake.

A same-provider second pass is the AI version of pairing someone with their own reflection.

What this looks like in practice

The technique is almost too simple to call a technique: write or review the first pass with one assistant, then hand the result to a genuinely different one, from a different provider, and ask it specifically to find the problem, not to summarize or approve. I've started treating this as a cheap extra pass aimed at exactly one category of mistake: the plausible-looking answer that's wrong in a way that's invisible from inside the first model's own assumptions.

What comes back is a list of things to look at, not a verdict. Someone still has to decide which of those flags are real problems, and that someone is a person who knows what the code is supposed to do.

It doesn't replace actually understanding the code, and it won't catch everything. What it catches is a specific failure mode that a same-model second look is badly placed to find: the blind spot that comes from training, not from carelessness, the kind that re-reading your own output almost never surfaces.