A senior engineer opens a pull request an AI coding agent wrote overnight. Two hundred lines, clean formatting, sensible names, a working demo attached. It reads like the best junior engineer you ever managed cleaned up their desk before quitting.
Nothing about it is wrong, exactly. But there is a database call with no retry logic inside a payment flow, a permission check that only runs on the happy path, and a migration that will lock a busy table for several minutes once the row count grows. None of these are bugs a linter catches. None show up in the demo. All three surface in the first month with real users.
This is the part of AI-assisted development nobody puts in the pitch deck: someone still has to catch what the model was never going to ask about. We call that person the senior reviewer, and the phrase "a senior engineer reviews it" gets used so loosely in this market that it has stopped meaning anything. Reviews it for what? For how long? What gets refused?
This article is our answer to those three questions, and roughly what a fractional CTO does inside an AI-augmented build.
The Real Problem: The Model Never Asks What It Doesn't Know to Ask
AI-generated code is confident, plausible, locally correct, and often globally wrong. The model optimizes for "does this satisfy the request," not "does this survive real users, real load, real attackers, and real mistakes." A human engineer writing the same feature wonders, without being told, what happens on the failed call, because they have been burned by that before. A model has read about being burned. It has not been burned, so it answers the question asked and stops, and the questions that matter most are the ones nobody asked.
That is why the failures we see in AI-built products cluster in the same handful of places every time: authentication that exists only on the front end, a permission model nobody designed on purpose, error handling that assumes the happy path and nothing else. The code is not sloppy. It is thorough about the wrong scope, and that is harder to catch than a typo.
The Twelve Questions the Model Never Asks
This is the actual checklist. It is not "read every line." It is a set of questions we ask about every non-trivial change before it ships, regardless of how clean the diff looks.
Who owns this data, and who is allowed to see it? A model builds the query that returns the record you asked for. It does not stop to ask whether the requester should see every matching record, or just their own. That distinction is the difference between a feature and a data breach, and it is invisible in a demo with one test user.
What happens when this fails halfway through? Real operations fail midway: a network call times out, a payment provider is slow, a process gets killed mid-write. The model writes the path where everything succeeds, because that is the path in the prompt. Nobody asked what state the system is left in when step two of three fails.
Where is the security boundary, and did we actually draw it? Every feature has an edge where "logged in" is supposed to become "authorized to do this specific thing." Models frequently skip that second check, especially on internal-looking endpoints. It is the finding we see most often in an AI-built app's first security pass.
Does this behave correctly if it runs twice? Retries, double clicks, and duplicate webhook deliveries are normal, not edge cases. Code that charges a customer twice or creates two accounts because it was called twice is code nobody asked to be idempotent.
What does this cost per call, and does that cost scale with usage? A model will happily wire a feature to call an external API, or itself, on every request. It has no visibility into your unit economics and no reason to flag that the pattern is fine at ten users and expensive at ten thousand.
How would we know this broke before a customer tells us? Logging and alerting are not part of "make this feature work." They are part of "know when this feature stops working," a different instruction the model was never given. Silence is the default, and it looks identical to success until it isn't.
What happens when two users do this at the same time? Concurrency bugs do not show up when one person is testing. They show up in week three, intermittently, and they are expensive to diagnose because the code was never written with the question in mind.
What is the rollback if this migration or deploy goes wrong? A model can write a schema change. It rarely writes the plan for undoing it under load, with data already written against the new shape. Forward-only thinking is fine until the first bad deploy.
Does this need to exist at all? The model will build whatever you ask for, competently. It has no incentive to push back and ask if there is a simpler, or no, way to solve the underlying problem. Scope discipline is a human judgment call, not a generation task.
What is the blast radius if this key or credential leaks? Secrets end up in client bundles, log lines, and error messages more often than anyone expects, because a working feature does not care where the key sits.
Who maintains this in six months without asking the model again? Code that only makes sense with the original prompt and conversation attached is a liability, however well it reads today. We ask whether an engineer joining next quarter could understand the intent from the code alone.
What did the model quietly decide not to build? Tests, structured logging, input validation, and sane error messages are the first things a model skips when the prompt does not mention them, because skipping them still produces something that runs. The five-step loop we use on small teams exists to make these omissions visible before merge, not after.
What Gets Blocked, Every Time
Most findings from these twelve questions get fixed and shipped the same day. A short list does not get a conversation, it gets a block: a secret readable from the browser, an endpoint with no permission check at all, a schema change with no rollback plan, or any code touching authentication or payment that has zero tests. If a build has enough of these piling up at once, the conversation stops being about one pull request. It becomes a question about whether the foundation underneath it is worth keeping.
What This Costs in Senior Time
Founders ask this before almost anything else, so here are our own working numbers, not a formula and not a claim about anyone else's practice.
For a normal feature, one that touches one or two parts of the product and does not cross a data or security boundary, we plan on roughly two to five hours of senior review time: reading the diff against the twelve questions, testing the failure paths by hand, and writing back what needs to change. Anything touching authentication, payments, permissions, or the data model goes up to six to ten hours, because those are the categories where a missed question is expensive to unwind later.
Across two weeks on a small AI-augmented team shipping continuously, that adds up to roughly ten to twenty hours of senior time, which is the shape of our standing fractional engagement, not the forty or fifty hours a week a traditional review process might consume, and nowhere near the zero that "the AI wrote it, it works" implies. The speed gain from AI tooling is real, but it shows up in how fast a feature gets built, not in how much senior judgment it still needs before it ships.
The Counterintuitive Part
The value of senior oversight is not mostly in catching bugs. It is in saying no. A reviewer who reads every line looking for defects will find defects, but the highest-leverage move most weeks is refusing to build something, or cutting a feature to a third of its original scope, before a line gets written. The model cannot do that: it was never asked to have an opinion about whether the feature should exist. That refusal, more than any line-by-line catch, is what "senior" is actually buying you.
What to Do Tomorrow
- Pick your last three merged AI-generated features and run the twelve questions against them. You will likely find at least one item from the "blocked, every time" list. Fix it this week, before it becomes someone else's incident.
- Assign one named person as the senior reviewer for every AI-assisted change, even if that person is a fractional hire for four hours a week. "The team reviews it" means nobody does.
- Write the twelve questions down somewhere your whole team can see them. A checklist everyone can reference outperforms a standard that lives in one person's head.
Frequently Asked Questions
Do I need a senior engineer to review AI-generated code line by line? No. Line-by-line reading catches typos, not the failure modes that hurt startups. The twelve questions above target where AI-generated code predictably breaks: data ownership, failure handling, security boundaries, and cost, which is a faster and more reliable use of senior time.
How is this different from a normal code review? A normal review mostly checks whether code matches team conventions and does what the ticket asked. AI-generated code usually does exactly what was asked, cleanly. The review has to look past that and ask what was never asked, a different exercise entirely.
Can I run any of this myself if I am not technical? Some of it. A non-technical founder can watch a demo and ask "what happens if I do this twice" or "who else can see this record" and get a useful answer. The deeper questions, like whether a migration is reversible, need someone who can read the code.
What if the AI-built app is already in production with users? Run the twelve questions against the highest-traffic and highest-sensitivity paths first: login, payments, and anywhere personal data is read or written. That is also the fastest way to see whether you have a tuning problem or a foundation problem.
Want a senior engineer running these twelve questions against your build every week? See how our AI-augmented build with senior oversight works, or book a free diagnostic call.