Three weeks after launch, a founder calls us. The product works. People signed up. Then a user figured out that changing a number in the browser's address bar showed them someone else's invoices.
That call is not unusual. We get some version of it every few months: a product that looked finished at demo day, and a hole nobody thought to check for. The scene above is a composite of several engagements, not one company. The pattern behind it is not.
After reviewing enough AI-built codebases, we see the same seven problems, in close to the same order, every time. That is the good part: a problem that repeats is one you can check for on purpose, before it costs you a launch or a customer.
Here are the seven, how to spot each one yourself, and roughly what it costs to fix.
The Real Problem: Nobody Asked It to Think About Production
An AI coding assistant answers the question you actually asked. If you asked for a signup form, you got one. What you almost never asked was: "and make sure no other customer can see this by changing the URL." That is the difference between a demo and a product, and a demo is what most AI-assisted builds optimize for by default.
This is not an argument against building with AI tools. It is an argument for knowing where the gap sits, so you can close it before a stranger finds it for you. The mechanism is new. Shipping what looks done instead of what has been checked is one of the oldest reasons projects fail.
The Seven Failure Modes
1. Authentication that exists on the front end only
The "admin only" button is hidden from regular users, but the underlying API endpoint never checks who is asking, only the interface does.
Check: Find the request the hidden button would trigger and replay it while logged in as a normal user. If it succeeds, your authorization lives in the wrong layer.
Fix: Move every check to the server, on every request, not just the ones your interface exposes.
2. No permission model between users
This is the invoice example above, and it is the single most common issue we find. The database has no concept of "this row belongs to this account," so any authenticated user can, in principle, reach any other user's data by changing an ID.
A security review of apps built on a popular AI app-building platform found that a meaningful number of them exposed user data this way, because the database's access permissions had simply never been configured. It is a default, not an edge case.
Check: Log in as two test accounts, note an ID that belongs to account A, then try it while logged in as account B. Our weekend security checklist walks through this and several other tests a non-engineer can run alone.
Fix: Add access rules at the database or application layer that check the authenticated user against the record requested, on every table holding customer data.
3. Secrets committed to the repository or shipped to the browser
API keys and database passwords end up hardcoded, or sent to the browser as part of the front-end code, where anyone can read them by viewing the page source.
Check: Search your codebase, then your deployed site's page source and network tab, for the words "key," "secret," and "password."
Fix: Move every secret to server-only environment variables. Rotate any key that was ever exposed.
4. A schema that cannot be migrated
The database was built by hand through a visual dashboard, with no record of what changed or when, and every future change is a manual, unrepeatable act.
Check: Ask how the last ten database changes were made. If clicking through a dashboard is the answer, there is no history to inherit.
Fix: Introduce a migration tool, write the current schema as the first migration, and require future changes to go through it. This often decides whether a rewrite or a repair is the right call.
5. Performance that collapses well before real usage
The app feels instant with the ten test records used to build it, then falls over at a few hundred real users, usually from queries that fetch rows one at a time in a loop, missing indexes, or no caching anywhere.
Check: Load your busiest page with a few thousand rows of test data instead of ten, and watch how many queries fire.
Fix: Index the columns your queries filter and join on most. Batch the queries currently run one at a time. Cache what does not need recalculating on every request.
6. Zero tests, so every change is a gamble
Nothing tells you what broke when something changes. Every fix risks silently breaking something that used to work, and nobody finds out until a customer does.
Check: Change one file. Is anything automated reporting whether the rest of the product still works.
Fix: Skip full coverage on day one. Write tests around the paths that would hurt the business most if they broke silently: payments, authentication, and data writes.
7. Lock-in to a single AI platform's runtime
The code only runs inside the tool that generated it, with no plain, exportable codebase you could hand to an engineer or move to a different host.
Check: Ask whether the code can be exported in full and run on a plain server, outside the platform that built it.
Fix: Move to a standard language and framework in your own repository as early as possible. The longer you wait, the more it costs, a point we cover when we walk through what senior oversight of an AI build actually looks like.
What Each One Costs to Fix
The ranges below assume a typical seed-stage MVP: one codebase, one primary framework, under fifty thousand lines of code, and an engineer who did not write the original code but already knows the stack. A larger or more tangled codebase costs more at every row.
| Failure | How to spot it | Typical repair effort (hours) | Who can do it | |---|---|---|---| | Front-end-only authentication | Replay a hidden action's API call while logged in as a normal user | 4 to 8 | A mid-level engineer | | No permission model between users | Log in as two accounts, swap an ID between them | 8 to 20 | A mid-level to senior engineer | | Secrets in the repo or the browser | Search the codebase and deployed page source for keys | 2 to 6, plus rotating every exposed key | Any engineer with server access | | Schema that cannot be migrated | Ask how the last ten schema changes were made | 10 to 30 | A senior engineer familiar with the database | | Performance collapse under real load | Load the busiest page with a few thousand test rows | 8 to 25 | A senior engineer who can read a query plan | | Zero tests | Change one file, see if anything reports what broke | 20 to 40, for the highest-risk paths only | A mid-level to senior engineer | | Vendor lock-in to one AI platform | Ask if the code can be exported and run elsewhere | 40 to 120 or more, sometimes a rewrite | A senior engineer or a small team |
The Playbook We Use
When a founder asks us to look at an AI-built product, we run the same seven checks above, in order, on the running product and codebase, not a description of it: a full codebase read, the two-account test, realistic data volumes on the busiest pages, a secrets search, and a look at the schema's history and the test suite, or lack of one.
What comes out is a written list, ranked by risk and by hours to fix, not a vague sense that "the code needs work." That list is the starting point of the Fractional CTO & Technical Advisory work we do with early-stage teams: senior judgment applied to a build that was mostly or entirely AI-assisted, so the founder does not have to guess which problems matter.
The Counterintuitive Part
Founders often treat this list as evidence they made a mistake building with AI tools. It usually is not.
Every one of these seven problems is fixable in days, not months, when caught early. The expensive part was never the fix. It is the delay: months of the product running with the gap open, accumulating data and integrations on top of it, until the fix has to work around everything built since.
The founders who avoid the expensive version of this story are not the ones who wrote every line of code by hand. They are the ones who had someone check for these seven things early.
What to Do Tomorrow
-
Run the two-account test. Log in with two accounts, note an ID that belongs to one, and try it while logged in as the other. Ten minutes, and it catches the most common problem on this list.
-
Search your codebase and deployed site for "key" and "secret." Check the network tab too while you use the product normally.
-
Ask how the last database change was made. A dashboard click rather than a versioned script means you have failure four, and now you know it.
Frequently Asked Questions
How do I know if my AI-built app has these problems? Most of the checks above take ten minutes to an hour and need no coding: a second test account, a browser's developer tools, and a plain question to whoever built it. Start with the two-account test, it finds the most common and most damaging issue here.
Can I fix these myself without hiring an engineer? Some of it, no. Moving secrets and adding a permission model both touch how data is stored and accessed, and getting them wrong leaves the same hole in a different shape. Finding the problems does not require an engineer. Fixing the data-access ones reliably does.
Do I need to rewrite the whole app to fix this? Almost never. Six of the seven failures are targeted fixes measured in single-digit to low double-digit hours. The exception is deep lock-in to one AI platform's runtime, which sometimes means a move to a standard framework. Our rewrite versus repair framework covers that call.
How much does it cost to fix an AI-built MVP? It depends on which problems are present and how large the codebase has grown around them. Caught early, a product can often clear this list in under a week of senior engineering time. Caught a year later, it costs meaningfully more.
Want these seven checked properly, on your actual product? Our Fractional CTO & Technical Advisory work starts with exactly this list. Schedule a diagnostic call and we will tell you what we find.