The founder had the pitch down cold. "We built the whole thing ourselves. No CTO, just AI tools and about four months." The demo backed it up: sign up, dashboard, billing, a clean onboarding flow. He had already shown it to two investors, and both had asked good questions he could answer.
We asked for read access to the codebase and the database before the call ended.
A note before we go further: the company below is not one company. It is a composite, drawn from several audits we have run, folded into a single narrative so we can walk through what an audit actually does without hanging it on one client. Nothing here identifies a real founder, product, or investor. The findings are generic to the category. The sequence is real, and that is the part worth reading closely.
This is the walkthrough, not the sales version: what the founder believed he had, what the first hour turned up, what a full pass found, the order we fixed things in, and what was different a few weeks later that had nothing to do with a chart.
What the Founder Believed He Had
His mental model was reasonable, given what he had watched happen. Users could sign up, log in, and stay logged in. They could pay. There was an admin screen where he could see who had signed up and refund a charge by hand. The AI tool had built all of it in response to plain-English requests, and every time he tested it, it worked.
"It's basically done," he told us. "We just need to polish it before the round closes."
A few hundred people were on a waitlist, with more expected once the raise was public. He was not being careless. He had tested the app the only way he knew how: by using it the way it was meant to be used.
The Real Problem: A Demo and an Audit Ask Different Questions
A demo answers one question: does this work when I click through it the way I'm supposed to? An AI coding tool is very good at making that answer yes, because that is the loop it optimizes against. Describe a screen, it builds the screen, you click through it, it works, you move on.
An audit asks a different question: what happens when someone does not click through it the way they're supposed to? A user who edits the URL. A request that arrives twice. A field that gets ten thousand characters instead of ten. Someone who is not you, using credentials that are not theirs.
Nobody had asked that second question of this product, because nothing in the build loop asks it on its own. That gap, not any specific bug, is what we look for first, and it is the same reason we tell founders to hire professionals, not amateurs for the review itself. We wrote more about why a working demo and a working product are two different engineering problems in our piece on vibe coding and the first round.
The First Hour
The first hour of an audit is cheap and fast on purpose, because the worst problems are usually the easiest to find. We ran the same checks any founder can run with a browser and no code: log in as one test account, then edit the URL or the request to reference a second account's data.
It took under two minutes to load another test user's invoice by changing one number in the address bar. Ten minutes later, the browser's network tab showed a live secret key for the payment processor sitting in a request payload, visible to anyone who opened developer tools. The admin screen used the same login as any other user; it was never hidden behind a role check, only left off the main menu.
None of this required reading a line of source code. It is close to the same list we cover in the weekend security check we recommend founders run themselves, worth doing before you pay anyone to look deeper.
The Findings, Ranked by Severity
Once the first hour flagged the loud problems, we went into the codebase and the database for a full pass, ranked by damage, not by how long each took to find:
No permission model. Any logged-in user could reach any other user's records by changing an identifier in the request. The app checked whether you were logged in, never whether the thing you asked for belonged to you. Fix: server-side ownership checks on every endpoint, not just the ones that seemed sensitive.
Secrets shipped to the browser. Live keys for the payment processor and the email service were both reachable from client-side code. Fix: rotate every exposed key the same day, then move secret-holding calls behind a server-side layer.
No migration history. The schema had been hand-edited in the database console dozens of times, and nobody, including the founder, could say with confidence what it actually was. Fix: reverse-engineer the live schema into a first migration and treat it as the new baseline.
Sessions that never expired, and a password reset link usable more than once. Ordinary account-hygiene gaps, common in AI-built auth flows because the tool implements the feature you asked for and stops there. Fix: expire sessions, invalidate reset tokens after one use, rate-limit login attempts.
Zero tests. Every deploy was a manual click-through of a fixed list of screens, done by the founder, about twenty minutes each time. Fix: tests for the paths that had already broken once, a floor, not full coverage on day one.
A dashboard query loading every record for every user into the browser, then filtering client-side. Fine with a handful of test accounts, due to fall over well before real users arrived at any scale. Fix: paginate and filter on the server.
Full vendor lock to one AI app-builder's hosted runtime, with no code that could run anywhere else. Not urgent, but worth knowing before it becomes a negotiating weakness or a blocker to switching hosts.
The Fix Sequence
We did not fix these in the order we found them. We fixed them in the order that exposure, divided by effort, was highest.
Same day: rotate every exposed key, the one finding where delay is pure risk. Week one: lock down the permission model, the largest exposure, done first and alone, bundled with rebuilding the dashboard query to paginate server-side while the engineer was already in those code paths. Week two: reconstruct the schema into a migration baseline, a prerequisite for touching the schema safely again, run alongside session and reset-token hygiene. Ongoing, as each fix landed: tests for the paths just touched. Deferred and flagged, not blocking: the vendor lock-in, filed as a known issue with a plan attached.
Roughly, the first two weeks of fixes ran close to eighty hours of senior engineering time. At our rate of $199 an hour, that is roughly sixteen thousand dollars, paid once, deliberately, instead of paid later in increments: a user who notices, a platform that suspends an account over a leaked key, a schema change that breaks in production because nobody knew what the schema was.
What Changed
Nothing above produces a clean before-and-after chart, and we are not going to invent one. What changed was observable, not numeric.
The founder could open the admin panel in front of an investor's technical advisor without steering the conversation away from it. A contract engineer could be onboarded with a migration history to read, instead of a live walkthrough of undocumented quirks. The support inbox stopped getting the occasional message that started with "this is probably nothing, but I think I saw someone else's." And the founder started asking "what's the permission story on this" about every new feature before it shipped, instead of after.
That last change matters more than any individual fix. The fixes closed the gaps we found. The habit closes the gaps we did not.
The Counterintuitive Part
The instinct after a list like this is to burn the product down and start over. Almost every founder asks. Almost never is it the right answer.
None of these fixes required a rewrite. Each one was a repair to something that was, underneath the gaps, a reasonably shaped application. The AI tool had built the features correctly. It had simply never been asked, and never asks itself, whether those features were safe to expose to a stranger with a browser and bad intentions. That is a different job, and it is the job an audit does. We laid out the fuller decision framework for when a rewrite actually is the right call in our piece on rewriting versus keeping a vibe-coded product, and this app, like most, landed on "keep and harden."
What to Do Tomorrow
-
Run the weekend check yourself, before you pay anyone. Log in as one test account, edit a URL or request to reference a second account, and see what comes back.
-
List every place your app touches money or personal data, and for each one, ask who besides the intended user can currently reach it. If the honest answer is "we don't know," that is the finding.
-
Get a senior technical review before an investor's advisor becomes the first person to find these gaps. A read on what senior oversight of AI-generated code actually means is a reasonable place to start.
Frequently Asked Questions
How long does a technical audit like this actually take? The first hour surfaces the loudest problems using nothing but a browser. A full pass, findings and fixes included, usually runs about a week of senior engineering time, stretching to two if the schema has to be reconstructed first.
Does an audit mean we have to rewrite the app? Rarely. Most AI-built products fail on the same handful of checkable things: permissions, secrets, schema history, tests. Those are repairs, not reasons to start over.
What if we can't afford to fix everything the audit finds at once? You usually cannot, and you do not need to. Fix in order of exposure divided by effort: the finding that does the most damage for the least work goes first.
Do we have to tell investors what an audit found? Disclosing known issues alongside a plan to fix them tends to land better than staying quiet and hoping nobody asks. Starting with "here's what we found and here's the order we're fixing it in" beats the advisor finding it first.
If you want this same walkthrough run on your own product, before an investor's advisor finds it first, book a diagnostic call or read more about our Fractional CTO & Technical Advisory work.