The most common data leak in an AI-built app takes about ninety seconds to test. Log in as yourself, open a page that shows one of your records, look at the address bar, and change the number at the end. If someone else's order, profile, or invoice loads instead of an error message, you have found it.
We have run this same test on a lot of AI-built products by now. It works more often than it should.
None of this requires you to read a line of code. It requires a browser, an incognito window, and thirty minutes on a Saturday. This is the checklist we would run ourselves before we did anything else.
The Real Problem Is Timing, Not the Tool
An AI coding agent is not careless. It is obedient, and it is fast: the speed advantage of AI-assisted coding is real and worth having. But speed only ever answers the prompt it was given, and a demo only ever asks for the happy path: log in, click through, show the investor the feature. Nobody prompts "and make sure a stranger cannot see everyone else's data," so that part often does not get built.
That gap is invisible while the only user is you. It stops being invisible the week you have a name, a domain, and your first hundred signups. Funding and traffic tend to arrive together, and so does the first person who tries a URL that was not meant for them, sometimes on purpose, sometimes by accident. We wrote about the broader shape of this problem in why vibe-coded demos fall apart the moment they become products: the app was never wrong, it was never finished.
Security is one slice of a longer list of things that go wrong in AI-built MVPs in production, but it deserves its own weekend, because it is the fastest way to lose a customer's trust for good. A slow page frustrates people. A leaked record makes the news.
The Weekend Checklist
Work through these in order. Each one takes a few minutes, uses a private or incognito browser window, and needs nothing but the app itself.
1. Exposed storage buckets. Most apps keep uploaded files, photos, PDFs, exports, in a separate storage service, not in the main database. That storage has its own permission settings, and AI builders often leave it open by default because an open bucket never breaks a demo.
Test: Open a file you uploaded in the app (a profile photo, a document) and copy its full web address. Paste that address into a fresh incognito window before logging in. If it loads without asking you to sign in, note the address structure, then change one part of it, a number or a file name, to something nearby and reload.
Pass: A private file never opens for a logged-out visitor, and changing the number in the address never pulls up someone else's file.
2. Missing row-level permissions. This is the database-level rule that a logged-in user can only read and write their own rows, not everyone's. It is the setting behind the URL test at the top of this article.
Test: Log in as yourself, go to any page that shows one specific record (an order, an invoice, a profile), and change the ID number in the address bar to a nearby number. Try this on two or three different pages, not just one.
Pass: Every attempt returns "not found" or "access denied." None return someone else's data.
3. API keys sitting in the front-end code. Some keys are meant to be public. Others are meant to stay on the server and never leave it. When they end up in the code your browser downloads, anyone visiting the site can read them.
Test: Right-click anywhere on the page and choose "View Page Source," or open your browser's developer tools and search (Ctrl+F or Cmd+F) for the words "key," "secret," and "token."
Pass: Nothing turns up, or anything that does is clearly labeled "public" or "publishable." Anything labeled "secret," "private," or "admin" should never appear here.
4. Unauthenticated API routes. The app you see is built on top of internal web addresses that fetch and save data behind the scenes. If one of those addresses does not check whether you are logged in, anyone who has the address can call it directly.
Test: Open your browser's developer tools, click the "Network" tab, and use the app normally for a minute. You will see a list of addresses the app calls. Copy one that clearly returns real data, then paste it directly into a fresh incognito browser window while logged out.
Pass: The pasted address returns an error, a redirect, or nothing, not real data. One caveat: this browser test only catches addresses that hand back data on a plain page load. Some routes need a different kind of request that a browser bar cannot send, so a pass here is reassuring, not conclusive.
5. Password and session handling. This covers whether the app accepts weak passwords and whether logging out actually ends your session.
Test: Try to sign up with an obviously weak password like "123456." Separately, log in, then log out, then press your browser's back button and reload the page. The reload matters: without it, the browser may show you a stale copy of the page from memory, which proves nothing either way.
Pass: Weak passwords get rejected, and reloading after logout sends you to the login screen instead of showing your private data.
6. No rate limits on forms. Without a limit, nothing stops a script from hammering your login form, your signup form, or your password reset thousands of times a minute. Best case, that is spam. Worst case, it is someone guessing passwords, or running up a bill on any AI or email feature the form triggers.
Test: On the login form, deliberately enter the wrong password ten or fifteen times in under a minute.
Pass: At some point the app slows you down, locks the attempt temporarily, or asks you to prove you are not a script.
7. Default admin accounts. Templates and AI-generated starter code sometimes ship with a built-in admin or demo account meant only for development, and it is easy to forget to remove.
Test: In an incognito window, try any admin or dashboard address the app might have with obvious guesses like "admin" and "admin," or "admin" and "password." Also check your own account settings for any toggle that lets you promote yourself to an admin role.
Pass: Obvious default credentials fail, and there is no self-service path to admin access.
The Two Things You Cannot Check Yourself
Two items belong on this list but do not belong to a weekend with a browser: whether the app's underlying open-source packages have known vulnerabilities, and whether the cloud infrastructure it runs on is configured correctly (network access rules, backup settings, who can reach the database directly).
Both need a tool built for the job, not a browser tab. A dependency scan checks your codebase's package list against a public vulnerability database, and several inexpensive tools do this automatically. Some source code hosts run one for free. An infrastructure review needs someone who has read enough cloud provider dashboards to know what a wrong setting looks like.
Neither is expensive to get checked once you know to ask. Both are worth doing before your seed round closes, not after.
What We Do Differently
When we run a technical audit, we start with this exact checklist, then go further than a weekend allows. We read the server-side code that decides who can see what, run a dependency scan and a configuration review, and rank every finding by how much damage it could do and how long it takes to fix. You get a short list ordered by risk, not a report nobody reads.
What we look for lines up with what senior oversight of AI-generated code actually means in practice: someone who knows which twelve questions the model never asked itself, and who checks the answers before anything ships to a real user.
The Counterintuitive Part
The alarming fact is not that your AI-built app has security holes. Almost every app does at launch, AI-built or hand-coded, because a first version is not supposed to be finished. The dangerous moment is treating a demo's checklist as complete the week you get your first paying customer, exactly when it stops being optional.
The good news that follows from this: most of what is on this list is a configuration setting, not a redesign. Fixing a missing permission rule or a public storage bucket is usually hours of work, not weeks. If the weekend check turns up more than a few of these, that is a reason to prioritize, not a reason to consider a rewrite. Our framework for deciding whether to rewrite or keep a vibe-coded product covers how to tell the difference, and a stack of security findings by itself rarely points to "rewrite."
What to Do Tomorrow
-
Run all seven checks this weekend, in order, and write down what fails. You do not need to fix anything yet, only find it.
-
Sort your findings by exposure, not by how easy each one looks to fix. A public storage bucket full of user documents outranks a weak password rule every time.
-
Get the two you cannot check yourself scheduled, the dependency scan and the infrastructure review, before you tell anyone the app is ready for real users.
Frequently Asked Questions
How do I know if my AI-built app has a security problem?
Run the checklist above. If any test fails, you have a confirmed problem, not a suspicion. Most AI-built apps fail at least one of these tests the first time someone checks, so finding something is normal, not a verdict on the tool you used.
Do I need to hire a developer to fix these issues?
For most of the seven checklist items, yes, though the fix is usually a configuration change rather than new code, often a few hours of a competent engineer's time per item. The two items that need scanning tools, dependency and infrastructure, are worth getting a second pair of eyes on either way.
Is it normal for an AI-built app to have security holes at launch?
Yes. A demo is built to prove an idea works, not to withstand a stranger trying to break it. The problem is not that the holes exist. It is shipping to real users without ever running a checklist like this one first.
How long does it take to fix a typical security hole from this checklist?
Most items on this list, missing row-level permissions, an exposed bucket, a missing rate limit, take a competent engineer somewhere between an hour and a day once the problem is confirmed. The scanning-based items take longer to review than to fix, since most fixes are also configuration changes.
If your weekend check turned up more than you expected, schedule a free diagnostic call and see what a full technical audit and fractional CTO engagement would find.