How to check if your AI-built app is secure

Why AI-built apps need a separate check

AI coding tools are very good at making an app work and much less careful about who is allowed to use it. The app looks finished in the preview because you are always logged in as yourself. The holes only show when someone else, or a script, uses it in a way you did not try.

In our own sample of 24 public apps built with AI tools, 16 had at least one high or critical problem. The most common were server functions anyone could call and database tables anyone could read.

1. The two-account test (data access)

Create two accounts in two different browsers. With account A, create something private: a note, an order, a profile. With account B, open the browser tools, find the request that loads data, and change the id to A's id. If B sees A's data, your database rules are missing or too loose.

On Supabase this almost always means row level security is off for that table, or a policy uses USING (true). On Firebase it means the security rules allow reads without comparing request.auth.uid to the document owner.

2. Search the built app for secret keys

Open your live site, then the browser tools, and search all loaded JavaScript (Sources, then search across files) for these strings. Any hit is a key a stranger can copy:

• sk_live_ or rk_live_ (Stripe secret or restricted key)
• sk-proj- or sk-ant- (OpenAI or Anthropic API key)
• service_role (a Supabase key that ignores all row level security)
• Any variable named SECRET, PRIVATE or ADMIN that has a value

3. Call your server functions without logging in

Log out, then call each API route or Edge Function directly. If it still does its job (sends an email, calls the AI, changes a record), anyone on the internet can do it, millions of times. A Supabase Edge Function with verify_jwt on still accepts the public anon key, so it must check the user in its own code.

curl -X POST https://YOUR-PROJECT.supabase.co/functions/v1/chat \
  -H "Authorization: Bearer YOUR_PUBLIC_ANON_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prompt":"hello"}'
# If this answers, the function does not check who is calling.

4. Try to get the paid plan for free

Look at how your app decides someone has paid. If the front end writes is_pro = true or plan = "pro" to the database after checkout, any user can write the same value. Payment status must only be set by a verified webhook from Stripe or your payment provider, and users must not be able to edit that column.

5. Send 50 requests in a row

Anything that costs you money per use (AI answers, emails, SMS, image generation) needs a login and a per-user limit. Send the same request 50 times from a script. If all 50 go through, one bored visitor can run up your bill overnight.

What these checks cannot tell you

Manual checks find the obvious holes. They will not find a policy that is wrong on only one table, a database function that skips the owner check, or a webhook that trusts its payload. Reading the code is the only way to see those. That is what Redproof does: it reads your repository, runs more than 40 checks in about a minute, and a developer confirms anything that is not a proven fact.

Questions

Is the built-in security scan in Lovable enough?

It is a good first step and catches common Supabase problems. Lovable itself says its scan does not catch everything, and it only covers apps built in Lovable.

Do I need a penetration test before launch?

Not for a small first launch. Fix the five problems above first; they are what automated attackers look for. A penetration test makes sense once you hold sensitive data or sell to companies that ask for one.

How long does it take to fix what I find?

Most small apps can be fixed in a day or two. Each Redproof finding comes with a prompt you can paste into your AI tool.