AI-built apps compared with apps from before AI coding tools

Are AI-built apps less secure? Not across the board. Against 188 similar apps created between 2019 and October 2022, AI-built apps leak fewer keys and carry fewer outdated packages, but get access control and payments wrong far more often.

8% vs 1%

serious data access problems, AI-built vs pre-AI apps

3% vs 0%

serious payment problems

5% vs 13%

serious secrets problems: here AI-built apps do better

Share of apps with each problem: AI-built apps against apps created before ChatGPT
MeasureAI-builtBefore ChatGPT
Any high or critical problem40%45%
A serious problem proven from the code5%8%
Leaked keys or secrets5%13%
Strangers can read or change data8%1%
Missing or broken login checks7%2%
Payment holes3%0%
Features anyone can use on your bill3%0%
Unsafe code (injection, unsafe HTML)29%33%
Launch problems0%6%
Key or password written in the code3%11%
Outdated package with a known flaw53%59%

Where AI-built apps do better

Keys and passwords written straight into the code appear in 3% of AI-built apps and 11% of the older ones. AI tools and platforms now default to environment variables, and GitHub blocks many known key formats on push. Outdated packages with known flaws are also less common (53% vs 59%), largely because AI-built apps are younger and start from current versions.

Where they do worse

The problems that grow are about who may do what: tables anyone can read (8% vs 1%), missing login checks (7% vs 2%), payment status the browser can set (3% vs 0%) and features anyone can use on the owner's bill (3% vs 0%).

These are business-logic mistakes. The code works, every line looks reasonable, and nothing breaks in a demo. They only show when someone asks "what stops a stranger from doing this?", a question AI tools rarely ask unprompted.

Why the overall number is higher for older apps

Counting any high or critical finding, 45% of the older apps are affected against 40% of AI-built ones (median score 90 vs 90). Most of that gap is old packages and old code habits. Treat the overall comparison with care; the category rows say more than the total.

Data: Redproof's automated checks on public GitHub repositories, November 2026. No repository is named. Method and limits