How the study was done, and its limits

How repositories were chosen

GitHub searches in eight groups (Lovable, Bolt and v0; full-stack Cursor and Claude Code; Python back ends; other back ends; mobile; SaaS integrations; AI automations; infrastructure), limited to repositories pushed in the last year and not forks. A repository counted as AI-built only with a clear marker: AI tool files (.cursorrules, CLAUDE.md, .bolt), the lovable-tagger package, or a README saying so. Templates, tutorials, starters and tool collections were skipped, as was anything with fewer than 12 source files. At most two repositories per owner.

What was checked

Only Redproof's automated rules: no AI review and no manual verification in this run. The code was read, never run, and no live app was touched. "Proven" means a check that confirms a fact from the code alone (a committed secret, a table with row level security off); other findings are strong signals that a person should confirm.

Limits and fair criticism

• Public repositories are not the same as live apps. Many are side projects; some never launched. Private, paid products may be better or worse.
• The AI-built label rests on markers. Apps built with AI that leave no marker are missing, and a README mention does not prove how much code AI wrote.
• Automated rules miss problems (they cannot judge business logic fully) and raise some false alarms. One check, hidden instructions for AI tools, was too noisy and is left out of all rankings; it was tightened after the study.
• The comparison apps are older. They had more time to collect outdated packages and less help from today's platform defaults. We capped them at 12 per search so no single kind of app dominates, but the groups are not perfectly matched.
• Redproof sells security checks, so we have an interest in problems being found. That is why the method, the limits and the checks are published, and why we report where AI-built apps do better.

Privacy

We never publish repository or owner names, never contact owners about findings from this study, and keep only anonymous totals. If you recognise your project in a description and want it excluded from future runs, write to hello@redproof.dev.