Monthly testing retainer
For teams and agencies that release every week.
Who this is for
You release often, the team is small, and review cannot keep up with the volume of AI-written changes. You want a human test pass as a standing part of the release.
Agencies use it to cover several client projects under one agreement.
Built for teams whose code is now mostly AI-written, and agencies handing AI-built products to clients.
What we check
All nine web categories, scoped per release to what changed plus a standing check of secrets, data access and auth.
Secrets and keys
API keys and passwords the app uses to talk to other services.
5 checks
Secrets and keys
API keys and passwords the app uses to talk to other services.
Data access
Rules that decide which user can read or change which rows in the database.
7 checks
Data access
Rules that decide which user can read or change which rows in the database.
- Firebase rules left in test modeCritical
- Record ids accepted without an ownership checkCritical
- Inverted policy logicCritical
- Any signed-in user can read every rowCritical
- Row Level Security disabled on a tableCritical
- RLS policy that allows everything with using (true)Critical
- Uploaded files in a public storage bucketHigh
Login and permissions
How the app knows who you are and what you are allowed to do.
7 checks
Login and permissions
How the app knows who you are and what you are allowed to do.
- Admin routes without any login checkCritical
- Authorization enforced only in the browserCritical
- Default admin credentials left in placeCritical
- Tokens trusted without server verificationCritical
- Role chosen by the client at signupCritical
- Password reset links that never expireHigh
- Sessions that survive logout and password changesHigh
Payments
Everything between the pay button and the money arriving.
5 checks
Payments
Everything between the pay button and the money arriving.
AI features
Calls to a language model inside the app, and what they can trigger.
5 checks
AI features
Calls to a language model inside the app, and what they can trigger.
Reliability
Whether the app keeps working when something fails.
5 checks
Reliability
Whether the app keeps working when something fails.
Performance
How the app behaves when there is more data or more users.
4 checks
Performance
How the app behaves when there is more data or more users.
Data integrity
Whether stored data stays correct over time.
5 checks
Data integrity
Whether stored data stays correct over time.
Deploy and config
How code reaches production and how the server is configured.
8 checks
Deploy and config
How code reaches production and how the server is configured.
- AI agent with production credentialsCritical
- Builder project set to publicHigh
- CORS open to any origin with credentialsHigh
- Dependencies with known vulnerabilitiesHigh
- Debug mode left on in productionMedium
- Missing security headersMedium
- No automated checks before deployMedium
- Every change goes straight to productionMedium
Why these checks: seen in public
Escape analysed more than 5,600 publicly deployed apps built with AI app builders and found more than 2,000 vulnerabilities, over 400 exposed secrets and 175 instances of exposed personal data.
Oct 2025 · Escape · all sources
Lovable reported that from 3 February to 20 April 2026 a backend regression let any logged-in Lovable user read the chat history and source code of public projects; a fix shipped within two hours.
Apr 2026 · Lovable · all sources
Each incident links to the pattern and the self-check you can run today, in the evidence list.
What you receive
- 01Testing of each release against the checklist
- 02Monthly findings report
- 03Priority channel with the reply commitment
See a full sample report (synthetic, fictional app) for the exact format.
How it works
- Step 1
Share access
Send the app URL and a test login by email. For code review packages, add read access to the repository.
- Step 2
We test
We run the checklist for your package against the live app, by hand and with scripts, inside the turnaround.
- Step 3
Report and call
You receive the findings report and a plain-English summary. We walk you through it if the package includes a call.
- Step 4
Re-check after fixes
Fix with your developer or your AI tool, then we re-test the critical findings within the re-check window, included.
Findings describe what we observed on the version you gave us access to, on the date of testing. Changes after that date are not covered until re-checked.
Questions about the monthly testing retainer
ongoing qa for ai generated code
A monthly retainer: we test each release against the checklist for your stack, keep a standing check on secrets, data access and auth, and send a monthly findings report. It covers the review capacity a small team lacks when most changes are AI-written.
how does the monthly testing retainer work
You tell us when a release is ready (or we watch a staging URL). We test what changed plus the standing checks, report within the reply commitment, and summarise monthly. Billed monthly, cancel any month.
Order the monthly testing retainer
from $1,500/mo, report in ongoing. The email opens with a short template: app URL, builder, stack, what you want checked.
Other packages: Pre-launch check ($490), Security sweep ($290), Ready-to-charge check ($390). Compare all.