For the engineer who approves
Every fix is tested as three people before you see it.
PatchLabs rebuilds your database from supabase/migrations in a throwaway lab, reads it as an anonymous visitor and as a signed-in stranger, writes a migration, and re-tests until the leak is closed and the owner can still read their own rows. Below is exactly how, and where it stops.
16:56:15 [VERIFY] patchlabs-verify.txt found [KEYS] tree + full history CLEAN [REBUILD] migrations → lab db OK [SEED] fake rows, 2 test users [PROBE] anon profiles 2 LEAK stranger profiles 2 LEAK [FIX] smallest model writing [PROVE] anon 0 rows DENIED stranger 0 rows DENIED owner 1 row OK 16:57:00 [DONE] proven · receipt signed
01Pipeline
Your repo goes in. A proven migration comes out.
Everything in the middle happens in a lab we build for one run and throw away. Your live app and database are never part of it.
Your GitHub repo
Public repos today. Private repos come with our GitHub App.
supabase/
migrations/*.sql
src/**
full git history
Disposable Postgres with Supabase’s roles
- REBUILDApply your migrations in filename order
- SEEDFake rows owned by two test users
- PROBERead as anon and as a signed-in stranger
- FIXSmallest Claude model writes a migration
- PROVEIndependent re-test as all three identities
Proof fails? Discard the attempt, retry with a larger model, stop at the spend cap. Also in every run: key scan of code and full git history.
Report and migration
Download the .sql, or commit it on GitHub in one click. Your engineer applies it.
…_patchlabs_leak_fix.sql
signed receipt
02Rebuild
What “rebuilt from your migrations” means.
Supabase gives the anon and authenticated roles access to new tables in public by default. Row level security policies are the only thing between your public key and your rows. A lab that skipped those defaults would miss the leak.
So the lab starts as a fresh Postgres that copies Supabase’s defaults: the anon, authenticated and service_role roles, an auth schema with auth.uid(), auth.role(), auth.jwt() and auth.email(), storage tables, and the default grants. Then we apply every file in supabase/migrations in filename order, so later files override earlier ones exactly as they did in production.
If a migration fails to apply, we record it and keep going, so one broken file doesn’t stop the whole check.
The lab is only as complete as the migrations in your repo. A policy someone edited in the Supabase dashboard and never wrote to a migration is invisible to us. If you suspect drift, pull your schema into a migration first.
-- roles your app's keys map to
CREATE ROLE anon NOLOGIN NOINHERIT;
CREATE ROLE authenticated NOLOGIN NOINHERIT;
CREATE ROLE service_role NOLOGIN NOINHERIT BYPASSRLS;
-- auth.uid() reads the signed-in user from the JWT claims
CREATE FUNCTION auth.uid() RETURNS uuid AS
$$ SELECT nullif(current_setting('request.jwt.claim.sub', true), '')::uuid $$;
-- Supabase's default grants: the reason RLS matters
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT ALL ON TABLES TO anon, authenticated, service_role;- Your code
- Never run. The lab reads SQL from your migrations; nothing from your repo is executed or installed.
- Agent config
.claude/,CLAUDE.mdandAGENTS.mdare removed before the fix agent reads the repo, so a repo can’t steer it.- Repo text
- Treated as untrusted data. Instructions written in it are ignored.
03Identities
Three people, simulated inside the database.
We seed fake rows owned by test users, then switch role and JWT claims exactly as PostgREST does for a real request. No traffic goes anywhere.
set role anon; -- no JWT claims at all select * from public.profiles;
Anyone holding your public key, which ships inside every page of your app.
set role authenticated; -- request.jwt.claim.sub = user 2 select * from public.profiles;
A real account that owns none of the rows. Anyone can sign up, so signing up must not unlock other people’s data.
set role authenticated; -- request.jwt.claim.sub = user 1 select * from public.profiles;
The user who owns the seeded row. A fix that locks them out is a broken app, not a fix.
Which tables
Tables in public with personal columns: names like email, phone, address and date of birth. Those are the ones where a leak hurts someone.
How rows get an owner
We fill the owner column (user_id, owner_id, created_by, profile_id and similar) with a test user’s id, and required columns with fake values.
Rows and columns
Rows and personal columns are counted separately. A fix that keeps a table public but hides its personal columns, with column grants or a safe view, can pass.
04Proven
“Proven” has a strict definition.
A fix is called proven only when every condition below holds, in a check we run ourselves after the fix agent has finished. What the agent says about its own work doesn’t count.
- fix_errors = 0
The migration applies cleanly
On top of all your existing migrations, with no errors.
- DENIED
The anonymous visitor reads nothing private
Zero private rows and zero personal column values.
- DENIED
The signed-in stranger reads nothing private
Same test, as a different signed-in user.
- OK
The owner still reads their own row
Including its personal columns. Zero here means the owner was locked out, and the fix fails.
{
"table": "profiles",
"anon_before": 2, "anon_after": 0,
"stranger_before": 2, "stranger_after": 0,
"owner_after": 1,
"personal_cols_after": { "anon": 0, "stranger": 0 },
"fix_errors": [],
"passed": true
}Smallest model first. Bigger only when the proof fails.
Each check starts with the smallest Claude model. If its fix doesn’t pass, the attempt and any code edits are discarded and a larger model tries, up to a spend cap per check. In our 9 Nov study, 15 of 20 apps needed only the smallest model (n=20).
05When proof fails
No proof, no “proven”. A person looks first.
If no model produces a fix that passes, we don’t ship a guess. The check is marked needs review, nothing is called proven, and a person reviews it before you act on it.
In the 9 Nov study, 2 of 20 apps ended this way (n=20). We publish them because they show where the line is:
The agent decided the data was meant to be public.
It judged the flagged table a false positive and wrote no change. The lab still saw the data readable, so the check could not pass. That’s the correct outcome: whether the data should be public is the owner’s call, so it became a question for the owner, not a silent dismissal.
An orders table readable by anyone, and no fix passed.
Three models in turn wrote fixes. None closed the leak while keeping the owner’s access in the lab. The report went to a person with two business questions for the owner about who should see orders.
Apps from the study are never named. How we ran it.
[PROBE] anon → orders LEAK [FIX] smallest model [PROVE] passed = false FAIL attempt discarded [FIX] larger model [PROVE] passed = false FAIL [FIX] largest model [PROVE] passed = false FAIL [STOP] nothing marked proven status: needs review 2 questions for the owner
Question for the owner · example
Should shoppers be able to see sellers’ email addresses?
- Yes, show them on listings
- No, only through a contact form
Most of the human moments are business questions like this one. The report asks them in plain English with a recommended answer, and signed-in members can record the answer.
06Approval
Nothing reaches production until your engineer applies it.
We never push to your repo or touch your database. You get a file and the evidence behind it, and you ship it the way you ship any migration.
Read the report
Who could read which table, the before-and-after counts for each identity, the migration, and a risk note (low, medium or high) saying what on the site could behave differently.
Answer any business questions
Only where the right fix depends on what you intend, with a recommended answer.
Approve or reject
Signed-in members record a decision. Rejecting says “these tables are meant to be readable”: we remember that for the app, and future re-checks don’t alert on them.
Apply the migration
Download the
.sql, or use “Commit this file on GitHub”, which opens GitHub’s new-file page withsupabase/migrations/<timestamp>_patchlabs_leak_fix.sqlfilled in. Automatic pull requests need our GitHub App, which is coming.Re-check
Run the check again, or let the schedule do it, and confirm the leak is gone against your merged code.
Signed receipt
Anyone can check the result wasn’t edited.
Every finished check has a receipt signed with Ed25519: repo, status, tables leaking, whether the lab proof passed, a SHA-256 of the fix, and every decision and answer with who made it. Verify it against our published key:
import base64, json, urllib.request
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
API, CHECK = "https://api.patchlabs.dev", "<check id>"
r = json.load(urllib.request.urlopen(f"{API}/receipts/{CHECK}"))
k = json.load(urllib.request.urlopen(f"{API}/.well-known/patchlabs-receipt-key"))
assert r["key_id"] == k["key_id"], "signed with a different key"
msg = json.dumps(r["receipt"], sort_keys=True, separators=(",", ":")).encode()
pub = Ed25519PublicKey.from_public_bytes(base64.b64decode(k["public_key"]))
pub.verify(base64.b64decode(r["signature"]), msg) # raises if changed
print("receipt is genuine:", r["receipt"]["status"])Full details in the API docs.
07Isolation
A lab built for one run, then dropped.
The lab runs on a PatchLabs worker machine, in a Docker container running Postgres. Each run gets its own database, and that database is dropped when the run ends.
- Database
- A fresh database per run, rebuilt from your migrations and dropped with
DROP DATABASE … WITH (FORCE)after the probe and after each proof. - Data
- Fake rows only (values like
x@example.com). Your customers’ data never enters the lab because we never connect to your database. - Network
- The pipeline makes no request to your app, API, database or hosting.
- Fix agent
- Can read the repo copy, write inside its own run folder, edit the repo copy, and call the lab verifier. Nothing else.
- Your repo
- Read only. Nothing is pushed: the pipeline runs in dry-run mode.
- Spend
- A cap per check stops model escalation, so one hard repo can’t run up an open-ended bill.
- Kept
- Your email, the repo name, the report and the fix, so the report link keeps working. Signed-in users can export or delete it.
08Limits
What a proven fix does not cover.
Proof covers the access rules we test, on the database your migrations build. Here is everything outside that line.
Missing or drifted migrations. Changes made in the dashboard and never written to supabase/migrations aren’t in the lab, so the proof can’t see them.
Non-Supabase backends. Firebase, a platform’s own built-in database, or an ORM with no Supabase migrations: we can’t rebuild those yet.
Custom auth. Policies that rely on auth outside Supabase’s auth.uid() and JWT claims may not map onto our three identities. If that’s you, email us before relying on the result.
Reads, not writes. The lab proof tests who can read personal data. Write rules (for example an UPDATE policy without WITH CHECK) are flagged by the free RLS check, which reads your migrations, not proven in the lab.
Tables, not everything. Storage buckets, edge functions, API routes and business logic aren’t proven. The free RLS check flags public buckets.
Not a pentest. We test one thing in depth: who can read which rows, plus exposed keys. For a high-stakes launch, get a pentest too.
09FAQ
Engineer questions.
Something else? Email the founder. You’ll get a reply from the person who built the pipeline.
Do you need my service-role key or database password?
No. We never ask for them and never connect to your database. Everything runs against a copy rebuilt from your repo.
Do you run my app’s code?
No. The lab only applies the SQL in your migrations. The fix agent reads your frontend to understand which pages use which tables, and can propose a small code change, but nothing from your repo is executed.
What if my migrations don’t build?
We record each migration that fails to apply and keep going. If the tables that matter never get built, there is nothing to prove, so email us and we’ll look. In our study, 318 of the 436 public repos we ran built a working database in the lab (PatchLabs lab study, Nov 2026).
Which models write the fix, and what do they see?
Claude models from Anthropic: the smallest first, larger ones only if the proof fails. They see the relevant code in your repo and the lab’s findings. They never see your production data, because we never have it.
Could the fix break my app?
That’s why the owner check exists: a fix that locks owners out fails. Every report also carries a risk note saying what could behave differently, such as a logged-out page that relied on the old rule. Review it, then apply.
Can I test against a pull request before merging?
Checks read your default branch today. Re-proving every pull request needs our GitHub App, which is coming. Meanwhile you can trigger a re-check from CI after a merge: see Monitoring.
How long does a check take?
Median 3.5 minutes from repo to proven fix in our 9 Nov study (n=20). Large schemas and model escalation take longer.
Free check
Watch a fix pass before you ship it.
Paste your public repo. You’ll get the leaks, the migration and the proof transcript on one page.