Repository navigation
Run File Search maintenance per workspace under RLS (#1418) - #1557
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
file_*tables useFORCE ROW LEVEL SECURITYwith anorg_isolationpolicy onapp.current_org. Every query that ran outsideFileSearchRepository.withTenantsaw no rows when the database user is neither superuser norBYPASSRLS(managed PostgreSQL: RDS, Azure, Supabase…). On such installs syncs never started, leases were never released, scans never ran, cleanups and share links did not work, and Enterprise folder rules were invisible, so deny rules were ignored (fail open).This PR runs every one of those queries one workspace at a time inside
withTenant. Theorg_isolationpolicies are unchanged; there is no maintenance switch or other bypass. The only new policy is a SELECT-onlyshare_by_tokenonfile_shares, which shows a row only to whoever already holds its secret token.Changes
20261106120000_file_share_token_policy:share_by_tokenonfile_shares,FOR SELECT,USING (token_hash = NULLIF(current_setting('app.share_token_hash', true), '')). An unset setting (NULL) or an emptied one ('') never matches. Permissive policies are ORed per command, so it adds nothing to INSERT/UPDATE/DELETE.FileSearchRepository:mode()(amcp_files / subject to RLS / bypassing RLS),withShareToken(hash, fn)(transaction-localapp.share_token_hash,app.current_orgcleared, amcp_files when granted),tenantsWithFileSearch()(connectorshas no RLS),visibility()for the startup check. The amcp_files role check no longer errors out when the role does not exist.withTenant(most overdue source, live lease, oldest queued scan, stale scans), sorts candidates bynext_sync_atacross workspaces, then claims each in its own workspace transaction. The claim takespg_try_advisory_xact_lockon the workspace, re-checks every lease condition and usesFOR UPDATE SKIP LOCKED. Leases this worker holds are kept in memory, and renew/release/stop run in the lease's workspace withorganization_idin the WHERE clause. Scans are claimed per workspace, oldest first. The sweep (drafts, scan samples, 30-day licence pause, File Share) runs per workspace. The workspace list is cached for 30 s, andkick()refreshes it.TICK_MSandLEASE_MINare unchanged.closeInterruptedRunsruns per workspace. At start the worker logs its mode and warns once when File Search folders exist but nofile_sourcesrow is visible.scopeForreadsfile_access_rulesinsidewithTenant. Any failure throws, andexecutereturnsfolder_rules_unavailablewithout querying the index (fail closed).byTokenreads throughwithShareToken, and the download count is updated insidewithTenant(row.organizationId). The workspace name comes from a separate query onorganizations. The signed download URL is now<id>.<orgId>.<exp>.<sig>with the workspace inside the HMAC (download-v2:domain prefix). The signature is verified before the row is read in that workspace. Old<id>.<exp>.<sig>URLs are refused; they lived 60 seconds (DOWNLOAD_URL_TTL_S).FileShareService.sweepruns per workspace.docs/connectors/file-search.md, new "Database user" section (works without superuser or BYPASSRLS, whatamcp_filesis, grants on managed databases, startup log modes). It replaces the outdated note that said superuser or BYPASSRLS was required.Cloud and self-hosted
amcp_files): same behaviour. Maintenance now also runs under the policies, so a scheduler bug cannot touch another workspace. Each 5 s tick costs one small transaction per workspace with a File Search connector (3 workspaces today), plus one per claim.amcp_files): syncs, scans, lease release on deploy, interrupted-run cleanup, sweeps, share links and folder rules now work. Folder deny rules are honoured.Type
Testing
New
src/ee/file-search/file-search-rls.integration.spec.ts, gated onAUDIT_TEST_DATABASE_URL(already set in the backend CI job;FILES_RLS_TEST_DATABASE_URLoverrides it). As the superuser it checks thatshare_by_tokenis SELECT-only, then runs the same checks in three variants: the superuser; a freshLOGIN NOSUPERUSER NOBYPASSRLSrole that owns nothing, has table/sequence grants and is a member ofamcp_files; and the same role withoutamcp_files. The checks cover claim order and one claim per workspace, a second worker finding everything busy, renew in the wrong workspace touching 0 rows, release andstop(), scan claim and stale-fail,closeInterruptedRuns, the draft and share sweeps, share by token, signed download (a tampered org is refused), folder deny rule, isolation (A cannot read or update B, a wrong hash shows nothing, the right hash shows only that row and cannot UPDATE/DELETE), and unset, emptied or session-reset settings revealing nothing, including a row whosetoken_hashis''. Negative check: withscopeForreverted to the bare connection, the deny-rule test fails in both non-superuser variants and passes as superuser.Commands (in
packages/backend, afterprisma generate):docker run -d --rm --name fsrls-pg -e POSTGRES_USER=amcp -e POSTGRES_PASSWORD=testpassword -e POSTGRES_DB=anythingmcp_test -p 55432:5432 postgres:17-alpinethenDATABASE_URL=postgresql://amcp:testpassword@localhost:55432/anythingmcp_test npx prisma migrate deploy(from main incl.20261106100000_tool_role_rate_limits)AUDIT_TEST_DATABASE_URL=postgresql://amcp:testpassword@localhost:55432/anythingmcp_test npx jest src/ee/file-search/file-search-rls.integration.spec.ts --maxWorkers=4AUDIT_TEST_DATABASE_URL=… npx jest src/file-search src/ee/file-search --maxWorkers=4npx jest src/file-search src/ee/file-search --maxWorkers=4(no DB)npx jest src/ee/file-search/sync/scheduler.spec.ts src/ee/file-search/files-worker.service.spec.ts src/file-search/file-share.service.spec.ts src/file-search/file-search-scope.spec.ts src/file-search/file-search.engine.spec.ts --maxWorkers=4npx eslint <the 11 changed .ts files>(no--fix)npx tsc --noEmit -p tsconfig.jsonnpx jest --maxWorkers=4(full backend, no DB)src/auth/jwt-or-management-key.guard.spec.tsneeds a live DB atDATABASE_URL(none locally; CI has one). Baseline, not touched by this PRChecklist
npm test): everything except the DB-backed guard spec above, which needs CI's databaseNotes
share_uploadhave noconnector_id. If a workspace later deletes every FILE_SEARCH connector, it leaves the sweep's workspace list, and its expired upload files stay in storage until a File Search connector exists again. Shares from a connector are cascade-deleted with it, which is existing behaviour.Fixes #1418