Repository navigation
Add File Search and File Share (#1379) - #1395
Merged
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
helpcodeai
added a commit
that referenced
this pull request
Nov 1, 2026
CodeQL flagged the slash-trimming regexes as quadratic on user and server input, and the unanchored S3 host patterns.
helpcodeai
added a commit
that referenced
this pull request
Nov 1, 2026
CodeQL flagged the slash-trimming regexes as quadratic on user and server input, and the unanchored S3 host patterns.
, #1387) Check an S3 endpoint given as a literal IP before connecting, take only the port from FTP PASV replies, and send HTML, SVG, XML and script files from share links as application/octet-stream.
Record the pen test and the same-domain download decision in the plan.
…osted (#1389) The network folder entry is self-hosted only, so the keyless count is 34.
CodeQL flagged the slash-trimming regexes as quadratic on user and server input, and the unanchored S3 host patterns.
…r Database (#1385, #1387) Settings -> File Share loads links 20 at a time with a file-name search, an access filter, Only mine for administrators, a count and an inline revoke confirmation; without Enterprise the policy shows as a read-only summary. The Files panel pages the files that could not be read. On Connectors -> New the File Search card follows Database.
The worker image crashed with MODULE_NOT_FOUND: nest build writes the entry to dist/src/ee/file-search/. Cap its heap below the container's 1 GB limit.
A Fritz!Box FTP server answers 'LIST -a /folder' with the root, so every folder looked like the root and a sync would never see the real files.
A model asks for 'Preisliste Brandschutztür' where the folder says Preisliste and the page says Brandschutztür: no page had both words, so the search found nothing. Quoted phrases and OR queries stay literal. _find without name or glob now says what to pass instead of listing every file.
…and Storage Box (#1389) Seven catalog entries open the File Search wizard with the protocol and the product preset (?preset=) filled in: S3 bucket, Nextcloud, ownCloud, Synology, QNAP, Fritz!Box storage and Hetzner Storage Box.
On a managed database whose user may not create roles, the migration skips amcp_files with a notice and the RLS policies apply to the application user itself; the repository switches role only when it is granted.
helpcodeai
force-pushed
the
helpcodeai/filesearch
branch
from
November 1, 2026 14:37
a3e86b9 to
d15df10
Compare
helpcodeai
enabled auto-merge (squash)
November 1, 2026 14:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
File Search connects a folder on an SFTP, FTP/FTPS, WebDAV or S3 server (and, self-hosted, a folder mounted into the install), indexes file names and the text of every page, and gives AI clients tools to search, read pages, grep exact codes and hand the file over. File Share turns files into temporary download links, also for files an AI client writes.
Plan and decisions:
docs/plans/file-search.md(epic #1379). User docs:docs/connectors/file-search.md.ee/ authorization: Matteo, the owner, explicitly chose "option B" for this feature: the sync engine (
packages/backend/src/ee/file-search/) and the text extractor (deploy/ee/file-extractor/) are under the AnythingMCP Commercial License, so a self-hosted install without a licence cannot use File Search after the 30-day trial. Nothing else underee/is touched.Behaviour in both deployments
FILE_SOURCE_ROOTS; private hosts via the outbound allowlistFILES_MAX_FILE_BYTESfiles-worker+file-extractor(Compose profilefiles)COMPOSE_PROFILES=filesNothing changes for an install that does not enable the
filesprofile: the wizard says File Search is not running, and no new process starts.Changes
FILE_SEARCHconnector type;file_sources,file_scans,file_sync_runs,file_entries,file_chunks(generatedtsvector, GIN),file_shares,file_access_rules;pg_trgm; Row-Level Security on all seven tables, enforced through the NOLOGIN roleamcp_files(SET LOCAL ROLEintenantTx)..msg/.emlwith attachments, RTF, legacy.doc/.ppt/.xls; zip-bomb guard; optional OCR image.SKIP LOCKED, per-workspace fairness, slices of 25 files, back-pressure on active DB sessions, delta sync by size/mtime/ETag and SHA-256, pauses (quota before download, licence, errors, admin) with one e-mail.guide,search,find,list,read,grep,status,get_file,share_fileoff by default),share_upload/list/revokefor the File Share connector; plan quotas and the next plan that really includes more; share links/f/<token>(hash stored), 60-second HMAC download URLs,attachment+nosniff+CSP sandbox, active types sent asapplication/octet-stream.sftp,ftp,webdav,network-folderentries open the wizard; generic create and catalog import refuse FILE_SEARCH.docker-compose.ymlanddocker-compose.cloud.yml(profilefiles, memory/CPU limits);deploy-cloud.ymlcopies the extractor build context.docs/connectors/file-search.md,.env.example, guides index, regeneratedllms.txt/llms-full.txt, adapter counts.Type
Testing
Commands run on the rebased branch (macOS, Node 22, local PostgreSQL 17 on 5433):
packages/backendnpm run lintpackages/backendnpx tsc --noEmit -p tsconfig.jsonpackages/backendnpx jest --maxWorkers=4packages/backendRUN_FILES_INTEGRATION=1 npx jest src/ee/file-search/file-search.integration.spec.tsFILES_IT_SFTP,FILES_IT_FTPS,FILES_IT_WEBDAV,FILES_IT_DATABASE_URLset to local test containers)packages/frontendnpm run lintpackages/frontendnpx tsc --noEmit -p tsconfig.jsondeploy/ee/file-extractor./test.sh(host) and container testsnode scripts/validate-adapters.mjs --warnnode scripts/adapter-count.mjs --checknode scripts/generate-llms-txt.mjs --checkManual, locally against SFTP (atmoz/sftp), FTPS (pure-ftpd, self-signed), WebDAV, S3 (MinIO) and a mounted folder with a 508-file seed: the wizard end to end, scans and budget, full and delta syncs, quota pause, licence pause and automatic resume, every MCP tool through a real MCP endpoint, File Share links, RLS; UI screenshot by screenshot at 1440×900 and 390×844, light and dark.
Pen test (two workspaces, plus VIEWER and EDITOR members): 31 API checks (cross-tenant on every File Search, File Share and generic connector route; secrets never returned; generic create and catalog import refused; malicious configs; forged tokens), MCP isolation, bounded regex, file-name XSS and header injection, 41 driver checks (loopback, RFC 1918, CGNAT, metadata IP, IPv6 and IPv4-mapped, decimal/hex/short IPv4, DNS to loopback, WebDAV redirect to an internal address, PASV to a foreign host, local symlinks and traversal, local driver on Cloud) and 29 role checks. Two findings, fixed in this PR with regression specs: an S3 endpoint given as a literal private IP skipped the lookup-based guard, and PASV handling refused NAT servers instead of ignoring the announced host.
Checklist
npm test)Related Issues
Closes #1380, #1381, #1382, #1383, #1384, #1385, #1386, #1387, #1388, #1389. Relates to #1379.