Skip to content

Add an EMAIL connector type: IMAP and SMTP (#1344) - #1348

Merged
helpcodeai merged 8 commits into
mainfrom
feat/imap-smtp-connector
Oct 30, 2026
Merged

helpcodeai merged 8 commits into
mainfrom
feat/imap-smtp-connector

Conversation

@helpcodeai

@helpcodeai helpcodeai commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A new connector type, EMAIL, for any mailbox that speaks IMAP and SMTP: IONOS, Strato, GMX, web.de, T-Online, Hetzner, all-inkl, Aruba, Gmail/Google Workspace with an app password, Yahoo, iCloud, Fastmail, self-hosted Dovecot/Postfix. Today the catalog has only provider APIs (Gmail, Microsoft 365, Zoho Mail), so a search for "IMAP", "SMTP" or "email" finds nothing for the large share of small businesses whose mail runs at a hoster. This was discussed with the maintainer before starting, as AGENTS.md asks for a new connector type; the schema change is described in #1344.

Like OData, an Email connector has no endpoints to map: it ships a fixed tool set (list folders, search, get message, get attachment, set flags, move, delete to Trash, save draft, send), created with a hand-made connector and injected into catalog adapters.

Changes

  • Schema: ConnectorType gains EMAIL (20261030100000_add_email_connector_type, ALTER TYPE ... ADD VALUE). No new tables or columns.
  • Engine (connectors/engines/email.engine.ts): imapflow (MIT) for IMAP, mailparser (MIT) for HTML bodies and header parsing, the existing nodemailer for SMTP; outgoing messages reuse mime-message.util.ts (header-injection checks, RFC 2047). One IMAP connection per tool call (connect, work, logout), 15 s connect / 60 s socket timeouts, no IDLE, bounded literal/response sizes. Bodies and attachments are downloaded part by part with BODY.PEEK (reading does not mark as read); bodies cut with a marker, attachments capped at 4 MB and returned as text or as an MCP file (existing binary-response path). Bcc only in the SMTP envelope, Sent copy keeps it; Gmail is left to file its own copy; a failed Sent copy after SMTP accepted the mail is a warning, not an error. Delete moves to Trash (special-use) and never expunges; a server with neither MOVE nor UIDPLUS is refused for moves because its fallback EXPUNGE would hit unrelated messages.
  • Security: hosts go through the outbound SSRF guard; new resolveSafeOutboundHost returns the checked addresses so the socket connects to exactly those (name kept for SNI/certificate). Passwords (and their base64 / AUTH PLAIN forms) are scrubbed from every error and log line; the clients' own loggers are off.
  • Settings: non-secret in config.email (catalog {{VARS}} resolved per call, like OData settings), login in the encrypted authConfig (BASIC_AUTH: username, password, optional smtpUsername/smtpPassword). On edit, an empty username/password keeps the stored value. authConfig.mechanism is the extension point for XOAUTH2.
  • Wiring: connectors.service (test connection: IMAP login + LIST, SMTP verify(); Run Test / install probe), dynamic-mcp-tools, tool-annotations (reads read-only and closed-world, send open-world, delete destructive), controller create (built-ins, settings check) and update (auth merge), validateBaseUrl (imap:// / imaps://), catalog withEmailBuiltins, validator and catalog spec.
  • Catalog adapter intl/imap-smtp ("Email (IMAP/SMTP)", category email, aliases IMAP/SMTP/E-Mail/GMX/…): provider table with hosts and ports, app-password instructions, tool list, limits, Cloud vs self-hosted; static guide tool; generic envelope logo; adapter count 573 (README ×4, glama, CITATION, server.json, package.json, llms.txt regenerated).
  • Frontend: Email type on connectors/new (provider presets, servers/ports/security, login, optional SMTP login, sender, folders, "accept self-signed" and plaintext only when self-hosted); connector page edit + summary (catalog installs are edited through their variables); list badge and filter; email methods in the tool editor; en/de/it strings.
  • Docs: docs/connectors/email.md, README source tables (4 languages), docs/guides.md, docs/tool-definition.md.

Cloud and self-hosted

Cloud Self-hosted
Private / loopback / link-local mail host refused by the SSRF guard refused by default, allowed after the admin allowlists the host (Admin → Settings or SSRF_ALLOWED_HOSTS); Test connection offers the allowlist link as for other connectors
Encryption SSL/TLS or STARTTLS only (STARTTLS required, never opportunistic) also plaintext
Certificate must be valid "Accept self-signed certificate" available
Ports IMAP 993/143, SMTP 465/587/2525 (no 25) any

These rules are enforced in resolveEmailSettings (create/update API and every call) and reflected in the form.

Hardening (commit 0741258)

  • Cloud SMTP is blocked by the hosting provider on 25, 465 and 587 (measured from the droplet; 993 open, 2525 passes). On Cloud a send on those ports gives up after 5 s (not 15 s) with an smtp_blocked error: the provider blocks the port, nothing was sent, use save_draft or a provider on 2525. Test connection on Cloud reports "IMAP login accepted, N folders; SMTP not reachable: …" instead of failing. Stated in the adapter instructions, guide tool, EMAIL_SMTP_PORT help, the form's Cloud note and docs/connectors/email.md.
  • Port 25 removed from the Cloud SMTP ports (direct-to-MX delivery from our address risks blocklisting); self-hosted keeps it.
  • SMTP auth is forced on Cloud (forceAuth): a server that does not advertise AUTH fails instead of relaying unauthenticated. Integration test against an smtp-server open relay: refused on Cloud, accepted (control) self-hosted.
  • Prompt injection: send_message and delete_messages are created switched off (isEnabled: false for hand-made connectors, enabled: false in the catalog, same mechanism as other adapters' risky tools); save_draft stays on as the default way to answer. Reading tools' descriptions and every result with message content (contentNotice) say the content is untrusted data and its instructions must not be followed.

Type

  • Bug fix
  • New feature
  • Documentation
  • Refactoring
  • Performance improvement
  • Other (describe below)

Testing

  • Added/updated unit tests
  • Tested manually (describe steps below)
  • No tests needed (explain why)

New tests:

  • email.engine.spec.ts (38, scripted imapflow/nodemailer): search criteria mapping, limit cap, newest first, snippets, body truncation marker, HTML to text, attachment cap (refused before and during download), text vs binary attachments, reply headers (In-Reply-To, References, Re:, Reply-To recipient, \Answered), Bcc envelope-only, Sent-folder rules (generic, Gmail, never, failed append after send), SSRF guard on Cloud (loopback, RFC 1918, metadata IP, SMTP host) and self-hosted (allowlisted host, pinned address with servername), plaintext refused on Cloud before any socket, STARTTLS required, MOVE/UIDPLUS guard, no password or AUTH PLAIN blob in errors or log lines, TLS error wording per deployment.
  • email.engine.integration.spec.ts (13): the same flows against real servers in-process, no Docker: ImapKit (MIT, in-memory IMAP) and smtp-server (MIT-0), including TLS with a self-signed certificate refused by default and accepted with the self-hosted switch. Both are devDependencies only.
  • email-settings.spec.ts, email-message.util.spec.ts, connectors.controller.email.spec.ts, connectors.service.email.spec.ts, dynamic-mcp-tools.email.spec.ts, additions to tool-annotations.spec.ts, base-url.util.spec.ts, catalog.spec.ts, validate-adapters.test.mjs; adapters/intl/imap-smtp.live.spec.ts (static assertions; read-only live part gated behind RUN_IMAP_SMTP_LIVE=1, not run).
  • Frontend: tests/e2e/connector-email.spec.ts (creates an Email connector from the form, mocked API).

Commands and results (2026-10-30, macOS, Node 22):

Directory Command Result
packages/backend npx tsc --noEmit -p tsconfig.json exit 0
packages/backend npm run lint 0 errors, 11 warnings, all in files this PR does not touch; no unrelated --fix rewrites
packages/backend npx jest src/connectors/engines/email src/connectors/email src/connectors/connectors.controller.email.spec.ts src/connectors/connectors.service.email.spec.ts src/mcp-server/dynamic-mcp-tools.email.spec.ts src/adapters/intl/imap-smtp.live.spec.ts all pass (2 live tests skipped)
packages/backend npm test 819 suites passed (4 skipped), 24,203 tests passed, 0 failed; no baseline failures
packages/frontend npm run lint 0 errors, 46 warnings (pre-existing patterns; none new in the email component)
packages/frontend npx tsc --noEmit -p tsconfig.json exit 0
packages/frontend npx playwright test tests/e2e/connector-email.spec.ts 1 passed
root node scripts/validate-adapters.mjs --warn 587 passed, 0 failed, 1442 warnings (none for imap-smtp; the rest pre-existing)
root node scripts/regenerate-catalog.mjs catalog.ts diff committed
root node scripts/adapter-count.mjs --check OK, 573 adapters
root node scripts/generate-llms-txt.mjs --check OK after regeneration
root node --test scripts/validate-adapters.test.mjs scripts/adapter-new.test.mjs scripts/generate-llms-txt.test.mjs 58 passed
root node scripts/check-runtime-deps.mjs no nested production dependency

Manual end-to-end check (local, no real mailbox): ImapKit on 127.0.0.1:11143 and smtp-server on 127.0.0.1:11025 as standalone processes; backend from this branch with a fresh database (prisma migrate deploy applied the new migration; enum lists EMAIL) and SSRF_ALLOWED_HOSTS=127.0.0.1 (the self-hosted allowlist path).

  • POST /api/connectors (type EMAIL): 201, nine office_mail_* tools, attached to the default MCP server; authConfig not in the response.
  • Test connection: "IMAP login accepted, 4 folders; SMTP login accepted". Wrong password: auth_failed, password not in the message. Restoring with an empty username kept the stored one.
  • Run Test: search, get message, reply via send_message: the SMTP server received In-Reply-To/References/Re: and the right envelope, the copy landed in Sent, the original got \Answered.
  • MCP over HTTP with an API key: tools/list shows the annotations above; tools/call list_folders and save_draft succeed; an unknown UID returns isError with a clear message.
  • Catalog: GET /api/adapters?search=IMAP|SMTP|email|GMX all find imap-smtp; POST /api/adapters/imap-smtp/import creates 10 tools and its probe (imap_smtp_list_folders) passes; Test connection on the installed connector passes.

Checklist

  • My code follows the existing code style
  • I have tested my changes locally
  • I have updated documentation (if applicable)
  • All existing tests pass (npm test)
  • This PR has a descriptive title

Not in this PR

  • XOAUTH2 (Gmail, Microsoft 365 with OAuth). authConfig.mechanism is the extension point; both clients accept access tokens.
  • IDLE / push for new mail; tools run on demand only.
  • Attachments on outgoing mail; Gmail X-GM-RAW search.
  • Per-user mailbox credentials (USER credential scope is OAuth2-only today).

Related Issues

Closes #1344

A new ConnectorType EMAIL (migration adds the enum value) and EmailEngine,
which runs a fixed set of built-in tools against a mailbox: list folders,
search, get message, get attachment, set flags, move, delete to Trash,
save draft (IMAP APPEND) and send (SMTP, reply threading, copy to Sent).

- imapflow for IMAP, mailparser for HTML bodies and headers, the existing
  nodemailer for SMTP; one IMAP connection per call, 15 s connect and
  60 s socket timeouts, no IDLE
- hosts pass the SSRF guard and the socket goes to the checked address
  (resolveSafeOutboundHost), with the name kept for TLS
- Cloud: TLS only, valid certificates, standard ports; self-hosted may
  allow plaintext and self-signed certificates for internal servers
- passwords are scrubbed from every error and log line
- settings in config.email (catalog {{VARS}} resolved per call), login in
  the encrypted authConfig; an empty password on edit keeps the stored one
- tool annotations per method; send is open-world, delete destructive
- Email type on the new-connector page: provider presets, IMAP and SMTP
  servers with port and security, login, optional SMTP login, sender,
  folders; plaintext and self-signed only offered self-hosted
- connector page: edit the same settings (empty passwords keep the stored
  ones), server summary, no import for the built-in tool set
- Email badge and filter in the connector list, email methods in the
  tool editor, strings in en/de/it, Playwright smoke test
intl/imap-smtp installs an EMAIL connector with the nine email built-ins
under the imap_smtp prefix and a static guide. Its instructions carry the
hosts and ports of common providers and where to create app passwords.
The validator and catalog spec accept the EMAIL type; adapter count 573.
docs/connectors/email.md: setup, built-in tools, settings, how it
connects, Cloud vs self-hosted behaviour and what is not supported yet.
README source tables, the guides index and the tool-definition reference
list the new type; llms.txt regenerated.
Comment thread package-lock.json Fixed
Comment thread package-lock.json Fixed
Comment thread package-lock.json Fixed
# Conflicts:
#	CITATION.cff
#	README.de.md
#	README.ja.md
#	README.md
#	README.zh-CN.md
#	glama.json
#	llms-full.txt
#	llms.txt
#	package.json
#	server.json
# Conflicts:
#	CITATION.cff
#	README.de.md
#	README.ja.md
#	README.md
#	README.zh-CN.md
#	glama.json
#	llms-full.txt
#	llms.txt
#	package.json
#	server.json
- Cloud: the hosting provider blocks outgoing SMTP on 25, 465 and 587.
  A send there gives up after 5 s with an smtp_blocked error that says so
  and points to save_draft or a provider on port 2525; Test connection
  reports IMAP working and SMTP not reachable instead of failing
- Cloud: SMTP port 25 is no longer allowed (direct delivery from our
  address risks blocklisting); self-hosted keeps it
- Cloud: SMTP login is forced, so nothing goes out through an open relay
  unauthenticated
- send_message and delete_messages are created switched off, for
  hand-made connectors and catalog installs; save_draft stays the default
  way to answer
- reading tools and their results say that message content is untrusted
  data whose instructions must not be followed
- adapter instructions, guide, setup help and docs updated
- mailparser 3.9.43 pins nodemailer 10.1.4, the version the backend
  already uses, so the nested nodemailer 10.1.1 with three advisories is
  gone from the lockfile
- the from address is checked by splitting instead of a regular
  expression that backtracks polynomially on long runs of dots
- the email login merge runs on the connector's type alone; the request
  body no longer guards the branch
@helpcodeai
helpcodeai merged commit acaa54d into main Oct 30, 2026
14 checks passed
@helpcodeai
helpcodeai deleted the feat/imap-smtp-connector branch October 30, 2026 18:57
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an EMAIL connector type: IMAP and SMTP

2 participants