RustMailer is a self-hosted email middleware built in Rust that provides unified API access to IMAP/SMTP, Gmail API, and Microsoft Graph API. It serves as a protocol abstraction layer between email providers and applications, handling multi-account synchronization, local caching, event dispatch, and template-based sending.
Key characteristics:
MailerType enumnative_db for envelopes, cacache for raw content, redb for task queuesThis overview introduces RustMailer's architecture, core components, and integration patterns. For detailed subsystem documentation:
Sources: README.md1-48 README.md49-63
<old_str>
RustMailer's architecture separates concerns into distinct subsystems: external interfaces, business logic, provider adapters, data storage, and event dispatch.
Diagram: Overall System Architecture
Sources: README.md1-63 src/modules/api/ src/modules/grpc_impl/ src/modules/account/ src/modules/message/ src/modules/sendmail/ src/modules/hook/ src/modules/cache/
| Component | Key Types/Functions | Location | Purpose |
|---|---|---|---|
| REST API | ApiV1, poem::Route | src/modules/api/v1.rs | OpenAPI endpoints at /api/v1/* |
| gRPC Services | MessageService, AccountService, OAuth2Service | src/modules/grpc_impl/ | Protocol Buffer services |
| Account Management | AccountModel, MailerType enum | src/modules/account/entity.rs | CRUD, provider routing |
| Message Operations | list_messages_in_mailbox, search_messages, retrieve_raw_email | src/modules/message/ | Query, fetch, transfer |
| Send Queue | EmailTask, TASK_DATABASE | src/modules/sendmail/ | Async SMTP delivery |
| IMAP Adapter | ImapExecutor, reconcile_mailboxes | src/modules/cache/vendor/imap/ | UID-based sync |
| Gmail Adapter | GmailClient, handle_history | src/modules/cache/vendor/gmail/ | History API sync |
| Outlook Adapter | OutlookClient, execute_outlook_sync | src/modules/cache/vendor/outlook/ | Delta sync |
| Event System | EventHookTask, EVENT_CHANNEL | src/modules/hook/task.rs | Webhook/NATS dispatch |
| Envelope Cache | EmailEnvelopeV3, GmailEnvelope, OutlookEnvelope | src/modules/cache/nativedb/ | Metadata storage |
| Content Cache | DISK_CACHE, cacache::Cache | src/modules/cache/disk.rs | Raw message storage |
Sources: src/modules/ directory structure, Cargo.toml21-133 </old_str>
<new_str>
RustMailer solves email integration complexity by providing:
Protocol Abstraction
MessageService, AccountService, SendMailService)MailerType enum routes to IMAP/SMTP, Gmail API, or Graph API implementationsEnvelope response type hides provider differencesBackground Synchronization
reconcile_mailboxes with UID comparison and batch fetchinghandle_history with history_id checkpointsexecute_outlook_sync with delta tokensEvent Infrastructure
EVENT_CHANNEL async queue with EventHookTask dispatchStorage Architecture
native_db::Database for EmailEnvelopeV3, GmailEnvelope, OutlookEnvelopecacache::Cache for content-addressed raw .eml filesredb::Database for task queue management (EmailTask, EventHookTask)Asynchronous Sending
EmailTask queue with exponential backoff retrytemplate_paramsbb8Target Use Cases:
Sources: README.md17-48 README.md49-63 Cargo.toml1-50
RustMailer addresses the complexity of integrating multiple email providers by offering:
native_db for envelope metadata, cacache for raw message contentTarget use cases include:
Sources: README.md17-33 README.md49-63
RustMailer's feature set organized by system layer:
| Layer | Capability | Implementation Details |
|---|---|---|
| API Layer | REST + OpenAPI | poem web framework, Swagger/ReDoc/Scalar documentation, port 15630 |
| gRPC Protocol Buffers | poem-grpc, compression (gzip/deflate/brotli/zstd), port 16630 | |
| Web UI | React + Vite + Shadcn UI, account switcher, mailbox tree, message list | |
| Business Logic | Account Management | CRUD operations, MailerType enum (ImapSmtp/GmailApi/GraphApi), OAuth2 flow |
| Message Operations | List, search (34 condition types), retrieve, transfer (move/copy), tag/flag | |
| Template System | Handlebars-based with template_params, dynamic content, test endpoint | |
| MTA Integration | SMTP connection pooling via bb8, DSN support, save-to-sent options | |
| Sync Engines | IMAP Sync | UID-based reconcile_mailboxes, diff UIDs/flags, batch fetch (200 UIDs) |
| Gmail Sync | History API handle_history, history_id checkpoints, parallel fetch (limit 5) | |
| Outlook Sync | Delta tokens execute_outlook_sync, @odata.nextLink pagination | |
| Event System | Event Types | 12 types: EmailAddedToFolder, EmailFlagsChanged, EmailSentSuccess, EmailSendingError, UIDValidityChange, MailboxDeletion, MailboxCreation, AccountFirstSyncCompleted, EmailBounce, EmailFeedBackReport, EmailOpened, EmailLinkClicked |
| VRL Processing | Vector Remap Language for payload transformation and filtering | |
| Dispatch | HTTP webhooks (POST/PUT, custom headers) and NATS (stream publishing, subject routing) | |
| Metrics | Prometheus counters/histograms: dispatch count, success/failure rates, duration by type/status/destination | |
| Data Layer | Envelope Cache | native_db: EmailEnvelopeV3, GmailEnvelope, OutlookEnvelope, composite primary keys with internal_date |
| Content Cache | cacache: content-addressed .eml storage, streaming API | |
| State Management | AccountRunningState, GmailCheckPoint, FolderDeltaLink for sync resume | |
| Thread Computation | EmailThread table with References/In-Reply-To/Message-ID parsing (IMAP/Gmail), native conversation_id (Outlook) |
Sources: README.md49-63 Diagram 1 from high-level architecture, Diagram 3 from event system, Diagram 4 from sync/caching
Understanding RustMailer requires familiarity with core concepts and data types:
Account and Provider Types:
AccountModel: Database entity representing email account configuration (src/modules/account/entity.rs)MailerType: Enum with variants ImapSmtp, GmailApi, GraphApi routing to provider implementationsOAuth2AccessToken: Managed token with auto-refresh for Gmail/Outlook authenticationImapConfig/SmtpConfig: Connection settings for traditional mail serversMessage Representation:
Envelope: Provider-agnostic message metadata (unified response type)EmailEnvelopeV3: IMAP envelope stored in native_db with mailbox_id + uid keyGmailEnvelope: Gmail message with message_id + label_ids, tracks history_idOutlookEnvelope: Graph API message with message_id + folder_id, uses delta tokensFullMessageContent: Parsed MIME content with headers, body parts, attachmentsEmailThread: Conversation grouping computed from References/In-Reply-To or native thread_idSynchronization:
AccountRunningState: Tracks sync progress (current_folder, batch_number, sync_type)SyncType: Enum with FullSync, IncrementalSync, SkipSync variantsGmailCheckPoint: Stores history_id per label for incremental Gmail syncFolderDeltaLink: Outlook delta token for change trackingreconcile_mailboxes: IMAP function comparing local vs remote state (UID_VALIDITY, UID_NEXT)Event System:
EventType: Enum of 12 event variants (EmailAddedToFolder, EmailFlagsChanged, etc.)EventHooks: Configuration for webhook/NATS delivery with watched_events listEventHookTask: Queued dispatch task with retry logic and VRL transformationEVENT_CHANNEL: Async MPSC queue for event propagationTask Management:
EmailTask: Queued email send operation with template rendering and retryTaskStatus: Enum (Pending, Running, Completed, Failed, Stopped)RetryPolicy: Configuration with RetryStrategy (Exponential, Linear) and max attemptsTASK_DATABASE: redb instance managing task persistenceStorage:
DISK_CACHE: Global cacache::Cache for content-addressed raw message storageRUSTMAILER_CONTEXT: Singleton with native_db::Database and provider executorsENVELOPE_BATCH_SIZE: Constant (100) for batch write operationsSources: src/modules/account/entity.rs src/modules/message/envelope.rs src/modules/cache/nativedb/ src/modules/hook/events.rs src/modules/scheduler/ src/modules/context/
RustMailer provides a unified API over three fundamentally different email protocols through the MailerType enum and provider-specific clients:
Diagram: Provider Abstraction and Protocol Routing
Sources: Diagram 2 from high-level architecture, src/modules/account/entity.rs src/modules/cache/vendor/imap/ src/modules/cache/vendor/gmail/ src/modules/cache/vendor/outlook/
| Aspect | IMAP/SMTP | Gmail API | Outlook/Graph API |
|---|---|---|---|
| Message IDs | UIDs (per-mailbox) | String message_id (global) | String message_id (global) |
| Organization | Folder hierarchy | Label-based (multi-label) | Folder hierarchy |
| Threading | Computed (References header) | Native thread_id | Native conversation_id |
| Sync Strategy | UID comparison + UID_VALIDITY | history_id + history records | Delta tokens + @odata.nextLink |
| Move Operation | UID COPY + EXPUNGE | Modify labels (remove + add) | REST API move endpoint |
| Authentication | Username/password | OAuth2 (googleapis.com) | OAuth2 (graph.microsoft.com) |
| Client Library | async-imap | reqwest | reqwest |
| Connection Model | Persistent IMAP connection | Stateless HTTP | Stateless HTTP |
All three providers convert their native data structures to the common Envelope type before returning to API clients, hiding protocol differences.
Sources: src/modules/cache/vendor/imap/ src/modules/cache/vendor/gmail/ src/modules/cache/vendor/outlook/ Diagram 2 from high-level architecture
RustMailer employs provider-specific sync strategies optimized for each protocol's change detection mechanism:
Diagram: Sync Type Decision and Execution Flow
Sources: Diagram 4 from high-level architecture, src/modules/cache/sync/imap_poller.rs src/modules/cache/vendor/gmail/sync/ src/modules/cache/vendor/outlook/sync/
Sync Type Selection Logic:
Full Sync triggers when:
minimal_sync flag is false and first syncIncremental Sync occurs when:
date_since windowMinimal Sync when:
minimal_sync=true flagSources: src/modules/cache/sync/imap_poller.rs src/modules/cache/vendor/gmail/sync/sync_worker.rs src/modules/cache/vendor/outlook/sync/sync_worker.rs
RustMailer's event system provides sophisticated real-time integration with external systems through conditional event production, VRL transformation, and dual dispatch mechanisms:
Diagram: Event Flow from Sources to External Systems
Sources: Diagram 3 from high-level architecture, src/modules/hook/task.rs src/modules/hook/events.rs src/modules/hook/vrl/
Event Types (12 total):
EmailAddedToFolder - New message syncedEmailFlagsChanged - Read/flag/star status updatedEmailSentSuccess - SMTP delivery succeededEmailSendingError - SMTP delivery failedUIDValidityChange - IMAP mailbox rebuiltMailboxDeletion - Folder/label removedMailboxCreation - Folder/label createdAccountFirstSyncCompleted - Initial sync finishedEmailBounce - Bounce notification receivedEmailFeedBackReport - Spam/abuse reportEmailOpened - Tracking pixel loadedEmailLinkClicked - Link in email clickedConditional Event Production:
Event sources call EventHookTask::is_watching_*() methods before creating events. This prevents expensive operations (e.g., fetching full message content) when no consumers are configured.
VRL Transformation: Each event passes through optional VRL scripts that can:
nullRetry Logic:
Failed dispatches trigger exponential backoff (base 2) up to 10 attempts, with per-hook status tracking in EventHooks entity.
Sources: src/modules/hook/task.rs43-160 src/modules/hook/events.rs src/modules/hook/vrl/ src/modules/hook/entity.rs
RustMailer's implementation uses the following crates and technologies:
| Category | Component | Version/Crate | Purpose | File Reference |
|---|---|---|---|---|
| Async Runtime | Tokio | 1.48.0 | Async I/O, task scheduling, channels | Cargo.toml49 |
| Web Framework | Poem | 3.1.12 | HTTP server, routing, middleware | Cargo.toml29 |
| poem-openapi | 5.1.16 | OpenAPI spec generation, Swagger/ReDoc/Scalar | Cargo.toml31-38 | |
| poem-grpc | 0.5.7 | gRPC services, compression | Cargo.toml39-44 | |
| Email Protocols | async-imap | 0.11.1 | IMAP client, compression support | Cargo.toml93-96 |
| mail-send | 0.5.2 | SMTP client, DSN support | Cargo.toml71 | |
| mail-parser | 0.11.1 | MIME parsing, header extraction | Cargo.toml70 | |
| HTTP Clients | reqwest | 0.12.24 | Gmail/Outlook API, rustls-tls, SOCKS proxy | Cargo.toml55-62 |
| Storage | native_db | 0.8.2 | Embedded KV store, envelope metadata | Cargo.toml25 |
| cacache | 13.1.0 | Content-addressed disk cache, tokio runtime | Cargo.toml87-90 | |
| redb | 2.6.2 | Alternate embedded DB | Cargo.toml26 | |
| OAuth2 | oauth2 | 5.0.0 | OAuth2 flow, rustls-tls, token refresh | Cargo.toml79-82 |
| Event System | vrl | 0.28.0 | Vector Remap Language for payload transform | Cargo.toml125 |
| async-nats | 0.45.0 | NATS client, stream publishing | Cargo.toml108 | |
| Auth/Crypto | min_jwt | 0.10.0 | JWT token generation/validation | Cargo.toml45 |
| ring | 0.17.14 | Cryptographic operations | Cargo.toml46 | |
| rustls | 0.23.35 | TLS implementation | Cargo.toml98 | |
| Monitoring | prometheus | 0.14.0 | Metrics collection and export | Cargo.toml127 |
| tracing | 0.1.41 | Structured logging | Cargo.toml50 | |
| Templates | handlebars | 6.3.2 | Email template rendering | Cargo.toml77 |
| Concurrency | bb8 | 0.9.0 | Connection pooling (SMTP) | Cargo.toml101 |
| dashmap | 6.1.0 | Concurrent hashmap | Cargo.toml126 | |
| lru | 0.16.2 | LRU cache | Cargo.toml109 | |
| Frontend | React | - | UI framework | web/package.json |
| Vite | - | Build tool | web/package.json | |
| Shadcn UI | - | Component library | web/package.json | |
| TanStack Query | - | Data fetching | web/package.json |
Build Configuration:
strip=true, lto=true, opt-level=3, codegen-units=1 for maximum optimizationvendored-openssl feature for static OpenSSL linkingbuild.rs using poem-grpc-buildSources: Cargo.toml1-141 README.md242-258
The following sequence diagram illustrates a typical message retrieval request showing the interaction between API layer, authentication, provider routing, dual cache lookup, and external IMAP server:
Diagram: Message Content Retrieval with Cache Hierarchy
Sources: src/modules/api/v1.rs src/modules/message/full.rs45-119 src/modules/cache/disk.rs33-90 src/modules/context/executors/imap.rs src/modules/context/client.rs
This flow demonstrates:
ClientContext::require_account_accessMailerType enumcacache) before remote fetchRustMailer can be deployed via Docker or built from source. See Getting Started for detailed installation instructions.
Docker (Recommended):
Source Build:
Configuration Options:
RUSTMAILER_ROOT_DIR: Data directory (required)RUSTMAILER_HTTP_PORT: REST/Web UI port (default: 15630)RUSTMAILER_GRPC_PORT: gRPC port (default: 16630)RUSTMAILER_PUBLIC_URL: Callback URL (default: http://localhost:15630)Access:
Default Authentication: root / root (changeable via Web UI)
Licensing:
Sources: README.md76-121 README.md196-231 src/modules/settings/cli.rs
For deeper understanding of RustMailer's systems:
RustMailer's architecture prioritizes reliability, performance, and developer experience through strong typing, comprehensive error handling (src/modules/error/), and extensive observability via Prometheus metrics.
Refresh this wiki