In the past few years I've gone through a few self-hostable decentralized1 end-to-end encrypted messaging apps, including Matrix↗ (post), Snikket (XMPP)↗ (post), and SimpleX↗. I was really excited about each in turn, but lost enthusiasm over time for one reason or another. I think my fondness for SimpleX lasted the longest, but that did eventually fade as well2.
At about the time I was really starting to lose interest in SimpleX, I saw @jcrabapple proclaim↗, "#DeltaChat is cool as hell!" along with an invitation to join him on the platform. I had heard of Delta Chat↗ before but had never really given it a fair shake, perhaps having (unfairly) dismissed it outright upon learning it uses email as the underlying delivery mechanism↗. I decided to check it out, though, and quickly came away impressed↗.
In a way, building a decentralized encrypted messenger atop tried-and-true email infrastructure makes a lot of sense, especially with the specialized chatmail↗ relays optimized for this exact use case with anonymous automated account creation, mandatory end-to-end encryption, strict security enforcement, and very fast delivery. The Delta Chat community runs several public chatmail relays↗, and by default new users get an account on nine.testrun.org. But why would I use someone else's chatmail relay when self-hosting my own↗ is an option3?
After a few months of experimenting with Delta Chat itself (followed by a mandatory waiting period↗), I finally set up a chatmail server of my own: chat.vim.wtf↗. If you're reading this4, you're welcome to create an account on this server and/or say hi to me↗5.
What follows is a (hopefully) quick overview of what it took to get this thing up and running.
Chatmail relay overview
Unlike most of the things I've self-hosted lately, chatmail↗ is not distributed as a tidy little Docker image. That's initially a little disappointing, but then I took a moment to appreciate the complexity of the thing I was about to deploy:
The components of chatmail are:
- Postfix SMTP MTA accepts and relays messages (both from your users and from the wider e-mail MTA network)
- Dovecot IMAP MDA stores messages for your users until they download them
- Nginx shows the web page with your privacy policy and additional information
- acmetool manages TLS certificates for Dovecot, Postfix, and Nginx
- OpenDKIM for signing messages with DKIM and rejecting inbound messages without DKIM
- mtail for collecting anonymized metrics in case you have monitoring
- Iroh relay which helps client devices to establish Peer-to-Peer connections
and the chatmaild services, explained in the next section:
chatmaild implements various systemd-controlled services that integrate with Dovecot and Postfix to achieve instant-onboarding and only relaying OpenPGP end-to-end messages encrypted messages. A short overview of chatmaild services:
- doveauth implements create-on-login address semantics and is used by Dovecot during IMAP login and by Postfix during SMTP/SUBMISSION login which in turn uses Dovecot SASL to authenticate logins.
- filtermail prevents unencrypted email from leaving or entering the chatmail service and is integrated into Postfix's outbound and inbound mail pipelines.
- chatmail-metadata is contacted by a Dovecot lua script to store user-specific relay-side config. On new messages, it passes the user's push notification token to notifications.delta.chat so the push notifications on the user's phone can be triggered by Apple/Google/Huawei.
- delete_inactive_users deletes users if they have not logged in for a very long time. The timeframe can be configured in chatmail.ini.
- lastlogin is contacted by Dovecot when a user logs in and stores the date of the login.
- echobot is a small bot for test purposes. It simply echoes back messages from users.
- chatmail-metrics collects some metrics and displays them at https://example.org/metrics↗.
There are a lot of pieces there, and trying to put them all together in a single container image (or even stack) seems like it would be quite the challenge. But then, so would trying to configure them all manually, right?
Fortunately, the chatmail repo is built to take care of every aspect of installing, configuring, and even updating/maintaining these components using pyinfra↗. I'll clone the repo locally, create a simple configuration file, and then use the bundled scripts to automagically deploy everything to my target server6. Speaking of which...
Initial server setup
I opted to create a new Ubuntu 24.04 server with 2vCPUs, 4GB RAM, and 40GB storage hosted by Hetzner↗ in Germany, which will cost me less than $5 per month. This CX22 is the smallest server which Hetzner offers and is a bit overkill for this specific project, as the docs indicate "Chatmail relay servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a few thousand active chatmail addresses." But I had recently been moving more workloads to Hetzner and didn't feel like opening an account with another provider to try for a smaller server.
While creating the server, I made sure to add my SSH public key so that I'll be able to log in as root. And then I set to work creating the cloud firewall exceptions which will be needed for chatmail functionality:
| Program | Port | Protocol |
|---|---|---|
| Postfix | 25 | SMTP |
| Postfix | 587 | SUBMISSION |
| Postfix | 465 | SUBMISSIONS |
| Dovecot | 143 | IMAP |
| Dovecot | 993 | IMAPS |
| Nginx | 8443 | HTTPS-ALT |
| Nginx | 443 | HTTPS |
| acmetool | 80 | HTTP |
Blocked email ports
I found out the hard way↗ that most cloud providers (including Hetzner↗) block outbound traffic on ports 25 and 465. Fortunately Hetzner allows you to submit a request to have those blocks removed, but you've got to be a paying customer for thirty days first. Other providers likely have similar policies.
Just be sure to check and sort that out if needed before trying to spin up your own chatmail server. I wasted a lot of time trying to debug why I could receive but not send messages on my first attempt...
While configuring the firewall I also restricted SSH on port 22 to just my home IP to prevent it from being targeted by bad guys.
I then copied the new server's public IP address and verified that I could log into it as root without a password using the supplied key, and did the usual update dance followed by a reboot:
export SERVER_IP=192.0.2.1 # replace with the server's actual IP ssh root@$SERVER_IP apt-get update && apt-get upgrade -y reboot nowAnd that's it for the server prep. But IP addresses are clumsy, so let's create some handy DNS records next.
DNS records (part 1)
Email servers need a lot of DNS records, and fortunately the chatmail setup will let me know about all of them. But I'll go ahead and create a few to get started based on the documentation in the chatmail repo↗:
| Record | Type | Value |
|---|---|---|
chat.vim.wtf | A | 192.0.2.1 (the server's public IPv4 address) |
chat.vim.wtf | AAAA | 2001:db8::5 (the server's public IPv6 address) |
www.chat.vim.wtf | CNAME | chat.vim.wtf. |
mta-sts.chat.vim.wtf | CNAME | chat.vim.wtf. |
I'm using my DNSControl GitOps setup7 for creating and managing these records; being able to write all the changes to a file, test and preview the changes, apply them all at once - and quickly roll back the changes if there's an unexpected problem - is such a cool capability. This will really come in handy when I get into the more complicated SRV and TXT records later on.
Once the records are in place, I verify that I can log into the server with its new public name (which requires accepting its public key again):
The authenticity of host 'chat.vim.wtf (192.0.2.1)' cannot be established. This host key is known by the following other names/addresses: ~/.ssh/known_hosts:394: 192.0.2.1Are you sure you want to continue connecting (yes/no/[fingerprint])? yesWarning: Permanently added 'chat.vim.wtf' (ED25519) to the list of known hosts.Welcome to Ubuntu 24.04.2 LTS (GNU/Linux 6.8.0-57-generic x86_64)logout Deployment prep
Okay, I've prepped the server and I've prepped the initial set of DNS records. Now let's prep my deployment environment.
That starts by cloning the chatmail/relay repo to my local computer:
git clone https://github.com/chatmail/relay cd relayI'm going to use a Nix devshell↗ to install the required build tools rather than installing them globally and/or fumbling with virtual environments. So I add this flake.nix to the relay folder:
{ description = "chatmail devshell"; inputs = { nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; }; outputs = { self, nixpkgs }: let pkgs = import nixpkgs { system = "x86_64-linux"; }; in { devShells.x86_64-linux.default = pkgs.mkShell { packages = with pkgs; [ gcc libgcc python3 stdenv.cc.cc.lib ]; LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib"; }; };}Then git add flake.nix so that it will be tracked locally, and make the following .envrc file so that direnv↗ will load the shell automatically for me:
#!/usr/bin/env direnvuse flake .I then activate the shell with direnv allow and wait a moment while the required components get installed.
For doing this without the awesomeness complexity of Nix, I think sudo apt-get install build-essential python3-dev should do the trick - the setup script will alert if there's anything else missing.
In any case, once the required tools are in place I can run this handy-dandy script to stage all of the stuff that the chatmail deployment will need:
scripts/initenv.sh [...] # it does a LOT of things here Successfully installed cmdeploy-0.2 # but ends with this successAnd then I create the default config file:
scripts/cmdeploy init chat.vim.wtf created config file for chat.vim.wtf in chatmail.ini That chatmail.ini file contains all the configuration options I may potentially want to worry about. The only crucial piece is setting the mail_domain value, which was handled automatically when I passed my domain to the init command. There are some other options worth tweaking as well if you want to allow really short usernames or require really long passwords. At the bottom of the file there are a handful of privacy_ contact options. I added a newly-generated Cloaked↗ email address to the privacy_mail field so that I can be contacted with any potential questions/concerns without opening up one of my personal addresses to spammers, and leave the rest of them blank.
The www/src directory includes some Markdown files used to generate a simple landing web page for the chatmail server, and those include formatting templates for pulling in a few key values from the chatmail.ini config file, like the password length requirements and the privacy contact information. Since I'm only providing a privacy email address, I edited my copy of www/src/privacy.md to remove references to other types of contacts.
I'm now ready to run the deployment!
Deployment process
With all the prep out of the way, the actual deployment is pretty easy. Just run cmdeploy run and let it do its thing:
scripts/cmdeploy run [ssh] login to chat.vim.wtf Collecting initial DNS settings..............[$ pyinfra --ssh-user root chat.vim.wtf /home/john/projects/relay/cmdeploy/src/cmdeploy/deploy.py -y]--> Loading config...--> Loading inventory...--> Connecting to hosts... [chat.vim.wtf] Connected [...] --> Disconnecting from hosts...Deploy completed, call `cmdeploy dns` next. This script runs for a good long while, as it installs, configures, and starts all of the chatmail components that were mentioned above. Even once the process ends we're not quite finished, though, but that last line provides instructions for the next step.
DNS records (part 2)
Running cmdeploy dns will check for the existing DNS records and generate a list of the ones which are missing or incorrect:
scripts/cmdeploy dns WARNING: these recommended DNS entries are not set: chat.vim.wtf. TXT "v=spf1 a:chat.vim.wtf ~all"_dmarc.chat.vim.wtf. TXT "v=DMARC1;p=reject;adkim=s;aspf=s"chat.vim.wtf. CAA 0 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/4815162342"_adsp._domainkey.chat.vim.wtf. TXT "dkim=discardable"_submission._tcp.chat.vim.wtf. SRV 0 1 587 chat.vim.wtf._submissions._tcp.chat.vim.wtf. SRV 0 1 465 chat.vim.wtf.[...] # and so on....After carefully creating each of those newly required records in my DNS provider, I can recheck:
scripts/cmdeploy dns [ssh] login to chat.vim.wtf Collecting initial DNS settings..............Check expected zone file entries..............................................Great! All your DNS entries are verified and correct.Huzzah!
Validation
Now that the server is deployed and all of the DNS records are in place, how can I be sure that it actually works?
The chatmail team thought of that, too, and bundled in an automated test suite to confirm that things are all (likely) working correctly:
scripts/cmdeploy test [$ /home/john/projects/relay/venv/bin/pytest cmdeploy/src/ -n4 -rs -x -v --durations=5] ====================================== test session starts ======================================platform linux -- Python 3.12.9, pytest-8.3.5, pluggy-1.5.0 -- /home/john/projects/relay/venv/bin/python3cachedir: .pytest_cacheDeltachat core=v1.155.6 sqlite=3.45.3 journal_mode=walrootdir: /home/john/projects/relay/cmdeployconfigfile: pyproject.tomlplugins: deltachat-rpc-client-1.157.2, typeguard-4.4.2, xdist-3.6.1, deltachat-1.155.6, chatmaild-0.2, cmdeploy-0.24 workers [45 items]scheduling tests via LoadScheduling cmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_login_same_password[imap]cmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_concurrent_logins_same_accountcmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_initcmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_login_basic_functioning[imap][gw0] [ 2%] PASSED cmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_initcmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_capabilities[gw0] [ 4%] PASSED cmdeploy/src/cmdeploy/tests/online/test_0_login.py::test_capabilitiescmdeploy/src/cmdeploy/tests/online/test_0_qr.py::test_gen_qr_png_data[gw0] [ 6%] PASSED cmdeploy/src/cmdeploy/tests/online/test_0_qr.py::test_gen_qr_png_datacmdeploy/src/cmdeploy/tests/online/test_0_qr.py::test_fastcgi_working[...][gw2] [ 97%] SKIPPED cmdeploy/src/cmdeploy/tests/online/test_2_deltachat.py::TestEndToEndDeltaChat::test_securejoin[gw1] [100%] PASSED cmdeploy/src/cmdeploy/tests/online/test_2_deltachat.py::test_echobot[...]================================== 40 passed, 5 skipped in 49.34s ============================== So it passes the automated test suite, but does it actually work?
Using the new server
The bundled landing page (which I'm now serving at chat.vim.wtf↗) provides a link and a QR code that I can follow or scan on a device with the Delta Chat↗ app installed to create a new chat profile on the new chatmail server. Doing so prompts me to enter a display name (and optionally add a profile picture), and that's it8 - I'm soon presented with my new randomly-generated9 profile.
Delta Chat makes managing multiple profiles on the same device super easy, and you can receive notifications for any profile on the device rather than just the currently-active one. This made testing my new setup a breeze since I could just exchange messages between my new chat.vim.wtf profile and my initial profile on the default nine.testrun.org relay.
Talking to myself is great and all, but I'd love to chat with you, too:
Signal↗ is great, but I'm not wild about its centralized nature. Having all user traffic routed through servers under the control of a single organization makes that organization an attractive target to adversaries (I'm talking nation-states) and the traffic easier to block or disrupt. There are also plenty of other things I don't love about Signal, like its crypto-currency integration, phone number requirement, and eagerness to hoover up your phone's contacts so it can let your contacts know you're using Signal. Signal's encryption model is hard to beat, but these anti-features leave me thankful there are more open alternatives. ↩︎
It seems like it's very easy to fragment a group chat when users are connected through a variety of different servers. If one server stops processing messages for a bit then your keys can fall out of sync and you'll never exchange messages with those existing connections again. And unlike Matrix's dreaded "unable to decrypt" error, you probably won't even notice unless someone else in the group remarks on how you seem to be missing half of the conversation. It's happened to me frequently enough that it really soured me on SimpleX. So while I still think it's great for one-on-one chats, I think it's a bit too delicate for groups. ↩︎
It's possible that I have a self-hosting addiction. ↩︎
You're awesome. ↩︎
I'm treating this as a public profile (since I'm linking it publicly and using a deliberately created username rather than a random one) and use it primarily for "first contact" situations. Once we're connected and have independently verified each other I'll probably move any conversation to an unlisted/random/more private profile. ↩︎
Of course, the lack of containerization for this deployment does mean that I should probably dedicate an entire server to the job rather than trying to add a chatmail role to an existing one. ↩︎
Though I'm now using Forgejo instead of GitHub↗. ↩︎
I don't have to provide another email account, non-VoIP phone number, physical or mailing address, payment information, or any "security" questions and answers. ↩︎
Delta Chat automatically generates a random username and password when it creates the account on the server, along with the corresponding encryption keys. ↩︎
