Skip to content

Persist mail state directories#195

Merged
tomav merged 2 commits intodocker-mailserver:masterfrom
tve:persistence
Jun 1, 2016
Merged

Persist mail state directories#195
tomav merged 2 commits intodocker-mailserver:masterfrom
tve:persistence

Conversation

@tve
Copy link
Copy Markdown
Contributor

@tve tve commented May 24, 2016

This PR consolidates all directories with mail state into /var/mail-state, which can then be mounted as a volume. This is to address #191.

This PR is based off the changes in PR 194, please look at the one new commit 4ca39f9 for the changes.

If you like this approach, I will add docs to the Wiki. Dunno whether you want this to be in the docker-compose.yaml.dist or not. Also, I changed the tests to use the consolidated directories, but I don't really know how you want to proceed, so this PR is a bit of a WIP 'til I hear back...

@Josef-Friedrich
Copy link
Copy Markdown
Contributor

For now I store my mails in a local directory. @tve Is it possible to mount a local directory to this persistent state directory too?

@Josef-Friedrich
Copy link
Copy Markdown
Contributor

I would be very happy if this PR could be merged.

@tomav tomav added the wip label May 24, 2016
@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 24, 2016

@Josef-Friedrich if by "local directory" you mean a directory on the docker host, then yes, you can do that. My docker-compose-fu isn't well developed yet, but for sure it works if you use something like the following in your docker-compose.yaml:

services:
  mail:
    ...
    volumes:
    - /mnt/mail-state:/var/mail-state

Here /mnt/mail-state is a directory on the host. You can then back this dir up. I haven't figured out how to do it with the named volumes syntax.

@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 24, 2016

Docs for the wiki (waiting to hear from @tomav whether he likes the direction of this PR):

Configuring Backups

User mail directories

Assuming that you use docker-compose and a data volume container named maildata, you can backup your user mails like this:

docker run --rm \
--volumes-from maildata_1 \
-v "$(pwd)":/backups \
-ti tvial/docker-mailserver \
tar cvzf /backups/docker-mailserver-`date +%y%m%d-%H%M%S`.tgz /var/mail

Mail state directories

In addition to the user mail directories you may want to persist and even back up the state of the various daemons, such as postfix mail queues, clamav virus database, spamassassin learning, etc. Inside the container the files for this are normally found in various directories under /var/lib and /var/spool but by adding the ONE_DIR=1 option to the environment these all get moved to /var/mail-state at container start-up. This allows you to mount a volume there, which has the first benefit that all this state is persisted if you need to restart the container. As a second benefit, you can also back this volume up to secondary storage, although it is not really required, i.e., no email will get lost unless something was just in transit in the postfix queues.

In docker-compose.yaml you can enable the merging of all daemon state into one directory and the mounting of a volume as follows:

services:
  mail:
    ...
    environment:
      ...
    - ONE_DIR=1
    volumes:
    ...
    - mailstate:/var/mail-state

volumes:
  ...
  mailstate:
    driver: local

This configuration persists the mailstat volume across container restarts. If you want to back mailstate up to secondary storage you need to launch a backup container as suggested above for maildata backups. Alternatively, you can also mount one or both volumes from a specific path on the host. To do that do not use the named volumes syntax and instead use something like:

services:
  mail:
    ...
    environment:
      ...
    - ONE_DIR=1
    volumes:
    - /on/the/host/mail-data:/var/mail
    - /on/the/host/mail-state:/var/mail-state

This second form is useful if you want to place these volumes onto a filesystem/device where you perform snapshot backups or something similar.

@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 24, 2016

I'm having trouble getting a build to complete. Travis times out just building the container images :-(

@Josef-Friedrich
Copy link
Copy Markdown
Contributor

The download of the main.cvd is very slow. I'm just building the image on my laptop ...

@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 24, 2016

Yeah, I did set things up so I could build and test locally, and there the tests pass. It's just not nice to have a PR where "all checks have failed"...

@tomav
Copy link
Copy Markdown
Contributor

tomav commented May 25, 2016

@tomav
Copy link
Copy Markdown
Contributor

tomav commented May 26, 2016

A question: why don't we change where these daemons write their content to the single folder, using configuration files, instead of adding new code in start-mailserver.sh.

@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 26, 2016

That's certainly a very valid alternative. I didn't want to dive into the documentation for the 7 different daemons to find what I had to change and then test that I didn't forget something, etc...

@tomav
Copy link
Copy Markdown
Contributor

tomav commented May 28, 2016

I understand, but there's another easier way by adding the folder list to a data container.

version: '2'

services:
  mail:
    image: tvial/docker-mailserver:latest
    # build: .
    hostname: mail
    domainname: domain.com
    container_name: mail
    ports:
    - "25:25"
    - "143:143"
    - "587:587"
    - "993:993"
    volumes:
    - maildata:/var/mail
    - maildata:/var/spool/postfix
    - maildata:/var/lib/postfix
    - maildata:/var/lib/amavis
    - maildata:/var/lib/clamav
    - maildata:/var/lib/spamassasin
    - maildata:/var/lib/fail2ban
    - ./config/:/tmp/docker-mailserver/

volumes:
  maildata:
    driver: local

No code, and it's more docker compliant.

@tve
Copy link
Copy Markdown
Contributor Author

tve commented May 29, 2016

Interesting, I didn't know one could do that. I guess no PR needed in that case? I'll keep using my fork because I want to map these directories to an EC2 EBS volume so I can do snapshot-backups and I can't do that easily with docker volumes, but that may not be of interest to many other folks.

@Josef-Friedrich
Copy link
Copy Markdown
Contributor

@tomav I tried your docker-compose.yml file from above. I got this error:

mail | May 29 19:45:31 mail postfix/master[642]: fatal: open lock file /var/lib/postfix/master.lock: cannot create file exclusively: Permission denied

@tomav
Copy link
Copy Markdown
Contributor

tomav commented Jun 1, 2016

There are some ownerships/permissions to set.
Let's gonna list them here.

- maildata:/var/spool/postfix
- maildata:/var/lib/postfix => postfix:postfix
- maildata:/var/lib/amavis => amavis:amavis with default directories (tmp and db)
- maildata:/var/lib/clamav => clamav:clamav
- maildata:/var/lib/spamassasin
- maildata:/var/lib/fail2ban

@tomav
Copy link
Copy Markdown
Contributor

tomav commented Jun 1, 2016

No time for this right time.

I'm gonna merge your PR and will try to do something more docker compliant later.

@tomav tomav merged commit 5232935 into docker-mailserver:master Jun 1, 2016
@tomav
Copy link
Copy Markdown
Contributor

tomav commented Jun 1, 2016

Thanks for the work anyway!

RichardFevrier pushed a commit to RichardFevrier/docker-mailserver that referenced this pull request Aug 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants