Skip to content

check-for-updates monitors SSL files#976

Closed
pn2 wants to merge 1 commit intodocker-mailserver:masterfrom
pn2:master
Closed

check-for-updates monitors SSL files#976
pn2 wants to merge 1 commit intodocker-mailserver:masterfrom
pn2:master

Conversation

@pn2
Copy link
Copy Markdown

@pn2 pn2 commented May 20, 2018

These changes to check-for-updates allow for the monitoring of the SSL certificates. The names of these files are extracted from the configuration files. If the SSL files change, both postfix and dovecot are restarted.

The generation and checking of checksums has been moved to two functions. Also, the check function creates an array with the results from the sha512 check (last word of each line) rather than indexing the result string with character offsets. This seems more portable and robust to me.

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented May 21, 2018

If I understand this PR correctly, I could have another container run every week or two which would extract certs from Traefik's JSON file and overwrite the existing ones. If the files have not changed, the SHA512 check will pass, and no restart will be necessary. If the files have changed, the mail software automatically restarts, and it's all good. Right?

@pn2
Copy link
Copy Markdown
Author

pn2 commented May 22, 2018

@mathuin : Yes, this is the idea. I would monitor the acme.json file continuously (on the host or another container), either using inotify or every few second or minute with a script. If acme.json changed, then the script extracts the .pem files. These changes then are detected by docker-mailserver and dovecot and postfix restarted. It also works for manually generated keys. Changes are detected, and dovecot and postfix restarted.

@johansmitsnl
Copy link
Copy Markdown
Contributor

Can you verify why the tests fail?

@pn2
Copy link
Copy Markdown
Author

pn2 commented Jun 24, 2018

Problem is I don't know what to take from the test output. If I'd understand more specifically what goes wrong in the test and how to reproduce it, I'd be happy to inspect this further.

The script works flawlessly in my setup.

@johansmitsnl
Copy link
Copy Markdown
Contributor

@pn2 the issue looks like the smtp server is not running and therefore it fails to deliver the test data to the dockers.

When you run the tests locally on your system you should see that the dockers are not running as they should.

You can test this by running: make build-no-cache generate-accounts run generate-accounts-after-run fixtures tests

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented Aug 4, 2018

The expiration of my certs reminded me to check on this. Is anyone else moving forward?

@johansmitsnl
Copy link
Copy Markdown
Contributor

@mathuin I guess we could use some help on this. Would you mind to assist?

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented Aug 5, 2018

I'm willing to try, but my master's thesis is due in less than two weeks as I am defending at the end of the month. That being said, I am hopeful that the travis failure is due to something simple so I'm giving it a look today before going back to writing.

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented Aug 8, 2018

The tests are non-deterministic when I run them in my home directory, but when I run them on local storage they work fine.

An informal audit of the pull request shows no new test for the new functionality, which I'd like to address first. I also think the implementation as written doesn't quite work, which I can prove once that test is written. I have an alternative approach involving associative arrays which I will try if the existing code does not work. If all goes well, I should have something for you soon. Thank you!

@pn2
Copy link
Copy Markdown
Author

pn2 commented Aug 8, 2018

I also think the implementation as written doesn't quite work, which I can prove once that test is written?

Why is that? It certainly runs fine on my docker system and has recognized the certificate changes and reloaded the mail server components

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented Aug 8, 2018

I didn't have as much time as I would like, but here's the test I tried last night:

@test "checking ssl: changing cert updates correctly" {
  cp test/config/letsencrypt/mail.my-domain.com/fullchain.pem test/config/letsencrypt/mail.my-domain.com/fullchain.pem-backup
  run docker exec mail_manual_ssl /bin/sh -c "diff /etc/postfix/ssl/cert /tmp/docker-mailserver/letsencrypt/mail.my-domain.com/fullchain.pem"
  assert_success
  cp test/config/letsencrypt/mail.my-domain.com/chain.pem test/config/letsencrypt/mail.my-domain.com/fullchain.pem
  run docker exec mail_manual_ssl /bin/sh -c "sleep 5 && diff /etc/postfix/ssl/cert /tmp/docker-mailserver/letsencrypt/mail.my-domain.com/fullchain.pem"
  assert_failure
  mv test/config/letsencrypt/mail.my-domain.com/fullchain.pem-backup test/config/letsencrypt/mail.my-domain.com/fullchain.pem
}

If my test is poorly written (very possible), please let me know. The one thing which might be ambitious here is the sleep timing -- 15 or 20 seconds may be more appropriate, but I was wary of adding too much time to tests that already run for fifteen minutes at a time. I also don't like leaving unnecessary files hanging about on the filesystem, so I've reached out to the bats folks to see if there's a setup/teardown facility for a single test.

I do not think you did a bad job. I completely believe you when you say that it runs fine on your system.
I'm just saying that after applying your changes, this test does not pass and some additional tests which previously passed fail intermittently which implies that the patch has some other side effects, which falls under "doesn't quite work" in my book. I apologize if I was unclear.

For what it's worth, your patch taught me a new Unix command, something which hasn't happened in a long time. I would have used sed -e "s/.*: //" as I lean heavily on sed and awk but now that I know about rev I'll be looking for other opportunities to use it.

ETA: used current version of test, not previous broken one which assumed /etc/postfix/ssl/cert was a directory, not a file.

@mathuin
Copy link
Copy Markdown
Contributor

mathuin commented Sep 30, 2018

My mail server reminded me this weekend that I had not yet resolved this issue.

I have an alternative solution which runs in parallel to check-for-updates.sh instead of extending it. The benefits of a separate script include running for all relevant users (check-for-updates.sh currently runs for all folks not using LDAP regardless of whether they use SSL or not) and the risks of a separate script include "battling restarts" where one script interferes with the other.

Anyway, the aforementioned alternative solution does not fail any existing tests, but I have not written any tests or documentation yet. I'm hoping to do that this week -- but writing tests for this is harder than writing the code, as I think the PR's original author might understand. :-) If I finish tests and documentation before the PR's original author fixes the build here, I'll submit a PR and let the maintainers decide.

@SnowyLeopard
Copy link
Copy Markdown

SnowyLeopard commented Apr 28, 2019

What's the status on this PR? I see the related PR is closed.
Would very much like to see this to be implemented :). Also offering any help if needed.

@erik-wramner
Copy link
Copy Markdown
Contributor

A best practice for docker in general is to rebuild the image and restart the container to use the new version on a regular basis. We don't apply security updates in the container, so the only way to get them is to rebuild it. Thus there should be a new image version and a restart fairly often, perhaps daily. With that background I don't think this is very useful? It is great for a long-running container, but to really have a long-running container we would need to run apt-get update/upgrade as well.

The image is not rebuilt regularly yet, but that is something I would like to fix.

@erik-wramner
Copy link
Copy Markdown
Contributor

Having thought about it I realize this could be useful to many, so I will be happy to merge it with some changes:

  • Rebase from current master as there have been changes in the script and as the tests are now successful, making it possible to see if errors originate with the change or not. Note that the changes include a lock to prevent concurrent updates when there are multiple containers running.
  • Add basic tests, at least one where the script is triggered
  • Check the environment variables and skip the SSL checks for manual, self-signed etc where they are not needed (not critical, can be done later)
  • Update the README with notes about how this works (not critical, can be done later)

Anyone?

@marvincaspar
Copy link
Copy Markdown

Are there any updates? I'm looking forward to this feature

@pn2
Copy link
Copy Markdown
Author

pn2 commented Feb 23, 2020

@marvincaspar I solved the issue by having a cron job that checks for certificate changes. If these happen, it exports the certs and then runs

docker exec -ti mail supervisorctl restart postfix
docker exec -ti mail supervisorctl restart dovecot

This has worked well so far.

I agree that having the restart withing the docker image would be better, but I don't have the time to do that and what I tried failed the tests for reasons I did not understand.

@Its-Alex
Copy link
Copy Markdown

Up ?

@erik-wramner
Copy link
Copy Markdown
Contributor

Nothing has happened for years, I'll close this one. Hopefully we can get #1553 out instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants