Skip to content

chore(encryption): support for passworded user email storage encryption keys#2080

Closed
NorseGaud wants to merge 9 commits intodocker-mailserver:masterfrom
NorseGaud:issues/2058
Closed

chore(encryption): support for passworded user email storage encryption keys#2080
NorseGaud wants to merge 9 commits intodocker-mailserver:masterfrom
NorseGaud:issues/2058

Conversation

@NorseGaud
Copy link
Copy Markdown
Member

@NorseGaud NorseGaud commented Jul 9, 2021

Description

  1. Allows users to ENABLE_PER_USER_STORAGE_ENCRYPTION in their docker-compose to enable the feature
  2. Within the various password impacting scripts:
    • Notice mail_crypt is enabled and, by default, generated a key for the user on creation.
    • Notice mail_crypt is enabled and, by default, update encryption password when email password is changed.
    • Notice mail_crypt is enabled and, by default, when updating the user's password, create user keys/encryption if there aren't any yet.
  3. Allow users to change the mail_crypt_curve with PER_USER_STORAGE_ENCRYPTION_CURVE and default to secp521r1
  4. Allow users to change the scheme with PER_USER_STORAGE_ENCRYPTION_SCHEME and default to CRYPT

Related issue: #2058

Related: #2134

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (non-breaking change that does improve existing functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Copy link
Copy Markdown
Member

@wernerfred wernerfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall: nice addition!

Please see the comments in the review below.

Also please make sure to mirror the changes you introduced to setup.sh to the corresponding docs site that it reflects the current state.

Waiting for reply and further reviews 👀

Comment thread docs/content/config/security/mail_crypt.md Outdated
Comment thread setup.sh Outdated
Comment thread target/bin/updatemailuser Outdated
Comment thread docs/content/config/security/mail_crypt.md Outdated
@NorseGaud
Copy link
Copy Markdown
Member Author

NorseGaud commented Jul 9, 2021

Screen Shot 2021-07-09 at 11 34 31 AM

(NO LONGER RELEVANT)

@NorseGaud
Copy link
Copy Markdown
Member Author

@wernerfred

Also please make sure to mirror the changes you introduced to setup.sh to the corresponding docs site that it reflects the current state.

will do!

@NorseGaud
Copy link
Copy Markdown
Member Author

Current state: I want to avoid writing tests for now until the overall changes I made are reviewed by a few people. That way I'm minimizing time spent on rewrites.

@georglauterbach
Copy link
Copy Markdown
Member

Accidentally rebased again. I hope this is fine this time, otherwise excuse me again and force-push over it 🙈

@wernerfred
Copy link
Copy Markdown
Member

Bad habits never die 😅

@polarathene
Copy link
Copy Markdown
Member

I don't mind reviewing, but will be busy until early next week. Unfortunately no time until then due to a deadline.

@NorseGaud
Copy link
Copy Markdown
Member Author

There is no rush for this

Copy link
Copy Markdown

@DavyLandman DavyLandman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just read a few pieces, really nice that you are integrating it into the existing scripts!

Comment thread docs/content/config/security/mail_crypt.md Outdated
Comment thread docs/content/config/security/mail_crypt.md Outdated
Comment thread docs/content/config/security/mail_crypt.md Outdated
Comment thread target/bin/addmailuser Outdated
@NorseGaud
Copy link
Copy Markdown
Member Author

An opportunity for a PR if someone has time: check-for-changes.sh doesn't start on the very first start of the container and if no email is created. Where there is needed improvement is for the setup-stack.sh to loop at Unless using LDAP, you need at least 1 email account to start Dovecot instead of immediately return an error and proceed with the setup once the user creates the first email.

Comment thread docs/content/config/security/mail_crypt.md Outdated
@DavyLandman
Copy link
Copy Markdown

DavyLandman commented Jul 13, 2021 via email

@NorseGaud NorseGaud changed the title issues/2058 : support for user encryption keys [WIP] issues/2058 : support for user encryption keys Jul 19, 2021
@georglauterbach georglauterbach mentioned this pull request Jul 21, 2021
@wernerfred wernerfred added the meta/feature freeze On hold due to upcoming release process label Jul 30, 2021
@wernerfred wernerfred modified the milestones: v10.1.0, v10.2.0 Jul 30, 2021
@georglauterbach georglauterbach removed the meta/feature freeze On hold due to upcoming release process label Aug 10, 2021
@polarathene polarathene reopened this Nov 12, 2021
@github-actions
Copy link
Copy Markdown
Contributor

Documentation preview for this PR is ready! 🎉

Built with commit: fc6a4b2

@DavyLandman
Copy link
Copy Markdown

Is there a workaround that might be fine for now? (One that avoids destroying containers Everytime you add a new user?)

@NorseGaud
Copy link
Copy Markdown
Member Author

Is there a workaround that might be fine for now? (One that avoids destroying containers Everytime you add a new user?)

I would appreciate if someone else could pick this up and contact dovecot. I've recently become too overwhelmed with travel, work, and another side project to continue with this.

@polarathene
Copy link
Copy Markdown
Member

I'll take it over, but it is low priority for me, I should be able to work on it by december hopefully 👍

Once the hostname and related changes I've been sorting out are all merged, a v10.3 release can be done and this PR should be able go from there for v11 release.

@polarathene polarathene added the stale-bot/ignore Indicates that this issue / PR shall not be closed by our stale-checking CI label Nov 16, 2021
@georglauterbach georglauterbach added the meta/feature freeze On hold due to upcoming release process label Nov 16, 2021
@georglauterbach georglauterbach removed the meta/feature freeze On hold due to upcoming release process label Dec 16, 2021
@polarathene polarathene removed this from the v11.0.0 milestone Apr 4, 2022
@polarathene polarathene added this to the v11.2.0 milestone Jun 10, 2022
@polarathene polarathene removed the meta/help wanted The OP requests help from others - chime in! :D label Jun 10, 2022
@georglauterbach georglauterbach modified the milestones: v11.2.0, v12.0.0 Jun 15, 2022
@polarathene polarathene modified the milestones: v12.0.0, v11.2.0 Aug 6, 2022
@georglauterbach georglauterbach modified the milestones: v11.2.0, v12.0.0 Oct 5, 2022
@georglauterbach
Copy link
Copy Markdown
Member

I'd close this as is, but I feel like at least two persons will reach out and tell me this will be worked on. Hence, I will just move it to v13.0.0, because it is unlikely going into v12.0.0.

@georglauterbach georglauterbach modified the milestones: v12.0.0, v13.0.0 Mar 3, 2023
@DavyLandman
Copy link
Copy Markdown

DavyLandman commented Mar 3, 2023

I don't know if it helps, but I've been using a setup like this. Using only user-patch.sh changes and some volume mounts overwrites :)

The encrypted mails are working quite nicely. Except I didn't do (yet) @polarathene idea of using SHA512-CRYPT on the password.

But at some point I hope to switch the custom user-patches out, but I fully understand this will take effort to develop. And open source motivation comes from your own case, not from others. So here is hoping I'll find some time to properly take a swing at this, but I don't expect to find that time anywhere soon.

FYI: This is my setup. I WOULD NOT ADVICE ANYONE DOING THIS! You have to review changes to the patched files for every release:

10-crypt.conf:

mail_attribute_dict = file:%h/.attributes
mail_plugins = $mail_plugins mail_crypt

plugin {
  mail_crypt_curve = secp521r1
  mail_crypt_save_version = 2
  mail_crypt_require_encrypted_user_key = yes
}

user-patches.sh:

#!/bin/bash

TARGET='/etc/dovecot/conf.d/auth-passwdfile.inc'
ADDEDLINE='override_fields = userdb_mail_crypt_private_password=%{sha256:password} userdb_mail_crypt_save_version=2'
if ! grep -Fq "$ADDEDLINE" $TARGET; then
    sedfile -i 's|^\(  args = scheme.*userdb\)|\1\n'"$ADDEDLINE"'|' $TARGET
fi

TARGET='/usr/local/bin/addmailuser'
if ! grep -Fq "mail_crypt_private_password" $TARGET; then
cat << 'EOF' >> "$TARGET"
  while [[ $(stat -c '%U' "/var/mail/${DOMAIN}/${USER}") != docker ]] # handle slow ownership setting problem
  do
    echo "Waiting for proper owner to be set on /var/mail/${DOMAIN}/${USER}..."
    sleep 1
  done
  doveadm -o plugin/mail_crypt_private_password="$(echo -n "${PASSWD}" | sha256sum | awk '{print $1}')" mailbox cryptokey generate -u "${FULL_EMAIL}" -U
EOF
fi

TARGET='/usr/local/bin/updatemailuser'
if ! grep -Fq "mail_crypt_private_password" $TARGET; then
cat << 'EOF' >> "$TARGET"
  FULL_EMAIL="${USER}@${DOMAIN}"
  USER="${FULL_EMAIL%@*}"
  DOMAIN="${FULL_EMAIL#*@}"
  if [[ ! -f "/var/mail/${DOMAIN}/${USER}/.attributes" ]] # create key if it doesn't exist already
  then
    doveadm -o plugin/mail_crypt_private_password="$(echo -n "${PASSWD}" | sha256sum | awk '{print $1}')" mailbox cryptokey generate -u "${FULL_EMAIL}" -U
  else
    if [[ -z "${CRYPTOKEY_UPDATE_PASSWORD}" ]]
    then
      read -r -s -p "Enter Old Password (to update encryption key): " CRYPTOKEY_UPDATE_PASSWORD
      echo
      [[ -z ${CRYPTOKEY_UPDATE_PASSWORD} ]] && _exit_with_error "old encryption password must not be empty"
    fi
    doveadm mailbox cryptokey password -u "${FULL_EMAIL}" -n "$(echo -n "${PASSWD}" | sha256sum | awk '{print $1}')" -o "$(echo -n "${CRYPTOKEY_UPDATE_PASSWORD}" | sha256sum | awk '{print $1}')"
  fi
EOF
fi

custom mounts:

   volumes:
      - ./config/dms:/tmp/docker-mailserver # get user patches
      - ./config/dms/10-crypt.conf:/etc/dovecot/conf.d/10-crypt.conf:ro # add config

@polarathene
Copy link
Copy Markdown
Member

polarathene commented Mar 3, 2023

So here is hoping I'll find some time to properly take a swing at this, but I don't expect to find that time anywhere soon.

I've had something 90% complete since August 2022, but:

  • I had to put it on hold while I attended a course between Aug to Dec 2022.
  • I've been swamped with DMS contributions / reviewing since late Dec with v11.3 release.
    • Lots of churn and changes for v12 breaking changes and big new features or fixes requiring heavy investigation.
    • Push for a change I've been hesitant to approve until tests get into better shape / coverage.

I actually was looking into revisiting it roughly a week ago to adapt to all the v12 changes, but received more issues and reviews to go over (which I'm slow at), among other commitments outside of DMS also reaching out for their own time-sensitive needs. I'm exhausted.


I'd close this as is, but I feel like at least two persons will reach out and tell me this will be worked on.

I was wanting to tackle this in Jan, but needed to prioritize other activity within the project that came up. With v12 release, hopefully that settles down so I can allocate time to getting this wrapped up (the tests definitely aren't going to rebase anymore).

I can't make any promises if I have to shift priorities elsewhere, I have a backlog building up outside of DMS and really should be focusing on job search 😅 I'll do my best to at least get the work adjusted and pushed up as a draft PR in the event I am unable to finish it.

@DavyLandman
Copy link
Copy Markdown

I'm exhausted.

Damn, sorry about that. I did not mean to pile on. I think this project is doing great, and looking at it's history, it's kinda impressive you all manage to keep it running like this, even with multiple people coming and leaving.

So yeah, focus on that job search, and this issue will get back later, once you have the space.

@georglauterbach
Copy link
Copy Markdown
Member

Closing this for now. Reason: This PR has become stale. If you want to pick up these changes, please do so :) We encourage everyone to further work on this, and apply these changes and updates to the most recent version of master.

Related: #3289 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/documentation area/features area/security stale-bot/ignore Indicates that this issue / PR shall not be closed by our stale-checking CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Encryption with user keys

8 participants