Skip to content

fix(listmailuser): Don't query quota, if ENABLE_QUOTAS is not 1#2264

Merged
casperklein merged 19 commits intodocker-mailserver:masterfrom
casperklein:listmailuser-quota-fix
Nov 1, 2021
Merged

fix(listmailuser): Don't query quota, if ENABLE_QUOTAS is not 1#2264
casperklein merged 19 commits intodocker-mailserver:masterfrom
casperklein:listmailuser-quota-fix

Conversation

@casperklein
Copy link
Copy Markdown
Member

@casperklein casperklein commented Oct 29, 2021

Description

Fixes #2263 (comment) and introduces a new file /etc/dms-settings, which contains all DMS specific settings.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (non-breaking change that does improve existing functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@casperklein casperklein self-assigned this Oct 29, 2021
@casperklein casperklein requested a review from a team October 29, 2021 13:45
@casperklein casperklein added this to the v10.3.0 milestone Oct 29, 2021
@casperklein casperklein added area/scripts kind/improvement Improve an existing feature, configuration file or the documentation labels Oct 29, 2021
@casperklein casperklein enabled auto-merge (squash) October 29, 2021 13:46
@casperklein casperklein disabled auto-merge October 29, 2021 15:32
@casperklein casperklein changed the title fix(listmailuser): Don't query quota, if ENABLE_QUOTAS=0 Don't query quota, if ENABLE_QUOTAS=0 Oct 29, 2021
@casperklein casperklein marked this pull request as draft October 29, 2021 15:49
@casperklein casperklein marked this pull request as ready for review October 29, 2021 22:32
Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any logic that's trying to apply defaults that should have been applied already is a bit of a red flag, especially for maintenance.

For now, maintain this as a separate file (eg: /target/scripts/init-default-env.sh) and source that. helper-functions.sh is commonly sourced by utilities (and pretty much everything else), so is probably an appropriate place to import this workaround.

We should be able to avoid the issue entirely via an entrypoint script in future.

Comment thread target/bin/listmailuser Outdated
@casperklein casperklein marked this pull request as draft October 30, 2021 10:54
@casperklein casperklein marked this pull request as ready for review October 30, 2021 12:14
@casperklein casperklein requested review from a team and polarathene October 30, 2021 12:14
Comment thread target/scripts/startup/setup-stack.sh Outdated
Copy link
Copy Markdown
Member

@georglauterbach georglauterbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just the one change I pointed to with the comment above. Not sure whether we want to clear the /etc/dms-settings files beforehand - that's why I did not directly commit the changes :D

@casperklein
Copy link
Copy Markdown
Member Author

casperklein commented Oct 30, 2021

Not sure whether we want to clear the /etc/dms-settings files beforehand - that's why I did not directly commit the changes :D

I think clearing the files is a good practice to avoid duplicate entrys, when the container is restarted.

Copy link
Copy Markdown
Member

@georglauterbach georglauterbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏼

@polarathene
Copy link
Copy Markdown
Member

polarathene commented Oct 30, 2021

Entrypoint = PID1. If entrypoint is a bash script, it will not take care of signal forwarding, reaping processes etc. An init-system must be PID1.

-ENTRYPOINT ["/usr/bin/dumb-init", "--"]
+ENTRYPOINT ["/usr/bin/dumb-init", "/entrypoint.sh"]

CMD ["supervisord", "-c", "/etc/supervisor/supervisord.conf"]

Supposedly that would work fine, but allow conditional logic based on the CMD used?

But I think we can skip this discussion. If a script like listmailuser needs ENVs, it can just source /etc/dms-settings now 😎

I only bring it up because of docker run ... setup email list would not know about /etc/dms-settings or /root/.bashrc as neither exist.

I guess if docker run is never used with a CMD override it's a non-issue and any other commands should be run via docker exec on an existing instance instead.

I have had interest with adding an entrypoint script in the past, but start-mailserver.sh being started via supervisord via the mailserver service is for the most part equivalent, and anything I would add via an entrypoint should be ok there instead? (eg: there's been interest to support initializing with a dummy account instead of requiring separate external script/files to provide an account up front)


I can definitely understand how always setting up ENV could be an issue (eg: the open-dkim utility script has generic var existence checks like DOMAINS which aren't scoped to local vars/unset upfront), but there's also stuff like this in our tests (added in 2016 by original project author):

@test "checking amavis: VIRUSMAILS_DELETE_DELAY override works as expected" {
# shellcheck disable=SC2016
run docker run --rm -e VIRUSMAILS_DELETE_DELAY=2 "${IMAGE_NAME:?}" /bin/bash -c 'echo "${VIRUSMAILS_DELETE_DELAY}"'
assert_output 2
}

That's just running the container and echoing the ENV that was set on it. The echo CMD overrides the default supervisord command since this is docker run and not docker exec, thus none of the startup scripts are run. I guess that's fine, as VIRUSMAILS_DELETE_DELAY is only:

  • Dockerfile: ENV default to 7 (added in 2017 for cron, did not work but was retained)
  • start-mailserver.sh: default to 7 (makes the Dockerfile ENV a bit redundant? When the Dockerfile ENV was added in 2017 this var was removed, but then added back in 2018 to resolve an issue)
  • setup-stack.sh: writes it to /etc/environment (cron, same 2017 PR)

Likewise two tests that use delmailuser and addmailuser also call themselves via docker run, if they ever need to rely on something like /etc/dms-settings that won't exist.

Just a caveat to be aware of, doesn't seem like it would cause any problems for the project atm though 👍

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Just need to update the test to work with the new /etc/dms-settings approach 😀

Comment thread test/tests.bats
for VAR in "${!VARS[@]}"
do
echo "export ${VAR}='${VARS[${VAR}]}'" >>/root/.bashrc
echo "${VAR}='${VARS[${VAR}]}'" >>/etc/dms-settings
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see in the discussion what the point of dropping the export prefix was? Couldn't you just dump these to /etc/dms-settings as before instead of /root/.bashrc? Then have /root/.bashrc source /etc/dms-settings?

Or was there some reason to want to temporarily set/override ENV without making it available to subprocesses?

I'm fine with whatever you go with, you have a better understanding of this stuff than me 😅 It seems better than maintaining a bunch of conditional default assignments like the previous approach 👍 (except for loss of compatibility with docker run or the docker exec ... sh -c ... overrride)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is pretty simple: The file should only contain the current container settings and not any further commands/logic. So if a script needs them (afaik this is the first time), the file can just be sourced. There is no need for "export". If a script needs our variables exported to a subprocess, it can do it on its own.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just seemed more consistent in behaviour/expectation if we kept the export as if .bashrc isn't loaded (eg: because of bash -c), then we're still able to make the expected environment available to our scripts. And I say that in the sense that sometimes calling a command may have the environment already made available to it, while the other context is sourcing it without the export.

I think we're more likely to run into troubleshooting issues from that inconsistency more than we would be from issues of having the env exported when it's one of the expected contexts and the whole point of this PR is a workaround for when that hasn't happened to ensure defaults are correctly fallen back to.


Not my forte here, so if your more confident in avoiding export as the better approach go for it 😎

Copy link
Copy Markdown
Member Author

@casperklein casperklein Oct 31, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as if .bashrc isn't loaded

For clarification: the DMS variable stuff in .bashrc is not used in any way. It's just there (I think), to quickly check what ENVs the running container uses. So when you start a bash session, you can easily do something like echo $SOME_DMS_VAR.
That said, I don't think the exports in .bashrc aren't needed for anything DMS rely on.
Edit: The only exception is VIRUSMAILS_DELETE_DELAY.

The only script that need to access a DMS var (ENABLE_QUOTA) is listmailuser. I don't think introducing a entrypoint script is worth it. If there is more demand/scripts in the future, we can still implement it.

Copy link
Copy Markdown
Member Author

@casperklein casperklein Oct 31, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole .bashrc thing started with this for the VIRUSMAILS_DELETE_DELAY.

Oups. You already figured that out above.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start-mailserver.sh: default to 7 (makes the Dockerfile ENV a bit redundant? When the Dockerfile ENV was added in 2017 this var was removed, but then added back in 2018 to resolve an issue)

I think there is a room for improvements 😉 I am wondering, if any of the ENVs in Dockerfile nowadays are needed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: The only exception is VIRUSMAILS_DELETE_DELAY.

Or maybe not 😆 You also wrote, that VIRUSMAILS_DELETE_DELAY is also in /etc/environment.

@casperklein
Copy link
Copy Markdown
Member Author

casperklein commented Oct 31, 2021

@polarathene

I only bring it up because of docker run ... setup email list would not know about /etc/dms-settings or /root/.bashrc as neither exist.

Correct. Haven't consider this till now. The script will throw an error "/usr/local/bin/listmailuser: line 5: /etc/dms-settings: No such file or directory" but then continues flawless, which I didn't understand at first. See below why.

@georglauterbach

Some time back, you wanted us to use [[ expresssion ]], instead of [ expression ], which was used until then. I thought, there were only benefits and no drawbacks. However, today I noticed something that did not work as expected for me:

     1  #!/bin/bash
     2
     3  if [ "$SOME_UNSET_VARIABLE" -eq 0 ]; then
     4          echo "foo"
     5  fi
     6
     7  if [[ "$SOME_UNSET_VARIABLE" -eq 0 ]]; then
     8          echo "bar"
     9  fi
./test.sh: line 3: [: : integer expression expected
bar

I stumpled upon this, because I wondered, why listmailuser didn't fail, when $ENABLE_QUOTA was unset (called via setup.sh and no container running).

@polarathene
Copy link
Copy Markdown
Member

However, today I noticed something that did not work as expected for me

That works how I'd expect tbh.

The value is unset/null/false, so I expect the it to evaluate to equivalent of 0 (falsey). Some programming languages at least behave that way. Although ${SOME_UNSET_VARIABLE} == 0 will evaluate to false.. 🤷‍♂️

@georglauterbach
Copy link
Copy Markdown
Member

I guess this is some kind of Bashism, but it also kind of makes sense to me. What you brought up a sh specific error, and bash evaluates variables whose values are null to falsy and I guess therefore equal to zero. I wouldn't see this as a drawback with Bash to be honest, because the type system (haha, as if Bash had a real type system) is somewhat weird with shells in general.

If this was a programming language like C or Rust or something else with a strict typing discipline, I'd consider this a drawback too, but with Bash, I've learned to just accept it :D

@casperklein casperklein marked this pull request as draft October 31, 2021 16:57
polarathene
polarathene previously approved these changes Oct 31, 2021
Comment thread target/bin/listmailuser
Comment thread target/scripts/startup/setup-stack.sh Outdated
@casperklein casperklein marked this pull request as ready for review October 31, 2021 23:47
@casperklein casperklein changed the title Don't query quota, if ENABLE_QUOTAS=0 fix(listmailuser): Don't query quota, if ENABLE_QUOTAS is not 1 Nov 1, 2021
@casperklein casperklein merged commit c7dec1e into docker-mailserver:master Nov 1, 2021
@casperklein casperklein deleted the listmailuser-quota-fix branch November 1, 2021 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/scripts kind/improvement Improve an existing feature, configuration file or the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mailserver behind Traefik

3 participants