Skip to content

refactor: Internal HOSTNAME and DOMAINNAME configuration#2280

Merged
polarathene merged 11 commits intodocker-mailserver:masterfrom
polarathene:fix/setting-dms-fqdn
Nov 15, 2021
Merged

refactor: Internal HOSTNAME and DOMAINNAME configuration#2280
polarathene merged 11 commits intodocker-mailserver:masterfrom
polarathene:fix/setting-dms-fqdn

Conversation

@polarathene
Copy link
Copy Markdown
Member

@polarathene polarathene commented Nov 1, 2021

Description

This has been extracted from #2245 to simplify review.

These changes are the main focus of the original PR where the bulk of the discussion and investigation was oriented around.

Originally reworked by @NorseGaud with heavy refactoring by me afterwards.


Commits have been scoped with context via commit messages for ease of reviewing.

Summary of changes:

  • Better logical flow, handling and inline documentation.
  • Panic when $HOSTNAME is misconfigured.
  • Heavy inline docs for maintainers to understand and be aware of related functionality.
  • SRS_DOMAINNAME test should use --domainname not ENV DOMAINNAME (reverted this fallback that was added from the PR that originally introduced _obtain_hostname_and_domainname)

There was discussion to consider dropping config ENV OVERRIDE_HOSTNAME and the internal DOMAINNAME in future, and rename our HOSTNAME var to DMS_FQDN. Before that happens tests need to be overhauled to more easily perform matrix testing (different base configs where we can verify tests pass for various scenarios) to establish confidence in what may likely be a breaking change when introduced. As such that's delayed to a future PR.

Type of change

  • Improvement (non-breaking change that does improve existing functionality)

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • New and existing unit tests pass locally with my changes

@polarathene polarathene marked this pull request as draft November 1, 2021 04:55
@polarathene

This comment has been minimized.

@georglauterbach

This comment has been minimized.

@polarathene

This comment has been minimized.

Better logical flow, handling and inline documentation.

Rather than using the bash regex stripping for `DOMAINNAME` or conditional checks opted for a condition that checks DNS labels and splits by `cut`.
Despite the verbosity, it's better to make this visible here for maintenance and debugging purposes than trying to dig through issue/PR or commit history for it.
@polarathene polarathene marked this pull request as ready for review November 14, 2021 00:36
@polarathene polarathene changed the title fix: Internal HOSTNAME and DOMAINNAME configuration refactor: Internal HOSTNAME and DOMAINNAME configuration Nov 14, 2021
@polarathene
Copy link
Copy Markdown
Member Author

polarathene commented Nov 14, 2021

Full summary

Mostly useful as a reference for myself, but making it public here to associate it with the relevant PR, and the hopefully unlikely event that some poor maintainer in the future needs to wade through history about these changes/decisions 😅


This PR is an iteration improvement of a recent refactor of the same hostname/domainname handling logic (#2175).

While there was plenty of discussion and iteration in the original version of this PR (#2245), anything relevant to this change is now covered in this PR thread, all other changes in the original have been extracted and reference it as well (thus available at the end of the thread prior to closing), but were catering to unrelated concerns.

Anyone reading this comment should not need to wade through the original thread 🎉 Direct links to comments on it that are potentially worthwhile are linked below, in addition to the commit message history and inline comments of this PR for changes it applies.


I don't consider the following as blocking for merge of this PR (no regressions are introduced AFAIK), but they are relevant context wise, referencing my comments from the original PR discussion thread/review.

Including here for traceability for myself and future maintainers:

  • Not handled yet:
    • Bare domains (two DNS labels only, for the FQDN) will still result in misconfiguration with Postfix. This should be handled similar to how LDAP does to avoid issues (More details).
    • --network=host is a case where /etc/hosts may not be configured correctly. Present advice is to use OVERRIDE_HOSTNAME. I don't consider that sufficient as it does not update /etc/hosts, only a workaround for HOSTNAME usage. Needs investigation + documentation (More details).
  • Test mail_hostname.bats: Still needs to be extracted from original PR (now closed), it was mostly a refactor (but included some /etc/hosts checks).
    • I have a review comment to go over again relevant to that. I will handle that with a follow-up PR that additionally converts it to using the new common container template approach.
    • Note: This test file was originally two separate tests for OVERRIDE_HOSTNAME and SRS_DOMAINNAME that got merged into a single test file here.
  • v11: Should probably deprecate support for domainname, possibly with some related ENV, but more thorough testing is required (More details).
    • SSL_DOMAIN and OVERRIDE_HOSTNAME history detailed here and here (also details SRS_DOMAINNAME, and a 2017 switch for HOSTNAME from using the command hostname, to hostname -f as a workaround fix when using --link..), along with this comment about cert provisioning for a bare domain but configuring a hostname with a subdomain scenario.
    • Reference: HOSTNAME and DOMAINNAME handling in _obtain_hostname_and_domainname, with comparison to before/after changes in original PR, and other insights worth looking over again are detailed here.

Future work:

  • DOMAINNAME => DMS_DOMAINNAME (or similar), potentially with deprecation notice for the domainname user config.
  • OVERRIDE_HOSTNAME and SSL_DOMAIN => Dropping support with deprecation notice once test matrix is implemented and confirms safe for removal (with original scenarios requiring these better documented). SRS_DOMAINNAME may warrant consideration as well.
  • HOSTNAME => DMS_FQDN, after the prior two changes are able to ensure it's safe to make this change. Potentially switching back to hostname instead of hostname -f or considering other alternatives.

When looking into these changes, this comment itself provides ample reference for insights in doing so. In addition, hostname and domainname config options along with command output was tested with a variety of configurations on both Debian and Alpine base images (just the base images, without DMS additions), along with interaction of /etc/hosts content is detailed here (USEFUL REFERENCE TABLE).

That resource can be revised with the next iteration and added to maintainer docs, which in combination with an improved test suite will help avoid this area becoming problematic in future contributions.


Additional note about current behaviour, if you don't use a bare domain (two label FQDN), then we implicitly do remove the 1st (left-most) label from the derived FQDN and use the rest as DOMAINNAME such as for the postmaster address. I don't believe we document that and it can be inconsistent with the bare domain FQDNs.

Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Comment thread target/scripts/helper-functions.sh Outdated
Copy link
Copy Markdown
Member

@georglauterbach georglauterbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just this one comment :)

Comment on lines +308 to +311
# TODO: `OVERRIDE_HOSTNAME` was introduced for non-Docker runtimes that could not configure an explicit hostname.
# k8s was the particular runtime in 2017. This does not update `/etc/hosts` or other locations, thus risking
# inconsistency with expected behaviour. Investigate if it's safe to remove support. (--net=host also uses this as a workaround)
export HOSTNAME="${OVERRIDE_HOSTNAME:-$(hostname -f)}"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not like to see OVERRIDE_HOSTNAME being removed as is without introducing another variable, maybe for K8s, that does an equal job. I'd be very hesitant to remove this. Maybe adjust the comment, or remove it at all?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe adjust the comment, or remove it at all?

The comment stays. It makes it rather clear what the concern is and what needs to be done for it to be considered for removal:

  • Introduced for non-Docker runtimes that could not configure an explicit hostname (k8s was the particular runtime in 2017). --net=host also uses this as a workaround.
  • This does not update /etc/hosts or other locations, thus risking inconsistency with expected behaviour.
  • Investigate if it's safe to remove support.

I'm likely one of the only ones who will investigate it, and that will be a while off as it requires big adjustments to the test suite that I'm slowly chipping away at.

If you can confirm that in almost 2022, this remains a requirement for k8s that is helpful :)


I would not like to see OVERRIDE_HOSTNAME being removed as is without introducing another variable

I know of one case with --network=host where it is currently required, but I don't consider the workaround satisfactory due to the lack of /etc/hosts parity.

What is likely to happen is the tests should show if the /etc/hosts addition matters at all, other than hostname -f or similar commands or if we can shift away from hostname/domainname config and have all scenarios unify on a common ENV instead.

If that makes sense to do, it can't be done confidently AFAIK without the test matrix run I want to be able to perform 😅

The only way OVERRIDE_HOSTNAME will get removed or replaced is via PR anyway, I see no harm in the comment, only additional clarification to future maintainers :)

Copy link
Copy Markdown
Member

@georglauterbach georglauterbach Nov 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, we keep the comment, this is fine with me :) Just wanted to make sure we're all on the same page about this variable. Although I just found out that it is not strictly required, I still wanted to mention it :)

@polarathene polarathene mentioned this pull request Nov 14, 2021
4 tasks
@jpduyx

This comment was marked as off-topic.

@polarathene
Copy link
Copy Markdown
Member Author

Not relevant to this PR, open a discussion for personal support requests.

Short answer is you can't do anything about that. The IP provider controls rDNS entries, you need a static IP that they agree to change rDNS entry for.

Changing your hostname to the rDNS record makes no sense when you don't own that domain.

You don't need the rDNS, if you're concerned about delivery reputation / success relay through a service like SendGrid.

@jpduyx

This comment was marked as off-topic.

@georglauterbach
Copy link
Copy Markdown
Member

You're not getting the point. It's not a Postfix issue; it's a DNS issue. Please use the discussions for this kind of question.

@polarathene
Copy link
Copy Markdown
Member Author

polarathene commented Feb 12, 2024

Thank for the reply. I'm sorry to put this question here and was not looking for personal support. But I would like to know if it would be technally wrong if postfix would could somehow be configured to announce itself with the reverse dns name to avoid spam points .... wouldnt' this be possible to automate  during the docker container setup, without breaking anything else?

This is still the kind of support questions that belong in Discussions, not our pull requests.

The check you're referring to varies by MTA AFAIK, with Postfix it is likely reject_unknown_reverse_client_hostname, I have written this comment to explain it better.

It is not as strict of a requirement as you are seeking with rDNS exact match. I don't think that needs to be that strict, so you should be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants