Subject

Something else that requires developers attention

Description

I have been working on getting familiar with the LDAP support in DMS, going over the history and issues / feature requests after revising the LDAP tests we have.

There's quite a few changes to work on, some for reduced maintenance burden, others breaking changes or feature work. I've made extensive notes for my own reference to support that work but as I revise them, I'll transfer them here as a potentially helpful reference for anyone else working on the project, especially if I run out of time for DMS.

TODO

WIP

• LDAP_BIND_PW ENV should support detecting a filepath to read a secret in, rather than requiring the secret be exposed to ENV: feature request: Adding LDAP_BIND_PW_FILE to support docker secrets #3457 (comment)
• Address the mail forwarding use-case.
  • In the postfix-book.schema using mailAlias to add an alias to an LDAP account that resolves to their mail attribute would seem similar to setup alias add, but when forwarding to an external address this would be incorrect as mail attribute should not really contain that address as an alias (or have some way to make it distinct from a DMS account aka setup email add, perhaps via filter constraint on mailEnabled=TRUE).
  • The schema source had been extended to include support for a forwarding address attribute (1 or more supported) that would probably be more appropriate. Without handling this properly, user lookup queries on mail or domain lookup queries that filter based on mail can be problematic.
  • This was discovered here, although I misunderstood the intended setup (no alias configured, just misconfiguration on domain lookup).

DMS LDAP feature History

• May 2016 - Original request for LDAP / external auth: LDAP #187
• Aug 2016 - Earliest LDAP PR attempt: added LDAP to dovecot and postfix (TODO: regression test & improve DOC) #251
• Oct 2016 - LDAP support implemented for both Postfix and Dovecot + initial LDAP tests contributed:
  • This PR was built off an earlier PR attempt in Sep 2016, which lacked Dovecot integration. The revised PR not only added support for Dovecot, but also adjusted the original PR content (LDAP config files / ENV) to use the uniqueIdentifier attribute for lookup queries (as it was introduced via the LDIF files).
  • Dockerfile added new packages: dovecot-ldap, postfix-ldap, sasl2-bin (Provides cyrus saslauthd, aka ENABLE_SASLAUTHD ENV).
  • postfix-ldap-*.cf files introduced, but only as example configs, with support to copy your own to an internal location (via the DMS config volume /tmp/docker-mailserver).
    • Nov 2016 - The postfix- filename prefix was dropped and the example configs moved into the image (at /etc/postfix/ via the Dockerfile).
      • The PR also added the domainname (DMS FQDN minus any subdomain) into the Postfix vhost file, which now is presently handled via the postfix.sh helper_).
      • However the PR also removed support for user supplied ldap-*.cf files (via /tmp/docker-mailserver), allowing only for ENV overrides or mounting each config file separately into /etc/postfix.
    • April 2017 - Support for user-supplied ldap-*.cf files in /tmp/docker-mailserver was restored.
  • Basic LDAP_* config ENV support introduced with simple multi-line sed expression to modify file.
• Sep 2023 - Some big refactoring with breaking changes introduced for the user-provided configs + ENV override support. See below section "Override config via ENV feature" for more details.

Additional Postfix LDAP configs

• July 2017 - Introduced ldap-domains.cf. Source additional vhost domains for Postfix virtual_mailbox_domains to be aware of through an LDAP config configured with query_filter + result_attribute.
• April 2021 - Introduced ldap-senders.cf for improved LDAP support with SPOOF_PROTECTION=1.
• Sep 2023 - ldap-senders.cf lacked support for being provided by the config volume like the other Postfix LDAP config files.
  • It is now also supported.
  • The files result_attribute default is no longer mail, uid. It now uses the same mail value as all other related configs.

Override config via ENV feature

• April 2017 - LDAP ENV override support was implemented to enable adjusting key/value pair settings in ldap-*.cf files.
• July 2017 - After some bug fixes since April, the LDAP ENV support was expanded to enable individually modifying the query_filter key of each ldap-*.cf file via an additional suffix to the ENV (mapped to the different supported filenames). That PR also introduced the DOVECOT_* key/pair override support for supporting the dovecot-ldap.conf.ext config file.
• July 2017 (issue & PR) - The ENV key/pair config file override functionality evolved into an external tool (used by DMS via git submodule) known as configomat.
• Feb 2023 - configomat was rewritten as a helper script for DMS to use and maintain directly without relying on an external submodule.
• Sep 2023 - I implemented a new approach with templates using variable replacement that brings improvements and broader support than we had prior.
  • Now both Dovecot and SASLAuthd config files can be user-provided via the config volume and the latter uses the same ENV override approach/method instead of a HereDoc.
  • Custom config files can be provided by the user via the config volume, but no longer needs to redundantly define all keys for replacement. ENV override/defaults can be used separately, and the configs are all merged together.
  • Postfix ENV overrides were given their own POSTFIX_ prefix instead of the common LDAP_ one previously used, while the config file specific suffix ENV variants for targeting query_filter has become generic to support any key, but the suffix was revised to align with the filename query kind, and become part of the ENV prefix (eg: POSTFIX_USERS_).
  • query_filter is no longer provided as a default per query kind file. That relied on postfix-book schema and due to a bug fix it was deemed best to make this config more explicitly required.
  • The PR additionally fixes a bug by not enabling all of these Postfix LDAP configs by default, they need a query_filter set explicitly to enable, as some LDAP users did not need some of the features configured which was causing Postfix to perform queries that were not supported by their LDAP service. The user bug report and DMS v12.1 LDAP docs used a workaround with (|) set as a dummy query filter.
  • Internally we no longer have the Postfix or Dovecot configs pre-configured in /etc/postfix / /etc/dovecot. They use the templates in /etc/dms/ldap and generate the configs for services at container startup. Additionally for Postfix these have dropped the ldap-* prefix in their filename and been relocated into /etc/postfix/ldap. No change to user-provided filename/location yet, a future PR will introduce that.
  • The Feb 2023 _replace_by_env_in_file() helper still exists for now, I revised it in a separate PR to be a little more compatible.