Adding this here in case anyone else has time to tackle any of it.
I am not likely to have the time until at least December, but I expect I'll have to prioritize my time on other commitments before I can come back to addressing these tasks.
Task Overview
I've provided plenty of helpful details below, but a summary of tasks to complete is:
SPOOF_PROTECTION=1 feature:
Related alias support:
SPOOF_PROTECTION=1 handling for smtpd_sender_login_maps
Jan 2021, @georglauterbach created and merged a PR to fix a user reported bug. The PR introduced a unionmap: { ... } table, which Postfix queries each listed table and merges the responses, rather than accepting the first matched response and exiting early with that outcome (as the result may fail, but have passed from another valid match rule later on).
Nothing wrong there, but it was the PR that split the non-LDAP handling for smtpd_sender_login_maps (used for ENV SPOOF_PROTECTION=1) into two values, only allowing the unionmap when /etc/postfix/regexp config was present. This does not seem necessary, and using the unionmap regardless is probably fine?
|
function _handle_postfix_regexp_config |
|
{ |
|
: >/etc/postfix/regexp |
|
|
|
if [[ -f /tmp/docker-mailserver/postfix-regexp.cf ]] |
|
then |
|
_log 'trace' "Adding regexp alias file postfix-regexp.cf" |
|
|
|
cp -f /tmp/docker-mailserver/postfix-regexp.cf /etc/postfix/regexp |
|
_register_setup_function '_setup_postfix_aliases' |
|
_register_setup_function '_setup_postfix_vhost' |
|
_register_setup_function '_setup_postfix_dhparam' |
|
_register_setup_function '_setup_postfix_postscreen' |
|
_register_setup_function '_setup_postfix_sizelimits' |
|
|
|
# needs to come after _setup_postfix_aliases |
|
[[ ${SPOOF_PROTECTION} -eq 1 ]] && _register_setup_function '_setup_spoof_protection' |
/etc/postfix/regexp is always created during startup, and it has always been before this logic is run, even before that PR which added the conditional check on it's existence. We would always get unionmap AFAIK. It may even be useful for the LDAP configs.
As a refresher for anyone reading SPOOF_PROTECTION=1 (opt-in) prevents submitting any mail to send unless the logged in account is authorized to send from that envelope address (FROM) in smtpd_sender_login_maps lookup tables. The ENV does this by adding this restriction: smtpd_sender_restrictions = reject_authenticated_sender_login_mismatch (prefixed in front of other sender restrictions).
Without this, our default config allows a logged in user to send mail "from" whatever address they'd like to, including other accounts for the same domain, or one that DMS doesn't manage. The related bug report that introduced the fix with unionmap, has a user explain a good use-case for wanting that, non-human service accounts being restricted to their account only for sending mail in-case the service is compromised, while using a regex to allow their [email protected] account (admin) to be able to send as any address for that domain.
When opting into SPOOF_PROTECTION=1, we permit local aliases (root, amavis, etc) (that linked PR introduced the POSTMASTER_ADDRESS ENV and alias, along with setting mydestination in main.cf), account aliases, and a few other lookup tables in the unionmap. One of these is a regexp file sender_login_maps.pcre that will match whatever the owners sender address is, and allow it (There is similar advice elsewhere, that suggests this approach, seems fine). A more specific match would have higher priority I think, but it may just depend on whatever is matched first from top to bottom of a table.
It should be noted that without the /etc/postfix/virtual table for account aliases included in the unionmap, you cannot send from an alias (eg: [email protected]), but you can authenticate over SASL as your aliased account by using that alias as the login account/username. That still logs the user in as the aliased account, not the alias, thus the actual account is still what is checked for allowed sender addresses that can be used regardless.
The original PR that introduced this feature with SPOOF_PROTECTION=1 also has discussion about enabling it by default in future breaking change. And some users sharing concerns about their workflows being broken, one from a previous maintainer who was using aliases to reply to mail keeping the same FROM address in the message, while only having a single mail account to manage.
Additional Notes
The /etc/postfix/regexp table was added in this PR, when the user previously tried to use a catch-all alias @example.com and noticed it did not work for them (received but could not send), possibly related to the later fix of using a unionmap table.
If we have any LDAP related issues with the feature, this may be helpful. That had a follow-up issue which also had a related PR merged for improving LDAP support with an optional filter config ENV - it demonstrates how to allow an additional sender address for [email protected].
The tables for smtpd_sender_login_maps are pretty simple, the left-side (1st column) is to match the desired sender address (explicit value or regex pattern depending on table type), and on the right-side (2nd column) is usually the owner address (login mail account, non-aliased), but it can be multiple addresses to allow too by using a , delimiter just like the /etc/postfix/virtual alias table allows to map an alias to multiple recipients.
If using a regex pattern, keep in mind that whatever matches in a table first is the pass/fail for that table, regardless of the unionmap usage. As noted by this users comment that had two lines /.*/ to approve two separate domains, only the first would ever be matched. The response suggests moving the 2nd line to a separate table, but they should be able to fold the 2nd domain into a single line for that same regexp as mentioned above with , delimiter.
The ENV docs for SPOOF_PROTECTION=1 notes that extension delimiters (eg: +) are no longer usable as we don't have a table to authorize them out of the box. Technically it should be possible with regex as well? Would need some testing to ensure it's not abused.
unionmap with smtpd_sender_login_maps is covered well with the problem it solves in this blog article.
The Postfix SASL docs briefly touch on related configuration discussed here, but seems to incorrectly reference using it with smtpd_recipient_restrictions rather than smtpd_sender_restrictions.
SPOOF_PROTECTION=1 only restricts the envelope FROM sender address, spoofing still feasible?
Despite SPOOF_PROTECTION=1 applying these settings to restrict what a sender can use in FROM (mail envelope), this doesn't prevent spoofing of the from: (mail message) header apparently. This is mentioned here, in a link it references, that resource links to another that clarifies the difference between envelope and message FROM headers, and how SPF+DKIM+DMARC are leveraged to better handle this.
Thus SPOOF_PROTECTION may not be the most appropriate ENV name, it's mostly to restrict your own authenticated accounts to what sender addresses can be used in the envelope FROM when they submit mail, which is still worthwhile to do. There is this project which is a Postfix milter designed to ensure the sender envelope and message are the same, which prevents OpenDKIM signing the message sender (From: header) for a mismatch on the same domain, but different account.
Related
Supporting send-only addresses without a regex table
A recent feature-request was raised for allowing accounts to opt-out from being considered managed by Postfix. They provide their solution to use another config for scripts/helpers/postfix.sh:_vhost_collect_postfix_domains() to exclude any account using a mentioned domain. I believe we have a similar opt-out config feature for when a relayhost is enabled (due to blanket adding all accounts to the relay..).
That prompted me as likely a misconfiguration (confirmed, due to lack of our docs covering how to support the use-case). Their use-case was to send mail as noreply@<external-domain>, however SPOOF_PROTECTION=1 was restricting this without a matching account for that address - but as they wanted to send to a user at that same external domain (which was now considered managed by Postfix), it tries to deliver internally instead of outbound to the intended external mail-server. The noreply sender address was only for sending to external accounts, not receiving any inbound mail, an account is therefore not necessary to resolve this use-case properly.
I investigated and figured that they'd be better to use an opt-in config for what account should be allowed to send as an external account instead, which the smtpd_sender_login_maps can allow just fine. My link to that issue provides an example for how to setup a local testing environment that proves this approach works well. Normally it's not an issue when SPOOF_PROTECTION=0 (default), as an authenticated user can use whatever sender address they like then.
Only issue is that using regex probably isn't always appropriate - Users need to be careful with what expressions they use, often forgetting to escape regex tokens like . when providing a domain (I came across quite a few snippets where this was the case). However all the other tables aren't specifically for this feature which could cause issues, so another table should be made available within the unionmap. I haven't looked over the earlier mentioned LDAP PR that added a related config ENV to extend support, but assume it provides similar flexibility when needed.
If SPOOF_PROTECTION=1 was to be enabled in future as a default, we should first support configuring with an alternative table (eg: texthash:/etc/postfix/senders-map.cf or similar would be fine), along with supporting documentation and release notes (probably warrants a notice period before switching the default for the feature?).
External accounts being handled as internal accidentally?
Related is another scenario / issue, again regarding a noreply sending account (but this one would belong to the internal domain). It's recipients are provided from LDAP accounts, and some users are remote / external addresses, such as [email protected].
I'm inclined to think this is more of an issue with their LDAP config, as DMS should not be treating @gmail as managed, unless your config is creating them as internal mail accounts for DMS (equivalent of postfix-accounts.cf), or misconfigured as aliases (since we collect any alias domain into the Postfix /etc/postfix/vhost to be treated as managed by DMS).
Their desire was to try match the recipient mail is to be sent out to in hopes of convincing Postfix to deliver as outbound mail. Again, these accounts were not to receive mail to DMS, or forward / relay any inbound mail arriving to DMS to these external mail-servers (eg: Gmail). The only intention was having the [email protected] send out notifications from a service to these LDAP users associated mail accounts (where some of the LDAP users are example.com accounts, but not all).
In this case rather than matching the FROM envelope, I believe we could workaround by using a transport_maps table that matches the TO (recipient). The prior send-only issue above did have a solution proposed to opt-out of managing entire domains (by excluding them from Postfix /etc/postfix/vhost), I'm just not convinced that's something we should maintain.
I lack LDAP knowledge, but cannot think of how this would be setup in a non-LDAP config where it'd make sense to create @gmail accounts - but I have seen users create aliases to remote addresses / accounts like Gmail? (which can introduce the same problem if the remote domain is incorrectly added as the alias, not the recipient that an alias should map to)
We have the following described in our docs at the end of this example:
Create email accounts and aliases:
With SPOOF_PROTECTION=0
./setup.sh email add [email protected] passwd123
./setup.sh email add [email protected] passwd123
./setup.sh alias add [email protected] [email protected]
./setup.sh alias add [email protected] [email protected]
./setup.sh email list
./setup.sh alias list
Aliases make sure that any email that comes to these accounts is forwarded to your third-party email address ([email protected]), where they are retrieved (eg: via third-party web or mobile app), instead of connecting directly to docker-mailserver with POP3 / IMAP.
Directly after that SPOOF_PROTECTION=0 example is one for SPOOF_PROTECTION=1, where the advice is to create an intermediate alias, to workaround the sender map problem. That works because we include the alias table of aliases mapping to whatever recipients, and treating those as owners. A workaround that shouldn't be needed / used.
The Postfix virtual docs on alias forwarding has a similar example (excluding the SPOOF_PROTECTION issue).
setup alias add
The setup alias add command "usage" / help could better match our addalias utility which is more clear on the inputs?:
|
${LBLUE}COMMAND${RESET} alias ${RED}:=${RESET} |
|
${0} alias ${CYAN}add${RESET} <EMAIL ADDRESS> <RECIPIENT> |
The internal utility makes it more apparent the first input is an alias that is mapped to recipient(s) (internal postfix-accounts.cf, external addresses, or other aliases):
|
${ORANGE}USAGE${RESET} |
|
./setup.sh alias add <MAIL ALIAS> <RECIPIENT> |
Apart from the bug issue, our aliases docs communicate roughly the same two scenarios described above for setup alias add 👍
Alias domains being added to /etc/postfix/vhost
We've had this support introduced way back in Oct 2015, which now lives in a dedicated vhost helper script that I created during a refactoring via this PR (June 2022).
I discussed further details regarding it and related Postfix main.cf config in the send-only linked issue above. As the postfix-regexp.cf for alias support came after that Oct 2015 PR, and our scripts can't realistically scrape domains from a regex, it could be better to have Postfix use the correct tables to lookup directly when it needs to, which would allow for the alias regex.
Postfix provides separate config to check for accounts and aliases regarding if it should be considered the final destination for delivery of mail received/sent. We could better leverage that support directly. It would also allow for users to more easily opt-out domains I think, should they really need to.
Adding this here in case anyone else has time to tackle any of it.
I am not likely to have the time until at least December, but I expect I'll have to prioritize my time on other commitments before I can come back to addressing these tasks.
Task Overview
I've provided plenty of helpful details below, but a summary of tasks to complete is:
SPOOF_PROTECTION=1feature:unionmapand drop the conditional, as the file always exists, there is no known reason to need the alternative without unionmap or additional regex table.unionmap, which is not a regex table. It will allow explicitly allowing additional senders for an account to send as.@gmailrelated issue section below.unionmapwould also be relevant for theldapequivalent value in that same linked method.noreply@external-domainfrom referenced send-only related issue below. Likewise with the LDAP specific config ENV that was added to support the feature specifically there.Related alias support:
setup alias addsection below notes a bug to resolve, and CLI help message to update. It also has poor error handling feedback if somesetupcommands do not receive the expected number of inputs IIRC (may have beensetup alias del)./etc/postfix/vhost", it has been identified that we could improve how we determine what domains are managed.main.cfsettings more appropriately./etc/postfix/vhostis presently sourced elsewhere, such as the DKIM generation support insetupcommand.SPOOF_PROTECTION=1handling forsmtpd_sender_login_mapsJan 2021, @georglauterbach created and merged a PR to fix a user reported bug. The PR introduced a
unionmap: { ... }table, which Postfix queries each listed table and merges the responses, rather than accepting the first matched response and exiting early with that outcome (as the result may fail, but have passed from another valid match rule later on).Nothing wrong there, but it was the PR that split the non-LDAP handling for
smtpd_sender_login_maps(used for ENVSPOOF_PROTECTION=1) into two values, only allowing theunionmapwhen/etc/postfix/regexpconfig was present. This does not seem necessary, and using theunionmapregardless is probably fine?docker-mailserver/target/scripts/helpers/aliases.sh
Lines 29 to 37 in ff96950
docker-mailserver/target/scripts/start-mailserver.sh
Lines 116 to 123 in ff96950
/etc/postfix/regexpis always created during startup, and it has always been before this logic is run, even before that PR which added the conditional check on it's existence. We would always getunionmapAFAIK. It may even be useful for the LDAP configs.As a refresher for anyone reading
SPOOF_PROTECTION=1(opt-in) prevents submitting any mail to send unless the logged in account is authorized to send from that envelope address (FROM) insmtpd_sender_login_mapslookup tables. The ENV does this by adding this restriction:smtpd_sender_restrictions = reject_authenticated_sender_login_mismatch(prefixed in front of other sender restrictions).Without this, our default config allows a logged in user to send mail "from" whatever address they'd like to, including other accounts for the same domain, or one that DMS doesn't manage. The related bug report that introduced the fix with
unionmap, has a user explain a good use-case for wanting that, non-human service accounts being restricted to their account only for sending mail in-case the service is compromised, while using a regex to allow their[email protected]account (admin) to be able to send as any address for that domain.When opting into
SPOOF_PROTECTION=1, we permit local aliases (root,amavis, etc) (that linked PR introduced thePOSTMASTER_ADDRESSENV and alias, along with settingmydestinationinmain.cf), account aliases, and a few other lookup tables in theunionmap. One of these is a regexp filesender_login_maps.pcrethat will match whatever the owners sender address is, and allow it (There is similar advice elsewhere, that suggests this approach, seems fine). A more specific match would have higher priority I think, but it may just depend on whatever is matched first from top to bottom of a table.It should be noted that without the
/etc/postfix/virtualtable for account aliases included in theunionmap, you cannot send from an alias (eg:[email protected]), but you can authenticate over SASL as your aliased account by using that alias as the login account/username. That still logs the user in as the aliased account, not the alias, thus the actual account is still what is checked for allowed sender addresses that can be used regardless.The original PR that introduced this feature with
SPOOF_PROTECTION=1also has discussion about enabling it by default in future breaking change. And some users sharing concerns about their workflows being broken, one from a previous maintainer who was using aliases to reply to mail keeping the sameFROMaddress in the message, while only having a single mail account to manage.Additional Notes
The
/etc/postfix/regexptable was added in this PR, when the user previously tried to use a catch-all alias@example.comand noticed it did not work for them (received but could not send), possibly related to the later fix of using aunionmaptable.If we have any LDAP related issues with the feature, this may be helpful. That had a follow-up issue which also had a related PR merged for improving LDAP support with an optional filter config ENV - it demonstrates how to allow an additional sender address for
[email protected].The tables for
smtpd_sender_login_mapsare pretty simple, the left-side (1st column) is to match the desired sender address (explicit value or regex pattern depending on table type), and on the right-side (2nd column) is usually the owner address (login mail account, non-aliased), but it can be multiple addresses to allow too by using a,delimiter just like the/etc/postfix/virtualalias table allows to map an alias to multiple recipients.If using a regex pattern, keep in mind that whatever matches in a table first is the pass/fail for that table, regardless of the
unionmapusage. As noted by this users comment that had two lines/.*/to approve two separate domains, only the first would ever be matched. The response suggests moving the 2nd line to a separate table, but they should be able to fold the 2nd domain into a single line for that same regexp as mentioned above with,delimiter.The ENV docs for
SPOOF_PROTECTION=1notes that extension delimiters (eg:+) are no longer usable as we don't have a table to authorize them out of the box. Technically it should be possible with regex as well? Would need some testing to ensure it's not abused.unionmapwithsmtpd_sender_login_mapsis covered well with the problem it solves in this blog article.The Postfix SASL docs briefly touch on related configuration discussed here, but seems to incorrectly reference using it with
smtpd_recipient_restrictionsrather thansmtpd_sender_restrictions.SPOOF_PROTECTION=1only restricts the envelopeFROMsender address, spoofing still feasible?Despite
SPOOF_PROTECTION=1applying these settings to restrict what a sender can use inFROM(mail envelope), this doesn't prevent spoofing of thefrom:(mail message) header apparently. This is mentioned here, in a link it references, that resource links to another that clarifies the difference between envelope and messageFROMheaders, and how SPF+DKIM+DMARC are leveraged to better handle this.Thus
SPOOF_PROTECTIONmay not be the most appropriate ENV name, it's mostly to restrict your own authenticated accounts to what sender addresses can be used in the envelopeFROMwhen they submit mail, which is still worthwhile to do. There is this project which is a Postfix milter designed to ensure the sender envelope and message are the same, which prevents OpenDKIM signing the message sender (From:header) for a mismatch on the same domain, but different account.Related
Supporting send-only addresses without a regex table
A recent feature-request was raised for allowing accounts to opt-out from being considered managed by Postfix. They provide their solution to use another config for
scripts/helpers/postfix.sh:_vhost_collect_postfix_domains()to exclude any account using a mentioned domain. I believe we have a similar opt-out config feature for when a relayhost is enabled (due to blanket adding all accounts to the relay..).That prompted me as likely a misconfiguration (confirmed, due to lack of our docs covering how to support the use-case). Their use-case was to send mail as
noreply@<external-domain>, howeverSPOOF_PROTECTION=1was restricting this without a matching account for that address - but as they wanted to send to a user at that same external domain (which was now considered managed by Postfix), it tries to deliver internally instead of outbound to the intended external mail-server. Thenoreplysender address was only for sending to external accounts, not receiving any inbound mail, an account is therefore not necessary to resolve this use-case properly.I investigated and figured that they'd be better to use an opt-in config for what account should be allowed to send as an external account instead, which the
smtpd_sender_login_mapscan allow just fine. My link to that issue provides an example for how to setup a local testing environment that proves this approach works well. Normally it's not an issue whenSPOOF_PROTECTION=0(default), as an authenticated user can use whatever sender address they like then.Only issue is that using regex probably isn't always appropriate - Users need to be careful with what expressions they use, often forgetting to escape regex tokens like
.when providing a domain (I came across quite a few snippets where this was the case). However all the other tables aren't specifically for this feature which could cause issues, so another table should be made available within theunionmap. I haven't looked over the earlier mentioned LDAP PR that added a related config ENV to extend support, but assume it provides similar flexibility when needed.If
SPOOF_PROTECTION=1was to be enabled in future as a default, we should first support configuring with an alternative table (eg:texthash:/etc/postfix/senders-map.cfor similar would be fine), along with supporting documentation and release notes (probably warrants a notice period before switching the default for the feature?).External accounts being handled as internal accidentally?
Related is another scenario / issue, again regarding a
noreplysending account (but this one would belong to the internal domain). It's recipients are provided from LDAP accounts, and some users are remote / external addresses, such as[email protected].I'm inclined to think this is more of an issue with their LDAP config, as DMS should not be treating
@gmailas managed, unless your config is creating them as internal mail accounts for DMS (equivalent ofpostfix-accounts.cf), or misconfigured as aliases (since we collect any alias domain into the Postfix/etc/postfix/vhostto be treated as managed by DMS).Their desire was to try match the recipient mail is to be sent out to in hopes of convincing Postfix to deliver as outbound mail. Again, these accounts were not to receive mail to DMS, or forward / relay any inbound mail arriving to DMS to these external mail-servers (eg: Gmail). The only intention was having the
[email protected]send out notifications from a service to these LDAP users associated mail accounts (where some of the LDAP users areexample.comaccounts, but not all).In this case rather than matching the
FROMenvelope, I believe we could workaround by using atransport_mapstable that matches theTO(recipient). The prior send-only issue above did have a solution proposed to opt-out of managing entire domains (by excluding them from Postfix/etc/postfix/vhost), I'm just not convinced that's something we should maintain.I lack LDAP knowledge, but cannot think of how this would be setup in a non-LDAP config where it'd make sense to create
@gmailaccounts - but I have seen users create aliases to remote addresses / accounts like Gmail? (which can introduce the same problem if the remote domain is incorrectly added as the alias, not the recipient that an alias should map to)We have the following described in our docs at the end of this example:
Directly after that
SPOOF_PROTECTION=0example is one forSPOOF_PROTECTION=1, where the advice is to create an intermediate alias, to workaround the sender map problem. That works because we include the alias table of aliases mapping to whatever recipients, and treating those as owners. A workaround that shouldn't be needed / used.The Postfix virtual docs on alias forwarding has a similar example (excluding the
SPOOF_PROTECTIONissue).setup alias addThe
setup alias addcommand "usage" / help could better match ouraddaliasutility which is more clear on the inputs?:docker-mailserver/target/bin/setup
Lines 46 to 47 in ff96950
The internal utility makes it more apparent the first input is an alias that is mapped to recipient(s) (internal
postfix-accounts.cf, external addresses, or other aliases):docker-mailserver/target/bin/addalias
Lines 21 to 22 in ff96950
setup alias add [email protected] [email protected]will create an alias[email protected]that maps to internal account[email protected].setup email listwill show that alias associated to the account as expected.setup alias add [email protected] [email protected]will create an alias[email protected]that will map to a remote[email protected]account. Mail received / submitted to[email protected]is forwarded to that remote address instead of stored in a local mailbox. Unless Postfix believes it manages that domain.setup email listif[email protected]exists as an account also, will list the[email protected]alias as an alias to the account itself.. but that's a side-effect bug from allowing such in the first place.Apart from the bug issue, our aliases docs communicate roughly the same two scenarios described above for
setup alias add👍Alias domains being added to
/etc/postfix/vhostWe've had this support introduced way back in Oct 2015, which now lives in a dedicated vhost helper script that I created during a refactoring via this PR (June 2022).
I discussed further details regarding it and related Postfix
main.cfconfig in the send-only linked issue above. As thepostfix-regexp.cffor alias support came after that Oct 2015 PR, and our scripts can't realistically scrape domains from a regex, it could be better to have Postfix use the correct tables to lookup directly when it needs to, which would allow for the alias regex.Postfix provides separate config to check for accounts and aliases regarding if it should be considered the final destination for delivery of mail received/sent. We could better leverage that support directly. It would also allow for users to more easily opt-out domains I think, should they really need to.