I am wondering, what the use case for this line is? Any idea?
If it doesn't serve any purpose (nowadays?), we could remove it.

If it doesn't serve any purpose (nowadays?), we could remove it.

git blame shows it belonging to a privacy related feature PR, although they have not created that as a separate feature that is applied during startup (which tends to be better for maintaining even if it's not toggled by an ENV), and instead opted to just have it always applied. We have a single test case only for it (current version), but it does not test anything else beyond the user-agent 🤷‍♂️

I will do a thorough analysis of the whole file. For now, I will merge this and correct the casing though. Expect an answer this week, maybe even today :)

Hi, just wanted to let you know that this is too broad and touches the headers sent for MIME parts as well, and thus breaks one of the most fundamental laws: don't touch the body of the mail. This leads – among other things – to signatures becoming invalid.

How come? Can you give an example?

I can:
The mail that was sent: https://0x0.st/HnHW.eml
The mail that was received: https://lists.sr.ht/~rjarry/aerc-devel/%3CCZ5QDOG8JAPL.1TC6E9E2754L1%40poldrack.dev%3E/raw

0a1,5
> Authentication-Results: mail-a.sr.ht; dkim=pass header.d=poldrack.dev [email protected]
> Received: from mail.moritz.sh (mail.moritz.sh [IPv6:2a03:4000:56:a29:58d0:1eff:fe79:88f])
> 	by mail-a.sr.ht (Postfix) with ESMTPS id C97B5200AD
> 	for <~rjarry/[email protected]>; Thu, 15 Feb 2024 14:43:33 +0000 (UTC)
> X-Virus-Scanned: Yes
4c9,23
< Mime-Version: 1.0
---
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=poldrack.dev; s=mail;
> 	t=1708008212; bh=SzoH5o0PoVfpHvSt/XUfo2VOBBxjrx0aY5lkBM/N924=;
> 	h=Cc:From:To:Subject:References:In-Reply-To;
> 	b=ysUORGN9gicPmWdM8ysA9Lw9gRJJWEgZaPiWOn1OKZu+wlVZb1umSpjj8T9l6mAsP
> 	 T894uQL5dRSVgXikU0F3SasGLgtodt5HzYKIWHfTQ0r3y65aVI1QFxnDSL2Lx1tTgU
> 	 n50vSGwygr2Spm8HXuPFS00IVR3egqDai/j+U9jLfENX6XCMyrJCgW9GsGFhYQAr3f
> 	 HCgHp8U2algOPN9k9fUsKcVkiDegvDjk3MJiN+hWVW0E5WTewrrEp9ukl+y3VTWQ5b
> 	 K6gj/siA7OOdZ1tJM+Ut9TYbZHFfzpcJ/tBgvIUJXQ9is/Z1NPkyiwrjHiP0p7x61q
> 	 g97xhw13UKFW8nHgXXTuLqBnwkiEtq96o4qyoIjppVi9qXz+tkjHp9X7hZNO3k3mz9
> 	 YSnfk88pWGHhKmSz/uyiNxeovhjZxwcjqlFyXgur6YBwMJNc+snClXU4pMuz68st08
> 	 cgDWa3r/taF+FUMComMQwGCETWf9fXPFIDXVmeXjI0Q3gV0tNgWCxgR+CcbL4bIxAc
> 	 DeBZ8m5CMmgyS1LiMSVJCVtqB0ibS3fX7Bt8IiKpp6ku/fdFkUIXxjo7pWcMkH1FWl
> 	 RYA/m3j0D+AHKU96jPWtGRyIWSpwqTkoU8WVZ6D8ymXuW3vFmpcG0/zSuKgR8LeAxs
> 	 mfn8xsGmSXaDyCN+7nA4/78E=
> MIME-Version: 1.0
13d31
< X-Mailer: aerc/0.17.0.r37.g3aa8b630
18c36
< Mime-Version: 1.0
---
> MIME-Version: 1.0

The latter of these being the culprit for the broken signature. I understand that the inclusion of the header with every MIME part is not strictly necessary, but the modifying of body parts is something I can not accept. For now I fixed the issue by overwriting the file in the container via the compose, but a more elegant™ solution would be appreciated. There is already enough weird stuff happening in my compose file :D

I understand you problem, but these defaults were and are working rather nicely for a majority of users. Overwriting the config is actually the "elegant" solution here. May I suggest you use user-patches.sh; it seems appropriate for what you want to do.

The mail that was sent

Was this sent by you through DMS on port 587/465?

It seems this privacy feature is configured in a way that it may try modify at multiple points. Unclear to me if one of those is prior to DKIM signing.

It would be good to know if this is the case, I agree it shouldn't be messing with signed content like that breaking the bh (while b in the signature should be valid AFAIK).

this is too broad and touches the headers sent for MIME parts as well

It seems that when you set header_checks, it will implicitly be the default for a separate setting mime_header_checks. Although from reading the header_checks config docs, it doesn't seem to suggest it'll affect the body content in anyway as you mentioned later.

I don't have time to investigate but wonder if this is what is actually affecting you? Perhaps try disabling that, it'll need to be done via postfix-master.cf override (although user-patches.sh could also handle it).

sender-cleanup/unix/header_checks=

That might work, otherwise to properly remove the entry from config instead of setting it to an empty value, you could try user-patches.sh with postconf:

postconf -PX sender-cleanup/unix/header_checks

Inspect /etc/postfix/master.cf and you should see that this -o line has been removed:

If that doesn't resolve it, then I assume it's the related header_checks config for outbound traffic smtp_header_checks, which is where I assume the DKIM signing would occur?:

Showing that header_checks is also configured in main.cf for some reason. Not sure why as that'd apply to any inbound mail from third-parties too? 🤷‍♂️ (while it is possible to submit mail to be sent outbound by sending to port 25 if PERMIT_DOCKER raises trust, it's not the correct approach, I know in the past we wrongfully supported SMTP auth on port 25 which would also have permitted it I think)

The original issue motivating this privacy focused addition doesn't discuss this much, nor the PR. I'm more inclined that the contribution has flaws.

DKIM milter is configured:

I'm not familiar with DKIM and milters well enough to comment on that, but OpenDKIM using non_smtpd_milters is apparently to run it for internal mail if something like sendmail is used (which behaves differently than swaks by dropping straight into the mail queue IIRC). While smtpd_milters is meant for signing?

Would be good to identify what part of that original PR config is causing this for you. It may be a fault of either of the main.cf referenced configs (user-patches.sh can remove those with postconf -X setting_name, no -P which is for master.cf).

The latter of these being the culprit for the broken signature. I understand that the inclusion of the header with every MIME part is not strictly necessary, but the modifying of body parts is something I can not accept

Looking at Postfix docs for milters, they appear to run during the smtpd stage while header_checks applies in the later cleanup stage.

References (for me):

• https://blog.tyk.nu/blog/postfix-and-privacy/
• postfix: adjust header checks for outgoing emails #3043 (comment)
• Improve the privacy of the client by removing sensitive details #708

If the cause can be better isolated and resolved, happy to accept a fix. Although we'd probably want a test-case which is going to add some friction AFAIK.

I understand you problem, but these defaults were and are working rather nicely for a majority of users. Overwriting the config is actually the "elegant" solution here.

I don't know if it's working rather nicely, or if the users just didn't notice/care about DMS sending mail out with invalid DKIM signatures (provided the body had MIME encoding headers).

I am not familiar with how that sort of content is produced, but I do recognize quoted-printable as related to this bug report.

If we can have swaks submit a mail that would have content to implicitly add those headers to the message body, we could probably reproduce a test-case for the test suite. DKIM only needs DNS for verification IIRC?

Thanks, will do.
Though my user-patches.sh is git pull --commit --autostash

@mpldr I think you pasted the wrong snippet there?

A separate bug report would be good to track this if anyone wants to do further investigation and/or contribute a fix.

The issue doesn't have to be too detailed, it can just link back to the comment discussion here for reference. Easier to track that way and for other users to discover (if it provides enough information for users searching issues when they're troubleshooting, not many are likely to search through PRs).

Since removing the line solves the issue, my money is on that it is actually the culprit

Could you be more specific please? Was it mime_header_checks? Or were you referring to header_checks?

There is several considerations, but since you reply by email I don't know if you see everything like the referenced code snippets or my edits after first posting a response.

Try identify if you can if it's due to any of these settings by removing them. I would assume either mime_header_checks but first might try just removing smtp_header_checks, technically I don't think header_checks in main.cf should be there either.

# main.cf:
postconf -X header_checks
# Quite possibly this one:
postconf -X smtp_header_checks
# Set to empty instead of default that uses header_checks:
postconf 'mime_header_checks='

# This one should be the last one to try, shouldn't be necessary to fix?
# master.cf on submission(s) ports (587/465):
postconf -PX sender-cleanup/unix/header_checks