Skip to content

Fix: Add SRS to fix SPF issues on redirect #611#814

Merged
johansmitsnl merged 6 commits intodocker-mailserver:masterfrom
jurekbarth:master
Feb 6, 2018
Merged

Fix: Add SRS to fix SPF issues on redirect #611#814
johansmitsnl merged 6 commits intodocker-mailserver:masterfrom
jurekbarth:master

Conversation

@jurekbarth
Copy link
Copy Markdown
Contributor

This PR fixes an issue regarding redirecting emails to an external provider. Previously it ended in a softfail SPFs, because no sender rewriting was applied to the from address.

For example if you add the following to your config, you should end up with softfails

# /config/postfix-virtual.cf

[email protected] [email protected]

Partly this fixes issue #611 if i understood everything correctly. Also huge thanks to @tuxpowered for pointing into the right direction 👍

There are no tests atm, but i'm not sure how to test for something like this?

@johansmitsnl
Copy link
Copy Markdown
Contributor

Are the tests failing locally to?

@jurekbarth
Copy link
Copy Markdown
Contributor Author

Thanks, yes it did in fact.

@johansmitsnl johansmitsnl merged commit e1e4542 into docker-mailserver:master Feb 6, 2018
@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

I now get a lot of these errors:

Feb 7 12:25:07 b79389992727 postfix/sender-cleanup/cleanup[15976]: warning: connect to TCP map localhost:10001: Cannot assign requested address

when I try to send an email.

@jurekbarth
Copy link
Copy Markdown
Contributor Author

@Starbix do you have some custom stuff running aswell, maybe on port 10001? You can try running the following, to see if postsrsd is running:

Sh into the container:
docker exec -it mail /bin/sh

Try to get running postsrsd process:
ps aux | grep postsrsd

Expected outcome (more or less):
postsrsd    25  0.0  0.0  12644   136 ?        S    11:35   0:00 /usr/sbin/postsrsd -f 10001 -r 10002 -d domain.test -s /etc/postsrsd.secret -a = -u postsrsd -p /var/run/postsrsd.pid -c /var/lib/postsrsd -D -X
root      3739  0.0  0.0  11112   924 pts/0    S+   11:48   0:00 grep postsrsd

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

Looks like it's not running at all

ps aux | grep postsrsd
root       100  0.0  0.0  11116   964 pts/0    S+   13:22   0:00 grep postsrsd

@jurekbarth
Copy link
Copy Markdown
Contributor Author

You may have a look at the log file. Once more sh into your container and run cat /var/log/supervisor/postsrsd.log that should bring up postsrsd errors

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

It looks like postsrs didn't start because SRS_DOMAIN= in /etc/default/postsrsd wasn't set.
What should be in there?

@jurekbarth
Copy link
Copy Markdown
Contributor Author

ok it should be your mailserver domain

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

hostname -d should equal example.com? or mail.example.com?

@jurekbarth
Copy link
Copy Markdown
Contributor Author

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

Ok the mails seem to be working again.
Thank you for you help!

@jurekbarth
Copy link
Copy Markdown
Contributor Author

Did it crash because of some quirks or is there something wrong with the code itself?

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 7, 2018

No, there's nothing wrong with the code, docker is a bit confusing because it looks like the docker compose can't directly be translated into docker run ...

@Drakulix
Copy link
Copy Markdown
Contributor

Drakulix commented Feb 8, 2018

This did break my setup, because this new option does not respect the OVERRIDE_HOSTNAME variable, which I was relying on.

I did manage to fix this by changing the hostname of my container, but this causes a lot of inconveniences and I would really like to be able to change the name back in the future.

Should I open a bug report about this?

@johansmitsnl
Copy link
Copy Markdown
Contributor

@Drakulix yes open a bug or if you could fix it submit a PR so that I can merge it.
Please add a test to ensure it is caught in the test for the future.

@Starbix
Copy link
Copy Markdown
Contributor

Starbix commented Feb 8, 2018

Using something like

DOMAINNAME=${$OVERRIDE_HOSTNAME:-`hostname -d`}

instead of

DOMAINNAME="$(hostname -d)"

might work.

@julianhille
Copy link
Copy Markdown

DOMAINNAME="$(hostname -d) why is it this way in the first place? It seems like the postsrsd init script does take care of it in a valid way.

SRS_DOMAIN=`postconf -h mydomain || true`

This takes also my overriden domain name into account.
Actually the SRS broke my complete system without really letting me know.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Feb 25, 2018

I don't know why, but this also breaks my setup. And the setup of several more: #852 #853 (at least for now that's what I assume) I don't know why though, never used the postsrsd.
The error is, that the address of a (foreign) sender([email protected]) to the server gets rewritten to the docker-mailservers domain: [email protected].
I'm sorry for not getting more in depth. But I'm really short on time currently.

The response from the docker-mailserver was:

A message from ... to ...
was considered unsolicited bulk e-mail (UBE).

Our internal reference code for your message is 01230-01/zcxxduaW

The message carried your return address, so it was either a genuine mail
from you, or a sender address was faked and your e-mail address abused
by third party, in which case we apologize for undesired notification.

We do try to minimize backscatter for more prominent cases of UBE and
for infected mail, but for less obvious cases some balance between
losing genuine mail and sending undesired backscatter is sought,
and there can be some collateral damage on either side.

First upstream SMTP client IP address: [xx8.171]
  mail-wr0-f171.google.com
According to a 'Received:' trace, the message apparently originated at:
  [20xx8.171], mail-wr0-f171.google.com mail-wr0-f171.google.com
  [20xx1]

Return-Path: <[email protected]>
From: XXX <[email protected]>
Message-ID: <[email protected]>
Subject: dfg

Delivery of the email was stopped!

@johndevs
Copy link
Copy Markdown

I am also seeing some strange behavior after this.

I'm using the OVERRIDE_HOSTNAME switch and my email addresses gets re-written from [email protected] -> [email protected] where the last part is the mail server domain.

Any suggestions how to workaround this issue?

@17Halbe 17Halbe mentioned this pull request Mar 1, 2018
@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 1, 2018

Yes, a workaround would be to deactivate postsrsd. See: #852 (comment)
@jurekbarth @tuxpowered Can you help on this?

@jurekbarth
Copy link
Copy Markdown
Contributor Author

I would investigate more, but i'm not able to let it fail. Maybe someone can provide more info about their setup?
The critical thing, which fails for a lot of people is that the SRS_DOMAIN is not set properly i think. It should happen on startup here: postsrsd-wrapper.sh

Can somebody with a failing setup check what is standing for them in this file? It's located here in your docker container: /etc/default/postsrsd

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 1, 2018

/etc/default/postsrsd:

# Default settings for postsrsd                                                                                                                                                                                      
                                                                                                                                                                                                                     
# Local domain name.                                                                                                                                                                                                 
# Addresses are rewritten to originate from this domain. The default value                                                                                                                                           
# is taken from `postconf -h mydomain` and probably okay.                                                                                                                                                            
#                                                                                                                                                                                                                    
SRS_DOMAIN=<My-Domain-1.net>                                                                                                                                                                                              
                                                                                                                                                                                                                     
# Exclude additional domains.                                                                                                                                                                                        
# You may list domains which shall not be subjected to address rewriting.                                                                                                                                            
# If a domain name starts with a dot, it matches all subdomains, but not                                                                                                                                             
# the domain itself. Separate multiple domains by space or comma.                                                                                                                                                    
#                                                                                                                                                                                                                    
SRS_EXCLUDE_DOMAINS=<My-Domain-2.net>,<My-Domain-1.net>                                                                                                                                                                        
                                                                                                                                                                                                                     
# First separator character after SRS0 or SRS1.                                                                                                                                                                      
# Can be one of: -+=                                                                                                                                                                                                 
SRS_SEPARATOR==                                                                                                                                                                                                      
                                                                                                                                                                                                                     
# Secret key to sign rewritten addresses.                                                                                                                                                                            
# When postsrsd is installed for the first time, a random secret is generated                                                                                                                                        
# and stored in /etc/postsrsd.secret. For most installations, that's just fine.                                                                                                                                      
#                                                                                                                                                                                                                    
SRS_SECRET=/etc/postsrsd.secret                                                                                                                                                                                      
                                                                                                                                                                                                                     
# Local ports for TCP list.                                                                                                                                                                                          
# These ports are used to bind the TCP list for postfix. If you change                                    
# these, you have to modify the postfix settings accordingly. The ports                                   
# are bound to the loopback interface, and should never be exposed on                                     
# the internet.                                      
#                                                    
SRS_FORWARD_PORT=10001                               
SRS_REVERSE_PORT=10002                               

# Drop root privileges and run as another user after initialization.                                      
# This is highly recommended as postsrsd handles untrusted input.                                         
#                                                    
RUN_AS=postsrsd                                      

# Jail daemon in chroot environment                  
CHROOT=/var/lib/postsrsd⏎                            

It doesn't change anything if I leave SRS_EXCLUDE_DOMAINS to the default value.
Postfix rewrites a valid gmail.com address to an My-server.net address, therefore fails(While sending from that Gmail account.)
Setup is as simple as it can get with two domains: Basic Docker-mailserver container with two domains setup via the adduser script of the container (setup.sh)
Docker-compose.yml:

mail:                                                                                                                                                                                                              
    image: tvial/docker-mailserver:latest                                                                                                                                                                                                                                                                                                                      
    hostname: mail                                                                                                                                                                                                   
    domainname: ${DOMAINNAME}                                                                                                                                                                                        
    container_name: mail                                                                                                                                                                                             
    ports:                                                                                                                                                                                                           
      - "25:25"                                                                                                                                                                                                      
      - "143:143"                                                                                                                                                                                                    
      - "587:587"                                                                                                                                                                                                    
      - "993:993"                                                                                                                                                                                                    
    volumes:                                                                                                                                                                                                         
      - maildata:/var/mail                                                                                                                                                                                           
      - mailstate:/var/mail-state                                                                                                                                                                                    
      - ./mailserver/config/:/tmp/docker-mailserver/                                                                                                                                                                 
      - certData:/etc/letsencrypt/live/                                                                                                                                                                              
    environment:                                                                                                                                                                                                     
      - ENABLE_SPAMASSASSIN=${ENABLE_SPAMASSASSIN}                                                                                                                                                                   
      - ENABLE_CLAMAV=${ENABLE_CLAMAV}                                                                                                                                                                               
      - ENABLE_FAIL2BAN=${ENABLE_FAIL2BAN}                                                                                                                                                                           
      - ENABLE_POSTGREY=${ENABLE_POSTGREY}                                                                                                                                                                           
      - ONE_DIR=${ONE_DIR}                                                                                                                                                                                           
      - DMS_DEBUG=${DMS_DEBUG}                                                                                                                                                                                       
      - PERMIT_DOCKER=${PERMIT_DOCKER}                                                                                                                                                                               
      - POSTMASTER_ADDRESS=${POSTMASTER_ADDRESS}                                                                                                                                                                     
      - SSL_TYPE=letsencrypt                                                                                                                                                                                         
      - NGINX_SHARED_SECRET=${NGINX_SHARED_SECRET}                                                                                                                                                                   
      - NGINX_DOWNLOAD_DOMAIN=downloads.${DOMAINNAME}                                                                                                                                                                
      - SRS_EXCLUDE_DOMAINS=${SRS_EXCLUDE_DOMAINS}                                                                                                                                                                   
    cap_add:                                                                                                                                                                                                         
      - NET_ADMIN                                                                                                                                                                                                    
      - SYS_PTRACE                                                                                                                                                                                                   
    restart: always                   

the variable values are:

ENABLE_SPAMASSASSIN=1                                
ENABLE_CLAMAV=0                                      
ENABLE_POSTGREY=0                                    
ONE_DIR=1                                            
DMS_DEBUG=1                                          
ENABLE_FAIL2BAN=1                                    
PERMIT_DOCKER=network                                
ENABLE_SASLAUTHD=0                                   
SRS_EXCLUDE_DOMAINS=<My-Domain-2.net>,<My-Domain-1.net>     

@jurekbarth
Copy link
Copy Markdown
Contributor Author

Ok that looks about right, at least i cannot see any issues here. Do you have problems receiving or sending emails? Because you wrote:

Postfix rewrites a valid gmail.com address to an My-server.net address, therefore fails(While sending from that Gmail account.)

So do your setup include Gmail Account to send from your [email protected]? And that fails? Sorry i'm confused :)

@tuxpowered
Copy link
Copy Markdown

17Halbe, you should not need to override the domain name in postsrsd, IF your postfix config is set correctly. In fact the only settings needed in

root@mail:/# cat /etc/default/postsrsd
# Default settings for postsrsd

# Local domain name.
# Addresses are rewritten to originate from this domain. The default value
# is taken from `postconf -h mydomain` and probably okay.
#
#SRS_DOMAIN=example.com

# Exclude additional domains.
# You may list domains which shall not be subjected to address rewriting.
# If a domain name starts with a dot, it matches all subdomains, but not
# the domain itself. Separate multiple domains by space or comma.
#
#SRS_EXCLUDE_DOMAINS=.example.com,example.org

# First separator character after SRS0 or SRS1.
# Can be one of: -+=
SRS_SEPARATOR==

# Secret key to sign rewritten addresses.
# When postsrsd is installed for the first time, a random secret is generated
# and stored in /etc/postsrsd.secret. For most installations, that's just fine.
#
SRS_SECRET=/etc/postsrsd.secret

# Local ports for TCP list.
# These ports are used to bind the TCP list for postfix. If you change
# these, you have to modify the postfix settings accordingly. The ports
# are bound to the loopback interface, and should never be exposed on
# the internet.
#
SRS_FORWARD_PORT=10001
SRS_REVERSE_PORT=10002

# Drop root privileges and run as another user after initialization.
# This is highly recommended as postsrsd handles untrusted input.
#
RUN_AS=nobody

# Jail daemon in chroot environment
CHROOT=/usr/local/lib/postsrsd

In your main.cf

# SPF policy settings
policyd-spf_time_limit = 3600
myhostname = <HOST.YOURDOMAIN.COM>
mydomain = <YOURDOMAIN.COM>

# Added postSRSd rules to process spf mail forwarding.
sender_canonical_maps = tcp:localhost:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:localhost:10002
recipient_canonical_classes = envelope_recipient,header_recipient
compatibility_level = 2

If you are relaying mail out to another server, such as your ISP, or Mailgun or REFLEXION or something like that, then you MUST disable spf on your domain, as they are doing the SPF.

I personally have not pulled a resent pull, because my mailserver is working fine, but if I find time I may pull a new instance, to see if I have any issues.

the 'HOSTNAME' of your real system should be set correctly. Docker will inherit this.

On your docker host make sure that your /etc/hostname file is set to your proper host. When you deploy the docker image. and make sure you define it in the docker-compose.yml under domainname:

Aside from this no additional magic is needed to make SRS working (at least based on my last pull when I originally posted the SRS info) which has worked well and as intended.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 3, 2018

No, the problem isn‘t a wrong rewriting, it‘s a rewriting wich shouldn‘t happen.
My Mailserver isn‘t relaying anything.
The scenario is as follows:
Address [email protected] is sending an Email to [email protected]
What then happens is that [email protected] gets (on my mailserver) rewritten to [email protected] which of course leads to a failure and a message to
[email protected]: #814 (comment)

Only way I could figure out to stop this behaviour was to deactivate postsrsd.

Hostname is set correctly and shouldn‘t have an effect on the described behaviour anyway... I guess! ;)

@tuxpowered
Copy link
Copy Markdown

17Halbe,
Postfix's virtual forward normally works the way you describe, [email protected] to [email protected]. It does this with no rewriting of the sender info.

Enter SPF. SPF says that email can only be sent from authorized servers, so as postfix does not change the @A.com domain when it sends out @Mailserver.com, it fails SPF.

SRS works around this by sending by taking the [email protected] and rewriting the address, actually rewriting the mail headers, so that the email comes "From" @Mailserver.com which matches your SPF settings.

https://en.wikipedia.org/wiki/Sender_Rewriting_Scheme

The description you just provided of [email protected] being overwritten to come from [email protected], sounds partly correct, though it would have actually been something closer to SRS0=HHH=TT=A.com=[email protected]

This sounds exactly how its supposed to work.

This allows you to have [email protected] sent to [email protected] for example.
When email arrives @gmail it will show up as coming from the original sender, with the To: being correctly returned to [email protected].

Your #814 comment, however shows a different error message from google, and not one that appears to be SRS related. Google does lots of other sanity checks on incoming mail, and yours was detected as bulk email, so effectively (google) things you are spamming, or has at some time associated your ip address with sending spam.

Return-Path: [email protected]
Do you have proper mail server information set up for DOCKER_MAILSERVER.DOMAIN ? DNS? MX? IF not, you will fail too.

If you have set your docker host name properly , your MX records accordingly and are not using an outbound smtp relay your email should go out fine. My suspicion is that your real problem is either you are spamming people as google has responding by telling you you have been blocked for sending UBE, or you have not configured DNS or your systems hostname correctly.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 3, 2018

😆 😆
Thanks a lot for your extensive answer.
However we got a communication problem! ;)
I know about the Rewriting Scheme, the rewriting works, but also for incoming mails with their final destination being my mailserver.

What's not a problem:

  • Sending Mails to the world

What is a problem:

  • receiving mails from the world

Those (incoming) mails got rewritten to MY domain and are then not deliverable(localy on my server).

What's helping:

  • deactivating SRS
    No errors whatsoever

The google mail I posted is a mail FROM my mailserver to the GMAIL user(who was trying to send me a mail). The Postmaster of my mailserver received the same mail. (This mail is only being sent with an active postsrsd)

I might have a misconfig somewhere. But MX is set up correctly. My mailserverdomain is set up as shown above:

 hostname: mail                                                                                                                                                                                                   
    domainname: mydomain.com           

my MX pointing to mail.mydomain.com

@tuxpowered
Copy link
Copy Markdown

So one interesting thing on SRS is that it applies SRS to everything
Are you excluding the incoming mail domains in SRS? are you doing mydomain.com or .mydomain.com?

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 4, 2018

Hm, I’m receiving mails from the „whole big wide internet“ though, excluding every domain worldwide would be the same like disabling SRS alltogether.
I did exclude my own domains, but that of course didn‘t solve the problem with the incoming mails being rewritten.
Might it be the case, that SRS should only be implemented if you are using this image as an relay?

@tuxpowered
Copy link
Copy Markdown

I think you have something else misconfigured honestly.
SRS will rewrite every email you receive, even local ones, and deliver just fine.
I for example run my server with several local accounts, and several aliases that are relayed to other domains (gmail, yahoo, etc) with no issues.

You said incoming mail is being rewritten to your domain, but you should not be receiving mail at all unless it is your domain. (or one your the MX for)...

SRS is specifically designed to work around a singular problem, as I stated in my first response. If you are not forwarding emails to other domains, using virtual tables, then there is no need for SRS, but even if you do deploy it, it wouldn't stop you from receiving email sent to you.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 4, 2018

Ok, I'll check again.. no the sender of the incoming mail is being rewritten as well! So from [email protected] it's being rewritten to [email protected].
And this is then rejected
But I'll check again, and just for your info:

If you are not forwarding emails to other domains, using virtual tables, then there is no need for SRS, but even if you do deploy it, it wouldn't stop you from receiving email sent to you.

That's my case (Just add some aliases in the mix )
I'll report back with a more coherent failure message. Thanks so far!

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 7, 2018

Sorry for taking so long. Here is the error log from the server:

postfix/postscreen[1170]: CONNECT from [74.125.82.42]:50848 to [xxx.xx.0.5]:25    
postfix/dnsblog[1174]: addr 74.125.82.42 listed by domain list.dnswl.org as 127.0.5.0                                                                                                     
postfix/postscreen[1170]: PASS NEW [74.125.82.42]:50848                           
postfix/smtpd[1218]: connect from mail-wm0-f42.google.com[74.125.82.42]           
postfix/smtpd[1218]: Anonymous TLS connection established from mail-wm0-f42.google.com[74.125.82.42]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)                      
policyd-spf[1226]: prepend Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=74.125.82.42; helo=mail-wm0-f42.google.com; [email protected]; receiver=<UNKNOWN> 
postfix/smtpd[1218]: 14D9F13A041F: client=mail-wm0-f42.google.com[74.125.82.42]   
postfix/cleanup[1201]: 14D9F13A041F: message-id=<[email protected]>                                                                                          
opendkim[268]: 14D9F13A041F: mail-wm0-f42.google.com [74.125.82.42] not internal  
opendkim[268]: 14D9F13A041F: not authenticated                                    
opendkim[268]: 14D9F13A041F: DKIM verification successful                         
opendkim[268]: 14D9F13A041F: s=20161025 d=gmail.com SSL                           
opendmarc[280]: 14D9F13A041F: gmail.com pass                                      
postfix/qmgr[1128]: 14D9F13A041F: from=<[email protected]>, size=3016, nrcpt=1 (queue active)                                                            
postfix/smtpd[1218]: disconnect from mail-wm0-f42.google.com[74.125.82.42] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7                                                       
postfix/smtpd[1200]: 9906813A0422: client=localhost[127.0.0.1]                    
postfix/cleanup[1201]: 9906813A0422: message-id=<[email protected]>                                                                                                        
postfix/smtpd[1200]: disconnect from localhost[127.0.0.1] ehlo=1 mail=2 rcpt=2 data=2 noop=1 quit=1 commands=9                                                                            
postfix/qmgr[1128]: 9906813A0422: from=<>, size=5875, nrcpt=1 (queue active)      
amavis[1130]: (01130-02) Blocked SPAM {BouncedInbound,Quarantined}, [74.125.82.42]:50848 [74.125.82.42] <[email protected]> -> <[email protected]>, quarantine: x/spam-x3f05wvgeFsn.gz, Queue-ID: 14D9F13A041F, Message-ID: <[email protected]>, mail_id: x3f05wvgeFsn, Hits: 7.624, size: 2820, 1382 ms
postfix/smtp[1192]: 14D9F13A041F: to=<[email protected]>, relay=127.0.0.1[127.0.0.1]:10024, delay=1.8, delays=0.44/0/0.01/1.4, dsn=2.5.0, status=sent (250 2.5.0 Ok, id=01130-02, BOUNCE) 
postfix/qmgr[1128]: 14D9F13A041F: removed                                         
postfix/smtp[1209]: Trusted TLS connection established to gmail-smtp-in.l.google.com[74.125.133.26]:25: TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)                    
postfix/smtp[1209]: 9906813A0422: to=<[email protected]>, orig_to=<[email protected]>, relay=gmail-smtp-in.l.google.com[74.125.133.26]:25, delay=0.78, delays=0.24/0/0.18/0.36, dsn=2.0.0, status=sent (250 2.0.0 OK 1520433476 n14si4105555wra.431 - gsmtp)
postfix/qmgr[1128]: 9906813A0422: removed                                         

Note:

Working:

  • Sending Mailserver -> GMail(Or any other MTA)

Not working:

  • Sending GMAIL -> Mailserver(That's where the log is from)

What happens when sending from GMAIL -> Mailserver:

  • Mail with following content: (Sent from MY_MAILSERVER)
to: alex@MY_MAILSERVER

was considered unsolicited bulk e-mail (UBE).

Our internal reference code for your message is 01130-02/x3f05wvgeFsn

The message carried your return address, so it was either a genuine mail
from you, or a sender address was faked and your e-mail address abused
by third party, in which case we apologize for undesired notification.

We do try to minimize backscatter for more prominent cases of UBE and
for infected mail, but for less obvious cases some balance between
losing genuine mail and sending undesired backscatter is sought,
and there can be some collateral damage on either side.

First upstream SMTP client IP address: [74.125.82.42] mail-wm0-f42.google.com
According to a 'Received:' trace, the message apparently originated at:
  [74.125.82.42], mail-wm0-f42.google.com mail-wm0-f42.google.com
  [74.125.82.42]

Return-Path: <SRS0=f0iH=F5=gmail.com=gmail_user@MY_MAILSERVER>
From: Alex <[email protected]>
Message-ID: <[email protected]>
Subject: asdf

Delivery of the email was stopped!

What solves this problem:

  • deactivating SRS

What's the cause?

UBE basically means the server blocked the mail, since it's considered spam because it either is sent as a mass mailing to huge numbers of recipients at a time or at least some of the intended recipients have not agreed to receive messages from the sender.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 12, 2018

Ok, couple of questions.
Is SRS supposed to rewrite an incoming mailaddress where the mailserver is the final destination?
Is this happening on your setup as well? Do mails still get delivered to your mailserver as the final destination with rewriting?
When or how does postsrsd decide to rewrite mailaddresses?
Since I'm not the only one, would you suggest to just invent a new env variable?

@tuxpowered
Copy link
Copy Markdown

Doesn't look like you have read the postsrsd page. Or my last comment. You really should.

https://github.com/roehling/postsrsd/blob/master/README.md

For the lazy. It rewrites all mail. Read the Readme.

No its not happening on my setup, but I also understand how postfix works more then most. you should address the ube error, and read about what and how postsrsd works. Then you will understand how it interacts with other things.

@johansmitsnl
Copy link
Copy Markdown
Contributor

@tuxpowered as you just wrote that you know how it works, could you suggest a PR for this issue with a test that solves the issue?

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 13, 2018

It's great that you

understand how postfix works more then most.

That's why I'm here! 😎

I'm sorry, I'm getting tired of solving this as well. And I read the SRS chapter, I used postsrsd before, but only for relaying servers. They are just not needed for servers which are serving only as a final destination.

I explained like 3 times what the error is. I'm sorry, that it still wasn't clear enough.
I just realized, if you read the link you provided till the Known Issues chapter it just might be my problem.

Due to the way PostSRSd is integrated with Postfix, sender addresses will always be rewritten even if the mail is not forwarded at all. This is because the canonical maps are read by the cleanup daemon, which processes mails at the very beginning before any routing decision is made.
Where piping into an external command is not a problem, Postforward offers an alternative way to integrate PostSRSd with Postfix which avoids this problem.

I know that postsrsd is providing rewriting to EVERY address. It just shouldn't or my server should recognize the rewriting and consider rerewriting or routing anyway. I don't care how it would do it, as long as it would do it at all! 😉
Whatever, rewriting an incoming mail from an @gmail.com address to @myserver.net when myserver is the destination of the gmail mail, is very likely to cause routing problems(I actually don't see how it wouldn't)

So, as mentioned above, for a possible fix have a look at Postforward:

Postforward is a mail forwarding utility which aims to compliment the Postfix Sender Rewriting Scheme daemon (PostSRSd).
The downside of using PostSRSd is that all mail is naively rewritten, even when no forwarding is actually performed. Such rewritten Return-Path addresses may confuse sieve scripts and other mail filtering software.
This is where Postforward comes in. Instead of rewriting all incoming mail regardless of final destination, mail systems may be configured to pipe mail into Postforward only when forwarding needs to happen, leaving non-forwarded mail unaltered by PostSRSd. Postforward will rewrite envelope addresses for piped mail using PostSRSd itself and re-inject these messages back into the queue, destined for the forwarding recipient(s).

You're welcome 😉
No, really, thanks for helping me and staying on the problem for so long. I know it's hard to focus on a "OPP" that you can't even reproduce!

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 13, 2018

Another problem might be, that postfixes mydomain only points to one of my two domains. But I don't recall, that writing to my mydomain was making any difference.

@tuxpowered
Copy link
Copy Markdown

Each mailserver is setup a little differently, and all to often you do not get all the information in posts like this because people will not submit unaltered configs, and I understand that. However, it does add complications.

Postforward, is a different solution for a different problem, most notably if you are using other postfix features for forwarding like sieve and external scripts.

PostSRSd will infact deliver to both local and forward destinations, IF it is acting as a simple mail server and not doing anything else.

For example emailing me at [email protected] gets delivered locally to me. But emailing [email protected] gets forwarded to gmail.com, actually *@mydomain.com gets forwarded to gmail.com and only local mail accounts expressly configured deliver to local mailboxes.

In addition to this, no configuration changes to the stock install of PostSRSd was needed.

I do only operate 1 domain in the image, and maybe your comment of having multiple domains could be a valuable difference, but it really shouldn't be so long as you can send and receive from it.

Remember that PostSRSd implements a specific function, SRS, If postforward helps you in your configuration go for it. The joy of these projects are they are intended to solve a unique problem.

In my case I needed a lightweight simple mail server to do forwarding for a single domain and a few local boxes. I found that the project worked as a mailserver but that forwarding caused SPF issues, I did some research deployed PostSRS and the problem went away, And I updated everyone here on how to resolve that specific issue.

Now if I were to be running multiple domains I would probably go a different route, and setup ISP style emails, if you want a really really good walk through on these types of thing and learn a lot, check out https://workaround.org/ispmail. And if I was doing the lazy-man hosting I might consider https://www.ispconfig.org/ which implements isp style mail web and so much more.
My main mail server operates in the above manner and not in this image

Now I am not saying you should go run off and change to using any of them, just that picking the tool to do the job needed. but if your needs step out side the box, some times you cant expect the situation to work right.

This is the exact reason I never created a pull request to implement the SRS action, because the use case was very specific and didn't really line up with the perceived goals fo this project. But felt that it would be useful for those that had a similar need, and did not want to turn off SPF.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 14, 2018

Thank you very much for your point of view and your explanation. I could give it a try and add postforward. I just don't have the time right now.
So since even you suggested, that this PR was for a quite specific usecase, Would you support a switch (env variable) to turn it off(or better on and leave the default to be off)?

@tuxpowered
Copy link
Copy Markdown

I have not pulled an update in a while, is it enabled by default now? I would hope not, and would be for supporting an Option to maybe turn it on.

@johansmitsnl
Copy link
Copy Markdown
Contributor

It is enabled by default and there is no option to turn it off. When fixed it is prevent SPF fails on forwarded emails (sounds like a sane default).

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 16, 2018

I'll submit a PR with a default for it being off. Is that alright for everyone?
Problem is, that the default right now is 'enabled' and that would break the setup for the guys already working with postsrsd.
So I guess this would then be released with a new version..

@tuxpowered
Copy link
Copy Markdown

Just thinking out loud here ...
I took a peak at the docker build, looks like it is installed from apt (not sure thats the current version)

It seems to me that it "should" be enabled if spf is enabled.
It would also seem that the only sane way to provide an option to toggle it would be to modify the setup and postsrsd wrapper, to do a sanity check as to not break installs, that might be a tricky task.

The bigger issue would be there is no way to assure that spf has been enabled with out additional checks, and I think that is out side the scope of this project. At that point your looking at a more advanced configuration, which is kinda not what this project was intended for. Also checking multiple domains and things can get really complex.

So IF we had an ENABLE_POSTSRSD option, the startup would need to alter main.cf to exclude the postsrs configs if set to 0 vs being set to 1. It would not 'technically' matter if postsrsd was running, although it would be ideal if it wasn't, for resource reasons. And that I think could be done with out breaking installations, but a setup note needs to be made some place so that future users know that if using SPF they need to enable postsrsd. Perhaps in the setup.sh script ?

I agree with @johansmitsnl that if SPF is configured then postsrsd should be as well. I fear that handling SPF checks is beyond the scope of this project.

Maybe the sanest of all options, is to use a full service mail package, if you need more advanced functions, that are not inline with this project?

I for one would be opposed to adding to much and would agree with the project creator that those add-on things should be different projects that link to this docker, and not included in it. e.g. Running Roundcube or nginx or any other thing not really core.

@17Halbe
Copy link
Copy Markdown
Contributor

17Halbe commented Mar 16, 2018

I for one do have SPF enabled and my setup only works if no SRS is taking place. So it's at least not true for everyone. ;) (But I don't forward mail anyway.. ¯_(ツ)_/¯)
I already implemented the main.cf configuration and the deactivation of postsrsd if not needed.
What about pointing to and explaining that env variable in the README file besides a wiki entry? So everyone new would become alerted that if they got a problem sending or receiving mail, they should give this option a try.
What do you think?

@xavivars
Copy link
Copy Markdown

xavivars commented Jun 3, 2022

I wanted to setup SRS for cases where I do forwarding.... but it doesn't work at all if you also want to use SMTP: those emails also get the headers rewritten, which is wrong.

So using PostSRS the way it's integrated seems to be only an option for cases where you want to forward, but only forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants