Fix: Add SRS to fix SPF issues on redirect #611#814
Fix: Add SRS to fix SPF issues on redirect #611#814johansmitsnl merged 6 commits intodocker-mailserver:masterfrom jurekbarth:master
Conversation
|
Are the tests failing locally to? |
|
Thanks, yes it did in fact. |
|
I now get a lot of these errors: when I try to send an email. |
|
@Starbix do you have some custom stuff running aswell, maybe on port 10001? You can try running the following, to see if postsrsd is running: |
|
Looks like it's not running at all |
|
You may have a look at the log file. Once more sh into your container and run |
|
It looks like postsrs didn't start because |
|
ok it should be your mailserver domain |
|
|
|
yes it should be the same value. Have a look here where it's getting set: https://github.com/tomav/docker-mailserver/blob/master/target/postsrsd-wrapper.sh#L4 |
|
Ok the mails seem to be working again. |
|
Did it crash because of some quirks or is there something wrong with the code itself? |
|
No, there's nothing wrong with the code, docker is a bit confusing because it looks like the docker compose can't directly be translated into |
|
This did break my setup, because this new option does not respect the I did manage to fix this by changing the hostname of my container, but this causes a lot of inconveniences and I would really like to be able to change the name back in the future. Should I open a bug report about this? |
|
@Drakulix yes open a bug or if you could fix it submit a PR so that I can merge it. |
|
Using something like instead of might work. |
|
This takes also my overriden domain name into account. |
|
I don't know why, but this also breaks my setup. And the setup of several more: #852 #853 (at least for now that's what I assume) I don't know why though, never used the postsrsd. The response from the docker-mailserver was: |
|
I am also seeing some strange behavior after this. I'm using the OVERRIDE_HOSTNAME switch and my email addresses gets re-written from [email protected] -> [email protected] where the last part is the mail server domain. Any suggestions how to workaround this issue? |
|
Yes, a workaround would be to deactivate postsrsd. See: #852 (comment) |
|
I would investigate more, but i'm not able to let it fail. Maybe someone can provide more info about their setup? Can somebody with a failing setup check what is standing for them in this file? It's located here in your docker container: |
|
/etc/default/postsrsd: It doesn't change anything if I leave SRS_EXCLUDE_DOMAINS to the default value. mail:
image: tvial/docker-mailserver:latest
hostname: mail
domainname: ${DOMAINNAME}
container_name: mail
ports:
- "25:25"
- "143:143"
- "587:587"
- "993:993"
volumes:
- maildata:/var/mail
- mailstate:/var/mail-state
- ./mailserver/config/:/tmp/docker-mailserver/
- certData:/etc/letsencrypt/live/
environment:
- ENABLE_SPAMASSASSIN=${ENABLE_SPAMASSASSIN}
- ENABLE_CLAMAV=${ENABLE_CLAMAV}
- ENABLE_FAIL2BAN=${ENABLE_FAIL2BAN}
- ENABLE_POSTGREY=${ENABLE_POSTGREY}
- ONE_DIR=${ONE_DIR}
- DMS_DEBUG=${DMS_DEBUG}
- PERMIT_DOCKER=${PERMIT_DOCKER}
- POSTMASTER_ADDRESS=${POSTMASTER_ADDRESS}
- SSL_TYPE=letsencrypt
- NGINX_SHARED_SECRET=${NGINX_SHARED_SECRET}
- NGINX_DOWNLOAD_DOMAIN=downloads.${DOMAINNAME}
- SRS_EXCLUDE_DOMAINS=${SRS_EXCLUDE_DOMAINS}
cap_add:
- NET_ADMIN
- SYS_PTRACE
restart: always the variable values are: |
|
Ok that looks about right, at least i cannot see any issues here. Do you have problems receiving or sending emails? Because you wrote:
So do your setup include Gmail Account to send from your [email protected]? And that fails? Sorry i'm confused :) |
|
17Halbe, you should not need to override the domain name in postsrsd, IF your postfix config is set correctly. In fact the only settings needed in In your main.cf If you are relaying mail out to another server, such as your ISP, or Mailgun or REFLEXION or something like that, then you MUST disable spf on your domain, as they are doing the SPF. I personally have not pulled a resent pull, because my mailserver is working fine, but if I find time I may pull a new instance, to see if I have any issues. the 'HOSTNAME' of your real system should be set correctly. Docker will inherit this. On your docker host make sure that your /etc/hostname file is set to your proper host. When you deploy the docker image. and make sure you define it in the docker-compose.yml under domainname: Aside from this no additional magic is needed to make SRS working (at least based on my last pull when I originally posted the SRS info) which has worked well and as intended. |
|
No, the problem isn‘t a wrong rewriting, it‘s a rewriting wich shouldn‘t happen. Only way I could figure out to stop this behaviour was to deactivate postsrsd. Hostname is set correctly and shouldn‘t have an effect on the described behaviour anyway... I guess! ;) |
|
17Halbe, Enter SPF. SPF says that email can only be sent from authorized servers, so as postfix does not change the @A.com domain when it sends out @Mailserver.com, it fails SPF. SRS works around this by sending by taking the [email protected] and rewriting the address, actually rewriting the mail headers, so that the email comes "From" @Mailserver.com which matches your SPF settings. The description you just provided of [email protected] being overwritten to come from [email protected], sounds partly correct, though it would have actually been something closer to SRS0=HHH=TT=A.com=[email protected] This sounds exactly how its supposed to work. This allows you to have [email protected] sent to [email protected] for example. Your #814 comment, however shows a different error message from google, and not one that appears to be SRS related. Google does lots of other sanity checks on incoming mail, and yours was detected as bulk email, so effectively (google) things you are spamming, or has at some time associated your ip address with sending spam. Return-Path: [email protected] If you have set your docker host name properly , your MX records accordingly and are not using an outbound smtp relay your email should go out fine. My suspicion is that your real problem is either you are spamming people as google has responding by telling you you have been blocked for sending UBE, or you have not configured DNS or your systems hostname correctly. |
|
😆 😆 What's not a problem:
What is a problem:
Those (incoming) mails got rewritten to MY domain and are then not deliverable(localy on my server). What's helping:
The google mail I posted is a mail FROM my mailserver to the GMAIL user(who was trying to send me a mail). The Postmaster of my mailserver received the same mail. (This mail is only being sent with an active postsrsd) I might have a misconfig somewhere. But MX is set up correctly. My mailserverdomain is set up as shown above: hostname: mail
domainname: mydomain.com my MX pointing to mail.mydomain.com |
|
So one interesting thing on SRS is that it applies SRS to everything |
|
Hm, I’m receiving mails from the „whole big wide internet“ though, excluding every domain worldwide would be the same like disabling SRS alltogether. |
|
I think you have something else misconfigured honestly. You said incoming mail is being rewritten to your domain, but you should not be receiving mail at all unless it is your domain. (or one your the MX for)... SRS is specifically designed to work around a singular problem, as I stated in my first response. If you are not forwarding emails to other domains, using virtual tables, then there is no need for SRS, but even if you do deploy it, it wouldn't stop you from receiving email sent to you. |
|
Ok, I'll check again.. no the sender of the incoming mail is being rewritten as well! So from [email protected] it's being rewritten to [email protected].
That's my case (Just add some aliases in the mix ) |
|
Sorry for taking so long. Here is the error log from the server: Note:Working:
Not working:
What happens when sending from GMAIL -> Mailserver:
What solves this problem:
What's the cause?
UBE basically means the server blocked the mail, since it's considered spam because it either is sent as a mass mailing to huge numbers of recipients at a time or at least some of the intended recipients have not agreed to receive messages from the sender. |
|
Ok, couple of questions. |
|
Doesn't look like you have read the postsrsd page. Or my last comment. You really should. For the lazy. It rewrites all mail. Read the Readme. No its not happening on my setup, but I also understand how postfix works more then most. you should address the ube error, and read about what and how postsrsd works. Then you will understand how it interacts with other things. |
|
@tuxpowered as you just wrote that you know how it works, could you suggest a PR for this issue with a test that solves the issue? |
|
It's great that you
That's why I'm here! 😎 I'm sorry, I'm getting tired of solving this as well. And I read the SRS chapter, I used postsrsd before, but only for relaying servers. They are just not needed for servers which are serving only as a final destination. I explained like 3 times what the error is. I'm sorry, that it still wasn't clear enough.
I know that postsrsd is providing rewriting to EVERY address. It just shouldn't or my server should recognize the rewriting and consider rerewriting or routing anyway. I don't care how it would do it, as long as it would do it at all! 😉 So, as mentioned above, for a possible fix have a look at Postforward:
You're welcome 😉 |
|
Another problem might be, that postfixes |
|
Each mailserver is setup a little differently, and all to often you do not get all the information in posts like this because people will not submit unaltered configs, and I understand that. However, it does add complications. Postforward, is a different solution for a different problem, most notably if you are using other postfix features for forwarding like sieve and external scripts. PostSRSd will infact deliver to both local and forward destinations, IF it is acting as a simple mail server and not doing anything else. For example emailing me at [email protected] gets delivered locally to me. But emailing [email protected] gets forwarded to gmail.com, actually *@mydomain.com gets forwarded to gmail.com and only local mail accounts expressly configured deliver to local mailboxes. In addition to this, no configuration changes to the stock install of PostSRSd was needed. I do only operate 1 domain in the image, and maybe your comment of having multiple domains could be a valuable difference, but it really shouldn't be so long as you can send and receive from it. Remember that PostSRSd implements a specific function, SRS, If postforward helps you in your configuration go for it. The joy of these projects are they are intended to solve a unique problem. In my case I needed a lightweight simple mail server to do forwarding for a single domain and a few local boxes. I found that the project worked as a mailserver but that forwarding caused SPF issues, I did some research deployed PostSRS and the problem went away, And I updated everyone here on how to resolve that specific issue. Now if I were to be running multiple domains I would probably go a different route, and setup ISP style emails, if you want a really really good walk through on these types of thing and learn a lot, check out https://workaround.org/ispmail. And if I was doing the lazy-man hosting I might consider https://www.ispconfig.org/ which implements isp style mail web and so much more. Now I am not saying you should go run off and change to using any of them, just that picking the tool to do the job needed. but if your needs step out side the box, some times you cant expect the situation to work right. This is the exact reason I never created a pull request to implement the SRS action, because the use case was very specific and didn't really line up with the perceived goals fo this project. But felt that it would be useful for those that had a similar need, and did not want to turn off SPF. |
|
Thank you very much for your point of view and your explanation. I could give it a try and add postforward. I just don't have the time right now. |
|
I have not pulled an update in a while, is it enabled by default now? I would hope not, and would be for supporting an Option to maybe turn it on. |
|
It is enabled by default and there is no option to turn it off. When fixed it is prevent SPF fails on forwarded emails (sounds like a sane default). |
|
I'll submit a PR with a default for it being off. Is that alright for everyone? |
|
Just thinking out loud here ... It seems to me that it "should" be enabled if spf is enabled. The bigger issue would be there is no way to assure that spf has been enabled with out additional checks, and I think that is out side the scope of this project. At that point your looking at a more advanced configuration, which is kinda not what this project was intended for. Also checking multiple domains and things can get really complex. So IF we had an ENABLE_POSTSRSD option, the startup would need to alter main.cf to exclude the postsrs configs if set to 0 vs being set to 1. It would not 'technically' matter if postsrsd was running, although it would be ideal if it wasn't, for resource reasons. And that I think could be done with out breaking installations, but a setup note needs to be made some place so that future users know that if using SPF they need to enable postsrsd. Perhaps in the setup.sh script ? I agree with @johansmitsnl that if SPF is configured then postsrsd should be as well. I fear that handling SPF checks is beyond the scope of this project. Maybe the sanest of all options, is to use a full service mail package, if you need more advanced functions, that are not inline with this project? I for one would be opposed to adding to much and would agree with the project creator that those add-on things should be different projects that link to this docker, and not included in it. e.g. Running Roundcube or nginx or any other thing not really core. |
|
I for one do have SPF enabled and my setup only works if no SRS is taking place. So it's at least not true for everyone. ;) (But I don't forward mail anyway.. ¯_(ツ)_/¯) |
|
I wanted to setup SRS for cases where I do forwarding.... but it doesn't work at all if you also want to use SMTP: those emails also get the headers rewritten, which is wrong. So using PostSRS the way it's integrated seems to be only an option for cases where you want to forward, but only forward. |
This PR fixes an issue regarding redirecting emails to an external provider. Previously it ended in a softfail SPFs, because no sender rewriting was applied to the from address.
For example if you add the following to your config, you should end up with softfails
Partly this fixes issue #611 if i understood everything correctly. Also huge thanks to @tuxpowered for pointing into the right direction 👍
There are no tests atm, but i'm not sure how to test for something like this?