Skip to content

Introduced auto. spam learning & global sieve filter#835

Closed
17Halbe wants to merge 6 commits intodocker-mailserver:masterfrom
17Halbe:antispam
Closed

Introduced auto. spam learning & global sieve filter#835
17Halbe wants to merge 6 commits intodocker-mailserver:masterfrom
17Halbe:antispam

Conversation

@17Halbe
Copy link
Copy Markdown
Contributor

@17Halbe 17Halbe commented Feb 12, 2018

Introduced auto-spam-learning via sieve.

Spam is automatically trained to spamassassin, when moved via imap-only to the spam folder.
And so is "Ham" when it's moved out of the Spam folder.

Additionally two new optional config files have been introduced:

  • global-before.dovecot.sieve
  • global-after.dovecot.sieve

The former being a global sieve file which is going to be executed before the user specific rules are applied and the later, you might have guessed it, is applied, after the user specific rules have been executed.

Attention:

This has not been tested with actual Spam, Ham movement. We need a testcase or at least someone to test it before merging, alone I don't know how to programmatically move mails between two imap-folders. Anyone else has a hint? Or can try this out?

Tests are passing on my machine.
closes #384

Copy link
Copy Markdown
Contributor

@johansmitsnl johansmitsnl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify the foldername and add some new lines to the end?

Comment thread target/dovecot/90-sieve.conf Outdated
# AntispamWithSieve (Derived from: https://wiki2.dovecot.org/HowTo/AntispamWithSieve)

# From elsewhere to Spam folder
imapsieve_mailbox1_name = Spam
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are so right.. Never base decisions on assumptions!!

Comment thread target/dovecot/90-sieve.conf Outdated

# From Spam folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Spam
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -0,0 +1,2 @@
#!/bin/sh
exec /usr/bin/sa-learn -u ${1} --ham No newline at end of file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline to the end of the file

@@ -0,0 +1,2 @@
#!/bin/sh
exec /usr/bin/sa-learn -u ${1} --spam No newline at end of file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline to the end of the file

set "username" "${1}";
}

pipe :copy ".sa-learn-ham.sh" [ "${username}" ]; No newline at end of file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline to the end of the file

set "username" "${1}";
}

pipe :copy ".sa-learn-spam.sh" [ "${username}" ]; No newline at end of file
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline to the end of the file

@johansmitsnl
Copy link
Copy Markdown
Contributor

Good PR and much wanted!

Not sure how to test this though.

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Feb 12, 2018

I don't know on those tests.
There is definitely a way to test that programmatically, client mail programs do that all the time. But I'll be having a lot less sparetime from tomorrow on until I guess mid September! O_o
So I won't be able to dive into "programming a imap-compatible mail client"
There is probably something flying around stackoverflow! ;P
But my suggestion would be, to just merge it and give it a try. It's not the nicest solution, but it might even work as intended!

@johansmitsnl
Copy link
Copy Markdown
Contributor

I tried testing but I have 2 questions:

  1. Should the dbpath be set to: --dbpath /var/mail-state/lib-amavis/.spamassassin for the sa-learn wiki?
  2. Unknown what the argument of the learn scripts are, but you should pass a file or a directory which you want to train or is this done trough the pipe?

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Feb 13, 2018

Yes, the respective mail will be piped to sa-learn.

And you are right. dbpath has to be set, I wasn't actually aware that we weren't using user specific spam-DBs.
Does that make sense though?
It's probably something to discuss somewhere else. But it might be worth considering giving every user it's own spam db?! (I'm ambivalent about it)
So for this PR I'll of course add the dbpath option. 👍
(So I can omit the user override as well.. :'( )
It just came to my mind, that we as well might have a permission problem... (Dovecot is executing sa-learn.)
I'll have a look the day after tomorrow..

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Feb 14, 2018

Please don't merge yet. I'll give this a try on a real mailserver..

Will take some time though. I'm really in a busy 2 months right now.

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Mar 1, 2018

Since learning every single mail at the time of movement is quite expensive (SA has to be started for every mail) I would suggest moving spam mails to a separate folder and delay learning to once a day by cron.
Any objections? Should we pick a specific time during the night or just a random @daily?

@johansmitsnl
Copy link
Copy Markdown
Contributor

johansmitsnl commented Mar 1, 2018

That sounds a good idea. On the wiki there is already something about this but it is only for one folder, not for all folders. #592

@johansmitsnl
Copy link
Copy Markdown
Contributor

@17Halbe will you update this branch or do you provide a new PR?

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Apr 2, 2018

Sorry for leaving you hanging! I really only get to work on this project while I‘m travelling. There‘s a lot of work related stuff I got to do till Friday. Thereafter I‘ll probably got a week off.
I‘ll update this branch. It‘s not been forgotten! ;)

@johansmitsnl
Copy link
Copy Markdown
Contributor

@17Halbe was just asking, no hurries ;)

@17Halbe
Copy link
Copy Markdown
Contributor Author

17Halbe commented Jul 9, 2018

Sorry for the "long time no see". I was sick and away for it. I won't be able to contribute for some time. Sorry. Feel free to close this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for dovecot-antispam plugin

3 participants