Score:0

rspamd Whitelis/Blacklist per domain before filtering

cn flag

since I'm sadly not getting any responses in rspamd's GitHub discussion board, maybe someone here has experience with rspamd prefilter mode.

I'm trying to get rspamd to accept any mail coming from a given list of domains, no matter how much it thinks it's spam. Sadly, this is necessary as too many people think it's a good idea to manipulate senders or do other shady things, usually used by people having something to hide. This should be easiest to accomplish with the prefilter mode, as those are supposed to run before even evaluating if the mail is spam or not. As an added bonus, this could also be used for a block list if some notorious spam mails simply don't get recognized. Of course, rspamd can be taught what should be spam and what not, but that doesn't seem to be effective enough. Also, in logs, I can't figure out why it refuses to apply the prefilters for most domains , so maybe one of you has a suggestion.

Right now I'm using the multimap configuration to set up the prefilters for the accept and block list. So in local.d/multimap.conf I have:

ACCEPTLIST_SENDER_DOMAIN {
      #type = "header";
      #header = "from";
      type = "from";
      prefilter = "true";
      filter = "email:domain";
      map = "file:///etc/rspamd/local.d/acceptlist.sender.domain.map";
      description = "Sender domain is on allowed";
      action = "accept"; # Prefilter mode
  }

BLOCKLIST_SENDER_DOMAIN {
      #type = "header";
      #header = "from";
      type = "from";
      prefilter = "true";
      filter = "email:domain";
      map = "file:///etc/rspamd/local.d/blocklist.sender.domain.map";
      symbol = "BLOCKLIST_SENDER_DOMAIN";
      description = "Sender domain is on blocked";
      score = 10.0;
      action = "add_header"; # Prefilter mode
  }

The two comments at the top of each block is from when I tried to filter for header and then the from header, but since you can directly filter for that I tried this one, it doesn't help though. Also, I tried adding the files in map with both just the path and file:///path/to/file, still no success either way. The rspamd.log also doesn't show anything helpful (only error constantly showing up is about not being able to load dkim key, as I'm not able to disable that module). Copying this config to overwrite.d also doesn't help. Any ideas?

I'm on Debian 11 running rspamd 3.4-1~bullseye straight from the rspamd repo.

in flag
Whitelisting/blacklisting is usually configured directly in the MTA, before it is even handed to a antispam daemon.
Richard Rosner avatar
cn flag
This wasn't really the question. And I'd rather like to get working what's already there than figuring out a way to tell dovecot which mails it should forward to rspamd and which it shouldn't.
in flag
dovecot is not an MTA. You should learn which component is responsible for what.
Richard Rosner avatar
cn flag
Then let me put it this way: I don't care. If you don't have any constructive solution for the question, you simply shouldn't answer at all. Like I said, rspamd should be perfectly capable to, it simply doesn't work as supposed. I am searching for a fix for that, nothing else.
Score:1
cn flag

My last answer does help quite a bit, but overall I have found some issues still unsolved:

for some reason it still adds ACCEPTLIST_SENDER_DOMAIN for a domain I deleted a while ago from the map file. I even restarted rspamd multiple times, but it refuses to forget about that domain. And I can't find any cache that I could delete that would help.

Adding to this, the multimap is supposed to accept mails coming from specified domains no matter what. But only when the domain is in the From: header, not in any other case. The domain I previously falsely added to the whitelist was the domain of our mailing list. So rspamd should never have accepted those mails in the first place based on the multimap, as no mailinglist of ours will replace the From: header with its own address. But still, many spam mails get still marked as spam, even if in theory they should be excluded by the false ACCEPTLIST_SENDER_DOMAIN. For now, I've re-added the prefilter=true; to the multimap config, maybe this will help. But all in all, rspamd still doesn't behave as expected.

EDIT: to solve the first problem I've switched back to explicitly check the From: header, as type:from will check the envelope from before checking this, so this should be much more reliable (of course it's quite simple to modify the From header but I can count the incidents this happened in the past years that haven't been caught by our main spam filter on one hand):

ACCEPTLIST_SENDER_DOMAIN {
      type = "header";
      header = "From:";
      prefilter = true;
      map = "/etc/rspamd/local.d/acceptlist.sender.domain.map";
      symbol ="ACCEPTLIST_SENDER_DOMAIN";
      description = "Sender domain is on allowed";
      action = "accept"; # Prefilter mode
  }

BLOCKLIST_SENDER_DOMAIN {
      type = "header";
      header = "From:";
      prefilter = true;
      map = "/etc/rspamd/local.d/blocklist.sender.domain.map";
      symbol = "BLOCKLIST_SENDER_DOMAIN";
      description = "Sender domain is on blocked";
      action = "add_header"; # Prefilter mode
  }

But for some reason mails have the following in their header once these symbols are triggered:

X-Rspamd-Pre-Result: action=no action;
    module=multimap;
    Matched map: ACCEPTLIST_SENDER_DOMAIN

Action in this case clearly should be "accept", as stated in the config. This is how it looks like in the config dump:

multimap {
    ACCEPTLIST_SENDER_DOMAIN {
        action = "accept";
        map = "/etc/rspamd/local.d/acceptlist.sender.domain.map";
        description = "Sender domain is on allowed";
        symbol = "ACCEPTLIST_SENDER_DOMAIN";
        prefilter = true;
        type = "header";
        header = "From";
}
    BLOCKLIST_SENDER_DOMAIN {
        action = "add_header";
        map = "/etc/rspamd/local.d/blocklist.sender.domain.map";
        description = "Sender domain is on blocked";
        symbol = "BLOCKLIST_SENDER_DOMAIN";
        prefilter = true;
        type = "header";
        header = "From";
}
    [...]
}
Score:0
gt flag

I don't think that there should be any quotes around the prefilter value :

prefilter = true;

Also, configuring a score in a prefilter map may be useless as the action is already enforced.

I hope it helps !

Richard Rosner avatar
cn flag
thanks, I'll try that out.
Richard Rosner avatar
cn flag
@ webslider sadly that was not the solution.
Score:0
cn flag

@cz.steve I've now built everything back to use rspamd as SpamAssassin wasn't warking the way I wan to either. Now the changes do enable the white- and blacklisting, although not exactly the way I inteded it to work. What I wanted was that it matches the given domains only to what's in the From: header, not in all other headers too. Is it possible to achieve that?

Right now my configuration is this:

ACCEPTLIST_SENDER_DOMAIN {
      type = "from";
      filter = "email:domain";
      map = "/etc/rspamd/local.d/acceptlist.sender.domain.map";
      symbol ="ACCEPTLIST_SENDER_DOMAIN";
      description = "Sender domain is on allowed";
      action = "accept"; # Prefilter mode
  }

BLOCKLIST_SENDER_DOMAIN {
      type = "from";
      filter = "email:domain";
      map = "/etc/rspamd/local.d/blocklist.sender.domain.map";
      symbol = "BLOCKLIST_SENDER_DOMAIN";
      description = "Sender domain is on blocked";
      action = "add_header"; # Prefilter mode
  }
Richard Rosner avatar
cn flag
It seems I spoke too soon. I just got three emails from a domain that's clearly whitelisted, yet there is no "ACCEPTLIST_SENDER_DOMAIN" symbol in the header and they where classified as spam. It's possible that's for the same reason the symbol "FORGED_SENDER" was added (the domain from the whitelist is part of that other domain in the symbol). I've now added both domains to the whitelist but I'll have to wait a few days, then I'll get more emails from them to see if that was the solution.
Score:0
in flag

This should work. I managed to implement exactly what you want. My local.d/multimap.conf contains:

For accepting mails from pecific domains

WHITELIST_SENDER_DOMAIN {
      type = "from";
      filter = "email:domain";
      map = "/${LOCAL_CONFDIR}/local.d/whitelist.sender.domain.map";
      symbol ="WHITELIST_SENDER_DOMAIN";
      description = "WHITELIST map for SENDERS Domain - Action accept";
      action = "accept";
  }
# For rejecting domains
BLACKLIST_SENDER_DOMAIN {
      type = "from";
      filter = "email:domain";
      map = "/${LOCAL_CONFDIR}/local.d/blacklist.sender.domain.map";
      symbol ="BLACKLISTED_DOMAIN";
      description = "Blacklist map for Senders Domain - Action reject";
      action = "reject";
  } 

You need to create the mentioned map files in local.d with the exact same name and populate them with the domain names what you want to filter. Only one domain name per line! The files should have read permission.

Also, note that if an action is set on a message most further checks should be skipped.

Richard Rosner avatar
cn flag
This is exactly what I tried first, yet it only works in very few cases.
cz.steve avatar
in flag
On the web interface, when you analyze each email and the tags, the prefilter rule is simply not hit ? Are you populating the map files with the correct domain names?
cz.steve avatar
in flag
In email headers there s a big difference between "from" field and "received" field. "From" can be forged easily. From my experience prefilter rules hit the "receive" field, so make sure to analyze the spam email header file in detail. You may be receiving emails from services which send out batches of emails from multiple mailservers.
Richard Rosner avatar
cn flag
yes and yes. exactly that's the problem. And no, as long as rspamd doesn't purposefully ignore the type = "from"; or type = "header"; header = "from";, there shouldn't be any confusion. The main purpose for this map is to reduce false positives, so purposefully deceiving the sender isn't a problem. Even though more people than you might think do think it's a great default to forge sender, in the cases this mapping fails, it's not even the case. The whitelist map does contain the right domain that's in the From header of the mails in question, yet it completely ignores the map.
Richard Rosner avatar
cn flag
I already issued an issue from the discussion board of its GitHub, yet the developer just closed it in a very rude manner. He behaves like he's god and doesn't make mistakes. In the meanwhile, I already gave rspamd the boot and hope SpamAssassin does a better job at this.
cz.steve avatar
in flag
try to test it out with a dummy email address. Rspamd is a really powerfull free product, and the support is practically inexistent, because its a one man show only. You need to tinker about it. You sad that in some cases the multimap works. Can you paste an email header , for which it doesn t ?
Richard Rosner avatar
cn flag
Why on earth should I create a dummy mail adress when it's clearly not working without any way to tell what's wrong? Header can be found here https://controlc.com/6f936a0e And yes, the domain in the From header is in the acceptlist.sender.domain.map file
cz.steve avatar
in flag
very strange. Is the config loaded? do an rspamadm configdump
cz.steve avatar
in flag
in ACCEPTLIST_SENDER_DOMAIN you didn t declare the "symbol = ...;" also, remove the prefilter = true; line
Richard Rosner avatar
cn flag
Isn't the "prefilter=true" the thing that actually allows for checks before any symbol checks are done? I mean the whole point is that rspamd shouldn't even process any symbols to decide whether it's spam or not, so it won't even add a spam header etc. but lets the mail pass immediately.
cz.steve avatar
in flag
In theory, yes, you are right. Prefilter rules should be processed before other mailscan processes. I have both prefilter rules and simple filter rules, and what i notice is that they behave the same way. Once the rule is hit, no other symbols appear in the history. I don t have an exact mailflow diagram how rules and further checks are processed, nor did i ever find one. Is the rule loaded in you configdump? Also you could enable more detailed logging. Notice , or even info level logging.
Richard Rosner avatar
cn flag
Thanks for the explanation. As already explained, I'm currently using SpamAssassin. I haven't yet found out if it's whitelisting as supposed or not. Therefore I just told postfix to remove the typical spam headers if present before any further processing. Once figured out, I can reinstall rspamd to see if the changes work. I removed it without purging, it still has all the configs around.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.