Join Log in
Score:0
Yanko Hernández Álvarez avatar Server

Duplicate message delivery to always_bcc with dovecot and Simple Content Filter (Postfix)

ru flag
Yanko Hernández Álvarez

I'm trying to set up a very low traffic postfix mail server using:

  • A Simple content filter (http://www.postfix.org/FILTER_README.html#simple_filter). It's way simpler than the advanced one and the traffic is low.
  • dovecot as a delivery agent. (Because of maildir quotas and sieve filters)
  • always_bcc to keep copies of every mail sent. (Bussiness rules)

My relevant configs are:

master.cf

smtp      inet  n       -       y       -       -       smtpd
        -o content_filter=filter:dummy
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       y       -       -       smtp
relay     unix  -       -       y       -       -       smtp
        -o syslog_name=postfix/$service_name
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
postlog   unix-dgram n  -       n       -       1       postlogd
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
filter    unix  -       n       n       -       10      pipe
  flags=Rq user=filter null_sender=
  argv=/var/spool/filter/scripts/filter.sh -f ${sender} -- ${recipient}
dovecot    unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${recipient}

main.cf (relevant parts only)

always_bcc = [email protected]
virtual_transport = dovecot
dovecot_destination_recipient_limit = 1

/var/spool/filter/scripts/filter.sh

#!/bin/sh

/usr/bin/cat | /var/spool/filter/scripts/mailfilter | /usr/sbin/sendmail -G -i "$@"

exit $?
  • All my users have emails in "(Initial)[email protected]" format
  • There are security groups that act as roles and each one has emails in "[email protected]" format.
  • Basically, a role email function as an alias to several users.

So, I can reuse the same Roles/Groups I use to set permissions, and not using the users directly.

For example

  • my user email is [email protected]
  • I'm a member of role I.T.Department ([email protected])
  • I'm a member of some other roles, but my main role is "I.T.Department"
  • Because I'm a member of this group/role I have certain privileges, as the other members and every mail sent to [email protected] will reach me and the other members.

So far, so good... But my boss wants that every mail I send uses the [email protected] (role) address instead of [email protected] (user). i.e., that every mail sent uses the principal role email, not the user email.

/var/spool/filter/scripts/mailfilter is an in-house made filter that just does that (changes "from" addresses from "(Initial).surname@" to "role@" checking an LDAP server).

It never fails (In case of any error condition, it just copies standard input into standard output and no modification is made, so no address change, but the mail is not lost).

The problem is, when I send an email, it delivers two copies to [email protected] as seen in the logs, not one copy as expected.

Dec 16 16:23:08 correo2 postfix/smtpd[32453]: connect from informatica1.mydomain.example[10.128.159.15]
Dec 16 16:23:08 correo2 postfix/smtpd[32453]: DD925380C87: client=informatica1.mydomain.example[10.128.159.15], sasl_method=LOGIN, [email protected]
Dec 16 16:23:08 correo2 postfix/cleanup[32456]: DD925380C87: message-id=<[email protected]>
Dec 16 16:23:09 correo2 postfix/qmgr[32275]: DD925380C87: from=<[email protected]>, size=2749, nrcpt=2 (queue active)
Dec 16 16:23:09 correo2 postfix/pickup[32274]: AAFEF380C8D: uid=110 from=<[email protected]>
Dec 16 16:23:09 correo2 postfix/pipe[32457]: DD925380C87: to=<[email protected]>, relay=filter, delay=0.81, delays=0.11/0/0/0.7, dsn=2.0.0, status=sent (delivered via filter service)
Dec 16 16:23:09 correo2 postfix/pipe[32457]: DD925380C87: to=<[email protected]>, orig_to=<[email protected]>, relay=filter, delay=0.81, delays=0.11/0/0/0.7, dsn=2.0.0, status=sent (delivered via filter service)
Dec 16 16:23:09 correo2 postfix/qmgr[32275]: DD925380C87: removed
Dec 16 16:23:09 correo2 postfix/cleanup[32456]: AAFEF380C8D: message-id=<[email protected]>
Dec 16 16:23:09 correo2 postfix/qmgr[32275]: AAFEF380C8D: from=<[email protected]>, size=2927, nrcpt=3 (queue active)
Dec 16 16:23:10 correo2 dovecot: lda([email protected])<32488><2DSfML3hnGPofgAA0V72BQ>: msgid=<[email protected]>: saved mail to INBOX
Dec 16 16:23:10 correo2 postfix/pipe[32473]: AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.28, dsn=2.0.0, status=sent (delivered via dovecot service)
Dec 16 16:23:10 correo2 dovecot: lda([email protected])<32490><Au2fML3hnGPqfgAA0V72BQ>: msgid=<[email protected]>: saved mail to INBOX
Dec 16 16:23:10 correo2 postfix/pipe[32471]: AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.34, dsn=2.0.0, status=sent (delivered via dovecot service)
Dec 16 16:23:10 correo2 dovecot: lda([email protected])<32489><BE2gML3hnGPpfgAA0V72BQ>: msgid=<[email protected]>: saved mail to INBOX
Dec 16 16:23:10 correo2 postfix/pipe[32470]: AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.35, dsn=2.0.0, status=sent (delivered via dovecot service)
Dec 16 16:23:10 correo2 postfix/qmgr[32275]: AAFEF380C8D: removed
Dec 16 16:23:11 correo2 postfix/smtpd[32453]: disconnect from informatica1.mydomain.example[10.128.159.15] ehlo=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=6

uid=110 is "filter" user uid. "From" is only changed in the headers, inside of the email, so it doesn't show on the logs.

Everything is running as it should be except this double delivery detail. I haven't tried, but I am sure that if I remove dovecot from the equation, it all works. But I lose the sieve rules and maildir quotas. So...

Any idea of how I can fix it?

331
1 + 0
Score:0
Aubs avatar Server
cn flag
Aubs

Tl;dr: The original email arrives at Postfix which adds always_bcc and sends the copy, drops the email as per filter. Your filter sends a new message to Postfix which adds always_bcc and sends the copy, then delivers the email.

Read the last three paragraphs.

The first email that hits Postfix will be the original DD925380C87, so it sends a copy of that straight to seccopies as per always_bcc rule. The second AAFEF380C8D is the one translated by the filter (changing the From header to your group) and is sent to seccopies too, but twice, because the first email already had the BCC of seccopies, and now Postfix thinks it needs to send to it to the BCC and seccopies so it gets it again.

So you should be getting three copies to seccopies, but 2 and [3] are identified as the same email (AAFEF380C8D):

  1. the first from y.hernandez to y.hernandez >> always_bcc seccopies
  2. the second from informatics to to y.hernandez >> always_bcc seccopies
  3. the third from informatics to seccopies >> always_bcc seccopies (as a BCC of 2)

The single copy of the original is delivered to seccopies:

**DD925380C87**: to=<[email protected]>, relay=filter, delay=0.81, delays=0.11/0/0/0.7, dsn=2.0.0, status=sent (delivered via filter service)

The line with uid=110 is where 2 is created, and the two deliveries, one because it is TO y.hernandez and one because it is BCCd to seccopies are on these lines:

AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.34, dsn=2.0.0, status=sent (delivered via dovecot service)
AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.28, dsn=2.0.0, status=sent (delivered via dovecot service)

And finally, the modified email is delivered to the intended recipient:

AAFEF380C8D: to=<[email protected]>, relay=dovecot, delay=1.1, delays=0.79/0/0/0.35, dsn=2.0.0, status=sent (delivered via dovecot service)

Deduping would prevent the two AAFEF380C8D emails from being delivered to seccopies, but I don't think it would stop the DD925380C87 email, because that's a separate email.

I think you need to follow this instruction, so that the content filter sees the original mail addresses instead of the result of ... automatic bcc, it'll then re-inject the email into Postfix which will then apply the always_bcc:

/etc/postfix/main.cf:
content_filter = scan:localhost:10025
receive_override_options = no_address_mappings

From the receive_override_options: when the "BEFORE content filter" receive_override_options setting is specified in the main.cf file, specify the "AFTER content filter" receive_override_options setting in master.cf (and vice versa).

+ 7
Yanko Hernández Álvarez avatar
ru flag
Yanko Hernández Álvarez
DD925380C87 is never delivered to my mailbox, it is just sent to the filter script as one email with this parameters to the filter: (-f [email protected] -- [email protected] [email protected]) It is never delivered to seccopies' mailbox. What ends in seccopies' mailbox are two identical copies of the modified email (AAFEF380C8D) . So, no dedup here. And that's is the problem. If I remove dovecot delivery from the equation, it goes as it should, only one message delivered.
0
Reply
Yanko Hernández Álvarez avatar
ru flag
Yanko Hernández Álvarez
I think the problem lies in the dovecot_destination_recipient_limit = 1. BUT it is required because dovecot only accepts one recipient mailbox. Or that's how it supposed to be years ago. I will recheck tomorrow.
0
Reply
Yanko Hernández Álvarez avatar
ru flag
Yanko Hernández Álvarez
On the other hand, I don't want two postfix copies running, that's why I chose the simple filter. I have another way to attack the problem (in the filter script: remove [email protected] from the list of recipients to "/usr/sbin/sendmail -G -i " or eliminate always_bcc and put [email protected] directly on the script, and one of these will -probably- work) but I wanna to try all other options first (no special cases).
0
Reply
Aubs avatar
cn flag
Aubs
Sorry, yes, I see "DD925380C87: to=<[email protected]>, relay=filter". I already discounted **DD925380C87** going to your mailbox as I saw it had *relay=filter*, I just missed it on the `seccopies` - In that case, ignore that element! I still suspect Postfix is sending a copy of the email to the `seccopies` mailbox that is already BCCd in from before the filter, and an additional copy after the filter because it is a new email. I still think it would be worth setting `no_address_mappings` in main.cf, let the filter tun, then re-enable `address_mappings` after the filter.
0
Reply
Yanko Hernández Álvarez avatar
ru flag
Yanko Hernández Álvarez
" I still think it would be worth setting no_address_mappings in main.cf, let the filter run, then re-enable address_mappings after the filter." How can I do that using a simple filter for every email received? is there any way to apply different config depending if the email was received using smtp o using /usr/sbin/sendmail (something in master.cf I guess)? That would be a solution (apply always_bcc only to /usr/sbin/sendmail). The filter will take care of the email received using smtp.
0
Reply
Aubs avatar
cn flag
Aubs
How are you calling your existing filter?
0
Reply
Yanko Hernández Álvarez avatar
ru flag
Yanko Hernández Álvarez
inside /var/spool/filter/scripts/filter.sh
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.