Score:1

Before-queue Content Filter won't inject data back to Postfix

cn flag

I have been building a before-queue filter with Postfix on NodeJS to retrieve the data from outgoing mails, remove the attachments, upload them on a CDN and inject back the URLs in the outgoing mail.

So far, nothing too complicated, I guess, as Postfix supports doing this via the Before-Queue Content Filter.

After implementing the filter and following the Postfix documentation, I am able to retrieve outgoing mails and to send them back to Postfix.

However, Postfix does not recognize the incoming mail via the "proxy"-specific configuration as being injected back via the before-queue content filter. It will still add it to the queue and send it properly but it won't close the session opened by the SMTP client at first.

I have been looking at this bug for some weeks now and browsed the entire web looking for answers, but this subject does not seem to be much documented. I saw that Milters could be another solution to the problem but it seems more complicated than just using SMTP to communicate with Postfix.

Here are the configuration files :

master.cf:

submission inet n                               -                               -                               -      smtpd
        -o syslog_name=postfix/submissions
        -o smtpd_proxy_filter=127.0.0.1:9830
        -o smtpd_proxy_ehlo=mail.clebard.cloud
        -o smtpd_proxy_options=speed_adjust
        -o smtpd_tls_auth_only=no
        -o smtpd_tls_dh1024_param_file=${config_directory}/dh2048.pem
        -o smtpd_enforce_tls=no
        -o smtpd_tls_security_level=may
        -o smtpd_sasl_auth_enable=yes
        -o smtpd_client_restrictions=permit_sasl_authenticated,reject
127.0.0.1:9821  inet    n                       -                       -                       -                      smtpd -v
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8
        -o syslog_name=postfix/afterfilter
        -o smtpd_enforce_tls=no
        -o smtpd_tls_auth_only=no
        -o smtpd_tls_dh1024_param_file=${config_directory}/dh2048.pem
        -o smtpd_enforce_tls=no
        -o smtpd_tls_security_level=may
        -o smtpd_sasl_auth_enable=yes
        -o smtpd_client_restrictions=permit_sasl_authenticated,reject

(this configuration is insecure to be used in production but I tried to remove as much complexity as possible)

127.0.0.1:9830 is the before-queue content filter server and I configured Postfix to listen for returning mails on 127.0.0.1:9821 with the second configuration of the file.

Here is what I send back to Postfix using my filter :

EHLO localhost
AUTH PLAIN {token}
XFORWARD HELO={client_name} NAME={client_name} ADDR={client_addr} PROTO=SMTP
XFORWARD IDENT={message_id}
MAIL FROM: {sender}
RCPT TO: {recipient}
DATA
...
.
QUIT

Here is the trace on the filter logs : Logs when a mail is sent using the filter

And here is what I have been seeing on Postfix : proxy-reject : queue file write error

All commands have been detected by Postfix when the filter sends them, but the data=0/1 response of the original inbound mail indicates that the DATA has not been received. In fact, the error seems to happen when the submission process tries to add the proxy content into the queue.

I hope that I explained everything clearly, as I would really need some help on this...

anx avatar
fr flag
anx
Quote a full set of logs - from postfix being (re)restarted to the message being rejected. I suspect *another* log line that clarifies what happened when trying to send the message to the filter - if not, add the `-v` flag to the receiving smtpd instance as well.
anx avatar
fr flag
anx
Not necessarily part of your current problem, but you *must* limit the number of simultaneous processes accepting messages (e.g.set the `maxproc` column in master.cf, you probably want that lower than the overall `default_maxproc_limit`). The default behaviour when freely competing for resources is inappropriate in SMTP context. Senders would experience unclear feedback when trying to send while CPU cycles, memory and/or filter processes are exhausted.
nkirchho avatar
cn flag
@anx Of course, here is the full log after reboot until the end of a mail sending : https://pastebin.com/TPsEN2K2
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.