Score:1

Postfix: deliver mail in order

cn flag

I’ve got a setup where ordering of eMails matters (passing commit messages to a Fediverse client which converts them to Markdown and posts them).

On the sending side, I have my sendmail set up to retain the order:

define(`confQUEUE_SORT_ORDER', `time')dnl

This results in…

O QueueSortOrder=time

… in the resulting sendmail.cf file.

Now the recipient side is Postfix on Debian, which then uses a script using python3-mastodon to post, triggered like this in /etc/aliases:

foouser: |/usr/local/libexec/fooscript

However, incoming mails are not delivered to foouser in incoming order (the sending sendmail batches messages and sends them every couple of minutes, so this problem occurs when two (or more) eMails are sent to foouser in the same SMTP connection.

How can I configure Postfix to retain order for message delivery whenever possible, too? (I’m mostly a sendmail user, that’s why I wouldn’t know.)

anx avatar
fr flag
anx
(out of personal interest, your question is clear enough) Why, though? You got a UTC-translatable `Date:` header where the sending side tells you a date arguably more useful than the time of your eventual receipt, and you got an ideally globally-unique `Message-ID` headers to produce a stable sort order in case multiple messages are generated using the same timestamp of some underlying event.
mirabilos avatar
cn flag
@anx because when posting, it always uses the current date, I cannot backdate the post (or haven’t found the option), so I’d at least keep the order
Nikita Kipriyanov avatar
za flag
I feel like this is incorrect problem. No mail standard I can cite that requires a software to maintain this order. Therefore, software is absolutely free to reorder. Also, there is no guarantee on the limit of delay of messages; if first-composed arrives a minute later than second one, that's totally fine. You have to use artificial in-message metadata to guarantee order of processing and to avoid skipping delayed messages ("jitter" so to speak), the suggested `Date:` header is a start. Just don't expect such functionality to exist in the MTA and even it seems to be there, don't rely on it.
mirabilos avatar
cn flag
@NikitaKipriyanov as I already wrote, `sendmail` can be configured to send it first, and I had the question whether Postfix can also be configured. (Turns out it probably can.) Yes, this is not in the SMTP standard, but extensions are definitely allowed, and I just want “best effort”. Sure, if the delivery of the first message is delayed, I lose, but in _normal_ operation it won’t. Please don’t apply your preconceived opinions about what problems people are allowed to have, thanks.
Score:0
fr flag
anx

Even the "new" postfix queue manager - within a single destination - is documented to deliver in the expected order:

that is, FIFO unless there was some preemption involved -- SCHEDULER_README.html

You want to look at these settings for globally disabling the default concurrency (multiple threads) and preemption (out of order) mechanism:

default_destination_concurrency_limit = 1
default_delivery_slot_cost = 0

Mind man 5 postconf notes on configuring these per-transport. Which you want, if that server is also processing other mail, turning off both globally would be wasteful.

mirabilos avatar
cn flag
oic, the config has `default_destination_concurrency_limit = 4` currently; I’ll see if I can override that for… local, I suppose. (I cannot do that for just that one account, right? Too bad… oh wait, actual local accounts use the `tmail` transport from UW mailutils, so that should work well. Or do I need `pipe`?
anx avatar
fr flag
anx
@mirabilos I did not write add that in the answer, because I have not tested, and have been wrong about a different option before. I *suspect* you can do it for just one accounts: add a (renamed) copy of your existing transport to your `master.cf`, add ` -o option=value` overrides below, then redirect that specific destination there via transport_maps.
mirabilos avatar
cn flag
hmm, from the postconf manpage, I *thought* I would need `pipe_destination_concurrency_limit = 1` and `pipe_destination_recipient_limit = 1` but that doesn’t seem to be correct (`postconf: warning: /etc/postfix/main.cf: unused parameter`)
mirabilos avatar
cn flag
oic, I need xxx_ where xxx is the LHS of a thing in master.cf but pipe is an RHS, not LHS… so I need to figure out which transport is used for these (from the `relay=local` in syslog, probably local?)
anx avatar
fr flag
anx
@mirabilos Is your *transport* really named `pipe` ;) That is the name of the used daemon processes, but ideally would would have given it a more verbose name. The first column in [master.cf](https://www.postfix.org/master.5.html) - (edit) yes, sounds good.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.