Score:0

Server

Postfix header_checks with multi-line headers and capture group

reactive-core

5/31/24, 5:53 PM

We have a header_checks regexp passed to the "cleanup" process, but it's not processing the header as a single header if it's on multiple lines.

header_check (pass value of header to FILTER):

/^X-OurHeader:\s+(.*?)/ FILTER $1

master.cf:

cleanup_service.        unix  n       -       y       -       0       cleanup
  -o header_checks=regexp:/etc/postfix/header_check

Example multi-line header where it fails:

X-OurHeader: a_special_filter_command:my.sub.domain
 b.mail.whatever.com

The original full header before getting multi-lined is:

X-OurHeader: a_special_filter_command:my.sub.domainb.mail.whatever.com

The result passed to FILTER is then: a_special_filter_command:my.sub.domain and missing the remaining hostname from the second line b.mailwhatever.com.

If I change the header_check to this instead, so it captures the next line:

/^X-OurHeader:\s+([\s\S]*)/ FILTER $1

The $1 capture group then contains the second line, but there's still a space (and presumably newline) in the captured text as well, so that doesn't work. That looks like this:

X-OurHeader: a_special_filter_command:my.sub.domain b.mail.whatever.com

Looking in the logs with the first approach, we see

postfix/cleanup[123456]: 27429A1FE8: filter: header X-OurHeader: a_special_filter_command:my.sub.domain? b.mail.whatever.com

Is there an alternative approach to this that anyone might suggest? The end goal is that we tag a header with a special cmd:hostname values that needs to be sent to the FILTER, but want to support that hdr value being broken up to multiple lines for a long header to meet the "length SHOULD be < 78" standard.

126

1 + 5

postfix

cleanup

headers

anx

5/31/24, 7:23 PM

Have you considered *unconditionally* injecting an early FWS, so you can just not care because the rest of the pattern will always fit and always have the same whitespace?

anx

5/31/24, 7:37 PM

Use `printf 'Header: Value\r\n\r\n' | postmap -h -q foo pcre:/path/to/bar.pcre -` to make your life easier as you verify a postfix (-h = header check) lookup in isolation.

reactive-core

5/31/24, 8:28 PM

I guess I consider my question to be more of an "am I not processing headers right?" question, because the documentation indicates that multi-line headers are fully supported. I'm just not sure why the header is passed to my header_check as a line with newlines/space in it, instead of a single unified header. Maybe a bug?

reactive-core

5/31/24, 8:29 PM

Maybe you're correct, and this just isn't the proper way to do this. I'm generating the header internally, so I know it will always have safe values. In fact, for now, the quick fix is to not split the header up onto multiple lines and just ignore the standard. It seems like most other services ignore this "should < 78 char" limit anyway.

anx

5/31/24, 9:06 PM

Trusting that the header you are looking at was inserted by you is a popular location for pitfalls in HTTP (reverse) proxy context, partly applicable also to emails. You may be generating one instance of some header internally, but are you prepared to deal with the mess of a malformatted multipart message containing your header multiple times in multiple places, some only recognized as such after a change in handling invalid UTF8-sequences after editing the message in other places?

Score:1

Server

anx

5/31/24, 7:16 PM

If you know or can ensure there is only a certain number of variations, add additional expressions to catch those separately /..:(.*) (.*)$/ -> /..:$1$2/ - postfix will continue with the next expression if there is no space, so keep the patterns simple by putting the complex match first.

If not but still within regular expressions power, there may be a way to chain multiple lookups. From the top of my head I can not piece together the full thing, but if its available on your distribution and works at all, the documentation should be helpful, start with the header_checks manual and the DATABASE_README file.

pipemap:{pcre:/extract/category.pcre,pcre:/map/category/to/filter/command.pcre}

Basic idea: Get your general direction first (e.g. MYFILTER: $1), and only then apply the necessary edits to make it a valid postfix command, carefully ensuring that all input leads to valid output (MYFILTER: (valid1|valid2) -> FILTER host:port plus a fallback line MYFILTER: -> FILTER: default:port)

Failing that, usually when you want to do things too powerful for regular expressions and header_checks you escalate to Turing-completeness:

A sufficiently dynamic lookup. e.g. if you like SQL, that should net you something not that exceedingly less readable/maintainable. Beware of unintended logical results from multiple lookups header_checks = sqlite:/path/to/definition.conf pcre:/remaining/simpler/rules.pcre
A milter/policy daemon. They might even call external binaries without redirecting the message at all. Depending on the interface of your current idea of the next FILTER in line, this may be more or less convenient.

I also believe there are ways to get postfix to correctly transport a given message through multiple instances of header_checks if needed, but I would not recommend that route, it does not mix to well with other (more or less dynamic) routing decisions.

+ 1

reactive-core

5/31/24, 8:31 PM

I like the initial approach, seems like this would work. The header would never be > 2 lines in my case. Will give that a shot.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Postfix header_checks with multi-line headers and capture group

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.