Score:0

HAProxy setting variables for logging

au flag

Goal

I am trying to output the full payload of the request as part of an error message, because I believe the requester is giving me a garbage payload. The original, working log was

  setenv TCP_LOG "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"
  log-format "${TCP_LOG} %[var(txn.sendercompid)] %[var(txn.errormessage)] %[var(txn.sendercompid)] %[var(txn.mapped_sendercompid)]"
  tcp-request content set-var(txn.errormessage) str("ERROR: SenderCompID not in request") unless { var(txn.sendercompid) -m found }

This is exactly the standard tcp log format from HAProxy plus some strings we care about.

I want to change the value of txn.errormessage to include the full request payload when it fails.


What I've tried so far:

tcp-request content set-var(txn.errormessage) str("ERROR: SenderCompID not in request %[req.payload(0,0),regsub(\001,|,g)]")

which fails with

> [ALERT] (1) : config : parsing [/bitnami/haproxy/conf/haproxy.cfg:44] : fetch method 'str' : expected ')' before ',0),regsub(\001,|,g)])'

and

tcp-request content set-var(txn.errormessage) req.payload(0,0)

which I believe logs the empty string in

$IP:43384 [11/Jul/2023:20:05:00.748] fix_listener fix_listener/ -1/-1/0 0 PR 1/1/0/0/0 0/0 - - -

Though I can see that I am being given a payload with

tcp-request content set-var(txn.errormessage) req.len

giving me

$IP:43536 [11/Jul/2023:20:25:26.805] fix_listener fix_listener/ -1/-1/0 0 PR 1/1/0/0/0 0/0 - 206 - -
$IP:43540 [11/Jul/2023:20:26:15.825] fix_listener fix_listener/ -1/-1/0 0 PR 1/1/0/0/0 0/0 - 206 - -
$IP:43544 [11/Jul/2023:20:27:04.879] fix_listener fix_listener/ -1/-1/0 0 PR 1/1/0/0/0 0/0 - 206 - -

The closest I could get to working was

tcp-request content set-var(txn.errormessage) req.payload(0,0),hex unless { var(txn.sendercompid) -m found }

Which outputs hex, and if I pipe it into xxd -r -p I get a bunch of garbage with an IP address at the end.

I have read a decent amount of the docs, but the docs for tcp-request content set-var (after redirecting to http-request set-var) only describe the expression I can use as

<expr> Is a standard HAProxy expression formed by a sample-fetch followed by some converters.

and there's no obvious section describing a standard HAProxy expression. I read through the quoting and escaping section and also didn't understand why I couldn't have a comma inside str().

Questions

  1. How do I quote / format str("ERROR: SenderCompID not in request %[req.payload(0,0),regsub(\001,|,g)]") correctly to be able to concatenate a string with a variable?
  2. Why doesn't setting txn.errormessage to req.payload(0,0) print anything in my logs?
  3. What is the easiest way to print out the full payload of a request in a log statement?
Score:0
au flag

Answers

  1. See below
  2. I have no idea
  3. I have no idea

After bashing my head into the wall enough times, I got something that did what I wanted:

tcp-request content set-var(txn.errormessage) \
  req.payload(0,0),regsub(\x01,|,g),regsub(^,"ERROR: invalid FIX: <"),concat(">") \
  unless { req.payload(0,0),fix_is_valid }

Some things I learned along the way that might help others:

  • , = | in bash, but , is ALSO used for delimiting arguments
  • You need to wrap hardcoded strings in str() (unless they're already arguments in a converter, in which case you don't even need quotes)
  • single quotes escape double quotes and start their own strongly-quoted string, I don't care what the docs say
  • You can't nest converters and you should not try.
  • concat() is worse than the docs lead you to believe and cannot make this line simpler
  • HAProxy has .if / .endif top level statements (to try to avoid the trailing unless) but they only accept 6 predicates and none were useful to me.
  • The docs say that nonprinting characters are automatically cast into #X where X is the ASCII hex of your character, but that was not my experience and \x01 was printed as that nonprinting character or the empty string (I was reading from stdout so if it would have been the actual SOH character, the terminal prints that as the empty string - regardless, it was not printed as #01)
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.