HTTPS POST-Request fails in connection with Content-Length

SlowFox

5/19/23, 9:45 PM

A PHP application posts XML data with curl; nothing fancy, the output looking like (c/p, but identifiers and passwords changed):

Host: foreign.host.example
Authorization: Basic dGVzdDpnZWhlaW0=
User-Agent: ourhost HTTP-connector/1.0
Accept: */*
Referer: https://our.host.example/
Content-Type: text/xml
Content-Length: 9218
Expect: 100-continue

[9218 bytes of XML data]

This is done on a couple of source hosts S1...Sn and posted to a couple of target hosts T1...Tn. A week ago host S1 was upgraded from Debian/buster to Debian/bullseye; the generating PHP software was upgraded as well, but I am not aware of any changes in this area (and the generated requests look identical). Exactly since this upgrade two (unrelated) target hosts T1 and T2 refuse our requests with response code 400. All other target hosts work fine, and T1/T2 work fine, if connected from other source hosts (with old revision level).

Corollary: The fault is on our side. But where?

Side note: dropping the Expect header does not change anything.

Supplemental information: While T1 responds only with response code 400, T2 is polite and includes a status message like:

[our.ip.address]:60760 has incomplete request: Estimated length was 3994 byte(s), received 3899 byte(s)

Unfortunately this hint is not enough to enlighten me (and contacts there have not enough knowledge to do that either), but it led me to the following

Workaround: If I suppress the Content-Length header, transmission works

This is better than nothing, but not satisfying - I want the problem to be solved; so I double- and triple-checked the number in the Content-Length header (as this is the only possible source of the error I can think of so far); it seems ok, and exactly the same calculation is used on hosts S2...Sn which are still working. And: the content length plus the estimated header (this will of course differ a little bit due to other host, authorization and referer) is neither the estimated length nor the received length, but somewhere in the middle. Responses so far have provided the following numbers:

Estimated	Received	Difference
4322	4228	94
3972	3877	95
3994	3899	95
6928	6768	160

Unfortunately, this is neither constant nor does it ring any bell.

I set a couple of tags because Apache (2.4.52), OpenSSL (1.1.1k), curl (7.74.0) and PHP (7.4.25) have been upgraded together with the server even though I cannot see any possible influence. But I don't have any other ideas, so I don't want to rule this out.

Any ideas are welcome.

0 + 0

https

openssl

curl

apache-2.4

php7

AlexD

5/20/23, 7:42 AM

Use `tcpdump` or a proxy to capture the outgoing request and confirm that you have the correct `Content-Length` on the wire.

SlowFox

5/29/23, 1:57 PM

This was driving me nuts: With mitmproxy I can intercept calls to one of our own sites - everything fine. If I try to contact T1 or T2, the request __hangs__ while mitmproxy does the handshake to the remote host; I don't see any headers at all. As soon as I drop Content-Length, everything works as expected.

SlowFox

5/29/23, 1:59 PM

But I had a look into the repository now: the Content-Length header has been set since the origin of the software in 2006. Most likely something has changed in the background (cURL) long ago, as if I simply omit it, cURL sets the correct header anyway. So even though I still do not understand what is going on here, I'll stop further investigations at this point.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: HTTPS POST-Request fails in connection with Content-Length

TH: คำขอ HTTPS POST ล้มเหลวในการเชื่อมต่อกับความยาวเนื้อหา

RO: Solicitarea HTTPS POST eșuează în legătură cu Lungimea conținutului

RU: Ошибка POST-запроса HTTPS в связи с Content-Length

VI: HTTPS POST-Request không kết nối được với Content-Length

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.