Score:0

What could be causing twitterbot to take 6s to download small image assets?

cn flag

Many (most) times twitter makes http requests for certain images, the response time (as recorded by nginx in the logs) takes 6s. Yet other sites (e.g. facebook) and sometimes twitter, typically get the response in 0.2s or less.

It's not that the response time is "just slow", it's that the response time is pretty much always close to 6s. Like that would be the modal response time.

Twitter say that it could be a routing issue and I should try hosting images on another domain (presumably they mean at another IP), but I'm unconvinced.

There's something about the timing that makes me think there's something else going on.

I've checked robots.txt (not that that should affect response time); I haven't implemented any rate limiting in nginx; also it's not that twitterbot is really hammering the site, there's usually a few minutes between requests!

The only strange thing I've noticed is that twitter does sometimes make two simultaneous requests for the same resource. e.g.

{"rsTime":"2022-01-05T23:07:49+00:00","rsStatus":"200","rqIp":"x.x.x.x","rq":"GET /sites/default/files/share-images/node-3211-202112151443.jpg HTTP/1.1","rqUser":"","rqRef":"","rqUA":"Twitterbot/1.0","rsB":"196275","rsTook":"6.604","phpTook":""}

{"rsTime":"2022-01-05T23:07:49+00:00","rsStatus":"200","rqIp":"x.x.x.x","rq":"GET /sites/default/files/share-images/node-3211-202112151443.jpg HTTP/1.1","rqUser":"","rqRef":"","rqUA":"Twitterbot/1.0","rsB":"196275","rsTook":"6.605","phpTook":""}

In case it's helpful, here's the headers from a request:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=2592000
Connection: keep-alive
Content-Length: 343376
Content-Type: image/jpeg
Date: Thu, 06 Jan 2022 17:45:21 GMT
ETag: "619d2951-53d50"
Expires: Sat, 05 Feb 2022 17:45:21 GMT
Keep-Alive: timeout=10
Last-Modified: Tue, 23 Nov 2021 17:48:01 GMT
Server: nginx

And I believe this is the relevant part of the nginx config:

sendfile on;             
tcp_nopush on;           
tcp_nodelay on;          
types_hash_max_size 2048;
server_tokens off;
client_body_timeout 60;                                
client_header_timeout 60;                   
keepalive_timeout 10 10;                               
send_timeout 60;                                       
reset_timedout_connection on;                          
expires 30d;
open_file_cache max=3000 inactive=120s;                           
open_file_cache_valid 45s;
open_file_cache_min_uses 2;
open_file_cache_errors off;

EDIT: tcpdump data, as suggested in comment

I used tcpdump to dump https data between mysite and twitter's site while I posted a link on twitter. Twitter made 2 http requests: one for the page (0.18s) and one for the image (6.67s).

I'm entirely new to tcpdump, so am not sure what I'm looking for.

07.301756 IP theirsite.36652 > mysite.https: Flags [SEW], seq 350223635, win 65535, options [mss 1460,sackOK,TS val 3744650926 ecr 0,nop,wscale 8], length 0
07.301814 IP mysite.https > theirsite.36652: Flags [S.E], seq 3699217149, ack 350223636, win 65160, options [mss 1460,sackOK,TS val 3624184444 ecr 3744650926,nop,wscale 7], length 0
07.400999 IP theirsite.36652 > mysite.https: Flags [.], ack 1, win 256, options [nop,nop,TS val 3744651026 ecr 3624184444], length 0
07.401067 IP theirsite.36652 > mysite.https: Flags [P.], seq 1:153, ack 1, win 256, options [nop,nop,TS val 3744651026 ecr 3624184444], length 152
07.401084 IP mysite.https > theirsite.36652: Flags [.E], ack 153, win 508, options [nop,nop,TS val 3624184543 ecr 3744651026], length 0
07.401638 IP mysite.https > theirsite.36652: Flags [.E], seq 1:2897, ack 153, win 508, options [nop,nop,TS val 3624184543 ecr 3744651026], length 2896
07.401676 IP mysite.https > theirsite.36652: Flags [P.E], seq 2897:4097, ack 153, win 508, options [nop,nop,TS val 3624184543 ecr 3744651026], length 1200
07.403373 IP mysite.https > theirsite.36652: Flags [P.E], seq 4097:4433, ack 153, win 508, options [nop,nop,TS val 3624184545 ecr 3744651026], length 336
07.500676 IP theirsite.36652 > mysite.https: Flags [.E], ack 4097, win 288, options [nop,nop,TS val 3744651125 ecr 3624184543], length 0
07.502402 IP theirsite.36652 > mysite.https: Flags [.E], ack 4433, win 300, options [nop,nop,TS val 3744651127 ecr 3624184545], length 0
07.503493 IP theirsite.36652 > mysite.https: Flags [P.EW], seq 153:246, ack 4433, win 300, options [nop,nop,TS val 3744651128 ecr 3624184545], length 93
07.503519 IP mysite.https > theirsite.36652: Flags [.E], ack 246, win 508, options [nop,nop,TS val 3624184645 ecr 3744651128], length 0
07.503926 IP mysite.https > theirsite.36652: Flags [P.EW], seq 4433:4691, ack 246, win 508, options [nop,nop,TS val 3624184646 ecr 3744651128], length 258
07.603028 IP theirsite.36652 > mysite.https: Flags [.E], ack 4691, win 311, options [nop,nop,TS val 3744651228 ecr 3624184646], length 0
07.603159 IP theirsite.36652 > mysite.https: Flags [P.EW], seq 246:595, ack 4691, win 311, options [nop,nop,TS val 3744651228 ecr 3624184646], length 349
07.603188 IP mysite.https > theirsite.36652: Flags [.E], ack 595, win 506, options [nop,nop,TS val 3624184745 ecr 3744651228], length 0
07.792960 IP mysite.https > theirsite.36652: Flags [.EW], seq 4691:6139, ack 595, win 506, options [nop,nop,TS val 3624184935 ecr 3744651228], length 1448
07.892024 IP theirsite.36652 > mysite.https: Flags [.E], ack 6139, win 322, options [nop,nop,TS val 3744651517 ecr 3624184935], length 0
07.892047 IP mysite.https > theirsite.36652: Flags [.EW], seq 6139:7587, ack 595, win 506, options [nop,nop,TS val 3624185034 ecr 3744651517], length 1448
07.990993 IP theirsite.36652 > mysite.https: Flags [.E], ack 7587, win 334, options [nop,nop,TS val 3744651616 ecr 3624185034], length 0
07.991014 IP mysite.https > theirsite.36652: Flags [.EW], seq 7587:9035, ack 595, win 506, options [nop,nop,TS val 3624185133 ecr 3744651616], length 1448
08.090023 IP theirsite.36652 > mysite.https: Flags [.E], ack 9035, win 345, options [nop,nop,TS val 3744651715 ecr 3624185133], length 0
08.090053 IP mysite.https > theirsite.36652: Flags [.EW], seq 9035:10483, ack 595, win 506, options [nop,nop,TS val 3624185232 ecr 3744651715], length 1448
08.189127 IP theirsite.36652 > mysite.https: Flags [.E], ack 10483, win 356, options [nop,nop,TS val 3744651814 ecr 3624185232], length 0
08.189150 IP mysite.https > theirsite.36652: Flags [.EW], seq 10483:11931, ack 595, win 506, options [nop,nop,TS val 3624185331 ecr 3744651814], length 1448
08.288177 IP theirsite.36652 > mysite.https: Flags [.E], ack 11931, win 368, options [nop,nop,TS val 3744651913 ecr 3624185331], length 0
08.288219 IP mysite.https > theirsite.36652: Flags [.EW], seq 11931:13379, ack 595, win 506, options [nop,nop,TS val 3624185430 ecr 3744651913], length 1448
08.387287 IP theirsite.36652 > mysite.https: Flags [.E], ack 13379, win 379, options [nop,nop,TS val 3744652012 ecr 3624185430], length 0
08.387316 IP mysite.https > theirsite.36652: Flags [.EW], seq 13379:14827, ack 595, win 506, options [nop,nop,TS val 3624185529 ecr 3744652012], length 1448
08.486350 IP theirsite.36652 > mysite.https: Flags [.E], ack 14827, win 390, options [nop,nop,TS val 3744652111 ecr 3624185529], length 0
08.486372 IP mysite.https > theirsite.36652: Flags [P.EW], seq 14827:16256, ack 595, win 506, options [nop,nop,TS val 3624185628 ecr 3744652111], length 1429
08.585430 IP theirsite.36652 > mysite.https: Flags [.E], ack 16256, win 402, options [nop,nop,TS val 3744652210 ecr 3624185628], length 0
11.555117 IP theirsite.37999 > mysite.https: Flags [SEW], seq 1059140554, win 65535, options [mss 1460,sackOK,TS val 2940663245 ecr 0,nop,wscale 8], length 0
11.555147 IP mysite.https > theirsite.37999: Flags [S.E], seq 4137859394, ack 1059140555, win 65160, options [mss 1460,sackOK,TS val 3624188697 ecr 2940663245,nop,wscale 7], length 0
11.654266 IP theirsite.37999 > mysite.https: Flags [.], ack 1, win 256, options [nop,nop,TS val 2940663345 ecr 3624188697], length 0
11.654320 IP theirsite.37999 > mysite.https: Flags [P.], seq 1:153, ack 1, win 256, options [nop,nop,TS val 2940663345 ecr 3624188697], length 152
11.654336 IP mysite.https > theirsite.37999: Flags [.E], ack 153, win 508, options [nop,nop,TS val 3624188796 ecr 2940663345], length 0
11.654658 IP mysite.https > theirsite.37999: Flags [.E], seq 1:2897, ack 153, win 508, options [nop,nop,TS val 3624188796 ecr 2940663345], length 2896
11.654676 IP mysite.https > theirsite.37999: Flags [P.E], seq 2897:4097, ack 153, win 508, options [nop,nop,TS val 3624188796 ecr 2940663345], length 1200
11.656243 IP mysite.https > theirsite.37999: Flags [P.E], seq 4097:4433, ack 153, win 508, options [nop,nop,TS val 3624188798 ecr 2940663345], length 336
11.753727 IP theirsite.37999 > mysite.https: Flags [.E], ack 4097, win 288, options [nop,nop,TS val 2940663444 ecr 3624188796], length 0
11.755266 IP theirsite.37999 > mysite.https: Flags [.E], ack 4433, win 300, options [nop,nop,TS val 2940663446 ecr 3624188798], length 0
11.756355 IP theirsite.37999 > mysite.https: Flags [P.EW], seq 153:246, ack 4433, win 300, options [nop,nop,TS val 2940663447 ecr 3624188798], length 93
11.756399 IP mysite.https > theirsite.37999: Flags [.E], ack 246, win 508, options [nop,nop,TS val 3624188898 ecr 2940663447], length 0
11.756842 IP mysite.https > theirsite.37999: Flags [P.EW], seq 4433:4691, ack 246, win 508, options [nop,nop,TS val 3624188899 ecr 2940663447], length 258
11.855874 IP theirsite.37999 > mysite.https: Flags [.E], ack 4691, win 311, options [nop,nop,TS val 2940663546 ecr 3624188899], length 0
11.856034 IP theirsite.37999 > mysite.https: Flags [P.EW], seq 246:627, ack 4691, win 311, options [nop,nop,TS val 2940663546 ecr 3624188899], length 381
11.856057 IP mysite.https > theirsite.37999: Flags [.E], ack 627, win 506, options [nop,nop,TS val 3624188998 ecr 2940663546], length 0
11.856303 IP mysite.https > theirsite.37999: Flags [.EW], seq 4691:6139, ack 627, win 506, options [nop,nop,TS val 3624188998 ecr 2940663546], length 1448
11.955431 IP theirsite.37999 > mysite.https: Flags [.E], ack 6139, win 322, options [nop,nop,TS val 2940663646 ecr 3624188998], length 0
11.955505 IP mysite.https > theirsite.37999: Flags [.EW], seq 6139:7587, ack 627, win 506, options [nop,nop,TS val 3624189097 ecr 2940663646], length 1448
12.054716 IP theirsite.37999 > mysite.https: Flags [.E], ack 7587, win 334, options [nop,nop,TS val 2940663745 ecr 3624189097], length 0
12.054805 IP mysite.https > theirsite.37999: Flags [.EW], seq 7587:9035, ack 627, win 506, options [nop,nop,TS val 3624189197 ecr 2940663745], length 1448
12.153874 IP theirsite.37999 > mysite.https: Flags [.E], ack 9035, win 345, options [nop,nop,TS val 2940663844 ecr 3624189197], length 0
12.153924 IP mysite.https > theirsite.37999: Flags [.EW], seq 9035:10483, ack 627, win 506, options [nop,nop,TS val 3624189296 ecr 2940663844], length 1448
12.253082 IP theirsite.37999 > mysite.https: Flags [.E], ack 10483, win 356, options [nop,nop,TS val 2940663943 ecr 3624189296], length 0
12.253134 IP mysite.https > theirsite.37999: Flags [.EW], seq 10483:11931, ack 627, win 506, options [nop,nop,TS val 3624189395 ecr 2940663943], length 1448
12.352340 IP theirsite.37999 > mysite.https: Flags [.E], ack 11931, win 368, options [nop,nop,TS val 2940664043 ecr 3624189395], length 0
12.352409 IP mysite.https > theirsite.37999: Flags [.EW], seq 11931:13379, ack 627, win 506, options [nop,nop,TS val 3624189494 ecr 2940664043], length 1448
12.451567 IP theirsite.37999 > mysite.https: Flags [.E], ack 13379, win 379, options [nop,nop,TS val 2940664142 ecr 3624189494], length 0
12.451615 IP mysite.https > theirsite.37999: Flags [.EW], seq 13379:14827, ack 627, win 506, options [nop,nop,TS val 3624189593 ecr 2940664142], length 1448
12.550697 IP theirsite.37999 > mysite.https: Flags [.E], ack 14827, win 390, options [nop,nop,TS val 2940664241 ecr 3624189593], length 0
12.550736 IP mysite.https > theirsite.37999: Flags [.EW], seq 14827:16275, ack 627, win 506, options [nop,nop,TS val 3624189693 ecr 2940664241], length 1448
12.649774 IP theirsite.37999 > mysite.https: Flags [.E], ack 16275, win 402, options [nop,nop,TS val 2940664340 ecr 3624189693], length 0
12.649810 IP mysite.https > theirsite.37999: Flags [.EW], seq 16275:17723, ack 627, win 506, options [nop,nop,TS val 3624189792 ecr 2940664340], length 1448
12.748972 IP theirsite.37999 > mysite.https: Flags [.E], ack 17723, win 413, options [nop,nop,TS val 2940664439 ecr 3624189792], length 0
12.749025 IP mysite.https > theirsite.37999: Flags [.EW], seq 17723:19171, ack 627, win 506, options [nop,nop,TS val 3624189891 ecr 2940664439], length 1448
12.848133 IP theirsite.37999 > mysite.https: Flags [.E], ack 19171, win 424, options [nop,nop,TS val 2940664538 ecr 3624189891], length 0
12.848171 IP mysite.https > theirsite.37999: Flags [.EW], seq 19171:20619, ack 627, win 506, options [nop,nop,TS val 3624189990 ecr 2940664538], length 1448
12.947232 IP theirsite.37999 > mysite.https: Flags [.E], ack 20619, win 436, options [nop,nop,TS val 2940664637 ecr 3624189990], length 0
12.947273 IP mysite.https > theirsite.37999: Flags [.EW], seq 20619:22067, ack 627, win 506, options [nop,nop,TS val 3624190089 ecr 2940664637], length 1448
13.046357 IP theirsite.37999 > mysite.https: Flags [.E], ack 22067, win 447, options [nop,nop,TS val 2940664737 ecr 3624190089], length 0
13.046386 IP mysite.https > theirsite.37999: Flags [.EW], seq 22067:23515, ack 627, win 506, options [nop,nop,TS val 3624190188 ecr 2940664737], length 1448
13.145398 IP theirsite.37999 > mysite.https: Flags [.E], ack 23515, win 458, options [nop,nop,TS val 2940664836 ecr 3624190188], length 0
13.145426 IP mysite.https > theirsite.37999: Flags [.EW], seq 23515:24963, ack 627, win 506, options [nop,nop,TS val 3624190287 ecr 2940664836], length 1448
13.244504 IP theirsite.37999 > mysite.https: Flags [.E], ack 24963, win 469, options [nop,nop,TS val 2940664935 ecr 3624190287], length 0
13.244532 IP mysite.https > theirsite.37999: Flags [.EW], seq 24963:26411, ack 627, win 506, options [nop,nop,TS val 3624190386 ecr 2940664935], length 1448
13.343650 IP theirsite.37999 > mysite.https: Flags [.E], ack 26411, win 481, options [nop,nop,TS val 2940665034 ecr 3624190386], length 0
13.343716 IP mysite.https > theirsite.37999: Flags [.EW], seq 26411:27859, ack 627, win 506, options [nop,nop,TS val 3624190486 ecr 2940665034], length 1448
13.442769 IP theirsite.37999 > mysite.https: Flags [.E], ack 27859, win 492, options [nop,nop,TS val 2940665133 ecr 3624190486], length 0
13.442795 IP mysite.https > theirsite.37999: Flags [.EW], seq 27859:29307, ack 627, win 506, options [nop,nop,TS val 3624190585 ecr 2940665133], length 1448
13.541819 IP theirsite.37999 > mysite.https: Flags [.E], ack 29307, win 503, options [nop,nop,TS val 2940665232 ecr 3624190585], length 0
13.541880 IP mysite.https > theirsite.37999: Flags [.EW], seq 29307:30755, ack 627, win 506, options [nop,nop,TS val 3624190684 ecr 2940665232], length 1448
13.640911 IP theirsite.37999 > mysite.https: Flags [.E], ack 30755, win 515, options [nop,nop,TS val 2940665331 ecr 3624190684], length 0
13.640976 IP mysite.https > theirsite.37999: Flags [.EW], seq 30755:32203, ack 627, win 506, options [nop,nop,TS val 3624190783 ecr 2940665331], length 1448
13.740069 IP theirsite.37999 > mysite.https: Flags [.E], ack 32203, win 526, options [nop,nop,TS val 2940665430 ecr 3624190783], length 0
13.740153 IP mysite.https > theirsite.37999: Flags [.EW], seq 32203:33651, ack 627, win 506, options [nop,nop,TS val 3624190882 ecr 2940665430], length 1448
13.839252 IP theirsite.37999 > mysite.https: Flags [.E], ack 33651, win 537, options [nop,nop,TS val 2940665530 ecr 3624190882], length 0
13.839316 IP mysite.https > theirsite.37999: Flags [.EW], seq 33651:35099, ack 627, win 506, options [nop,nop,TS val 3624190981 ecr 2940665530], length 1448
13.938391 IP theirsite.37999 > mysite.https: Flags [.E], ack 35099, win 549, options [nop,nop,TS val 2940665629 ecr 3624190981], length 0
13.938444 IP mysite.https > theirsite.37999: Flags [.EW], seq 35099:36547, ack 627, win 506, options [nop,nop,TS val 3624191080 ecr 2940665629], length 1448
14.037553 IP theirsite.37999 > mysite.https: Flags [.E], ack 36547, win 560, options [nop,nop,TS val 2940665728 ecr 3624191080], length 0
14.037618 IP mysite.https > theirsite.37999: Flags [.EW], seq 36547:37995, ack 627, win 506, options [nop,nop,TS val 3624191179 ecr 2940665728], length 1448
14.137132 IP theirsite.37999 > mysite.https: Flags [.E], ack 37995, win 571, options [nop,nop,TS val 2940665827 ecr 3624191179], length 0
14.137200 IP mysite.https > theirsite.37999: Flags [.EW], seq 37995:39443, ack 627, win 506, options [nop,nop,TS val 3624191279 ecr 2940665827], length 1448
14.236282 IP theirsite.37999 > mysite.https: Flags [.E], ack 39443, win 583, options [nop,nop,TS val 2940665927 ecr 3624191279], length 0
14.236354 IP mysite.https > theirsite.37999: Flags [.EW], seq 39443:40891, ack 627, win 506, options [nop,nop,TS val 3624191378 ecr 2940665927], length 1448
14.335398 IP theirsite.37999 > mysite.https: Flags [.E], ack 40891, win 594, options [nop,nop,TS val 2940666026 ecr 3624191378], length 0
14.335459 IP mysite.https > theirsite.37999: Flags [.EW], seq 40891:42339, ack 627, win 506, options [nop,nop,TS val 3624191477 ecr 2940666026], length 1448
14.434551 IP theirsite.37999 > mysite.https: Flags [.E], ack 42339, win 605, options [nop,nop,TS val 2940666125 ecr 3624191477], length 0
14.434616 IP mysite.https > theirsite.37999: Flags [.EW], seq 42339:43787, ack 627, win 506, options [nop,nop,TS val 3624191576 ecr 2940666125], length 1448
14.533722 IP theirsite.37999 > mysite.https: Flags [.E], ack 43787, win 617, options [nop,nop,TS val 2940666224 ecr 3624191576], length 0
14.533789 IP mysite.https > theirsite.37999: Flags [.EW], seq 43787:45235, ack 627, win 506, options [nop,nop,TS val 3624191676 ecr 2940666224], length 1448
14.632926 IP theirsite.37999 > mysite.https: Flags [.E], ack 45235, win 628, options [nop,nop,TS val 2940666323 ecr 3624191676], length 0
14.632993 IP mysite.https > theirsite.37999: Flags [.EW], seq 45235:46683, ack 627, win 506, options [nop,nop,TS val 3624191775 ecr 2940666323], length 1448
14.732801 IP theirsite.37999 > mysite.https: Flags [.E], ack 46683, win 639, options [nop,nop,TS val 2940666422 ecr 3624191775], length 0
14.732857 IP mysite.https > theirsite.37999: Flags [.EW], seq 46683:48131, ack 627, win 506, options [nop,nop,TS val 3624191875 ecr 2940666422], length 1448
14.831963 IP theirsite.37999 > mysite.https: Flags [.E], ack 48131, win 650, options [nop,nop,TS val 2940666522 ecr 3624191875], length 0
14.832026 IP mysite.https > theirsite.37999: Flags [.EW], seq 48131:49579, ack 627, win 506, options [nop,nop,TS val 3624191974 ecr 2940666522], length 1448
14.931211 IP theirsite.37999 > mysite.https: Flags [.E], ack 49579, win 662, options [nop,nop,TS val 2940666621 ecr 3624191974], length 0
14.931297 IP mysite.https > theirsite.37999: Flags [.EW], seq 49579:51027, ack 627, win 506, options [nop,nop,TS val 3624192073 ecr 2940666621], length 1448
15.030409 IP theirsite.37999 > mysite.https: Flags [.E], ack 51027, win 673, options [nop,nop,TS val 2940666721 ecr 3624192073], length 0
15.030478 IP mysite.https > theirsite.37999: Flags [.EW], seq 51027:52475, ack 627, win 506, options [nop,nop,TS val 3624192172 ecr 2940666721], length 1448
15.129575 IP theirsite.37999 > mysite.https: Flags [.E], ack 52475, win 684, options [nop,nop,TS val 2940666820 ecr 3624192172], length 0
15.129632 IP mysite.https > theirsite.37999: Flags [.EW], seq 52475:53923, ack 627, win 506, options [nop,nop,TS val 3624192271 ecr 2940666820], length 1448
15.228699 IP theirsite.37999 > mysite.https: Flags [.E], ack 53923, win 696, options [nop,nop,TS val 2940666919 ecr 3624192271], length 0
15.228766 IP mysite.https > theirsite.37999: Flags [.EW], seq 53923:55371, ack 627, win 506, options [nop,nop,TS val 3624192371 ecr 2940666919], length 1448
15.327877 IP theirsite.37999 > mysite.https: Flags [.E], ack 55371, win 707, options [nop,nop,TS val 2940667018 ecr 3624192371], length 0
15.327942 IP mysite.https > theirsite.37999: Flags [.EW], seq 55371:56819, ack 627, win 506, options [nop,nop,TS val 3624192470 ecr 2940667018], length 1448
15.427060 IP theirsite.37999 > mysite.https: Flags [.E], ack 56819, win 718, options [nop,nop,TS val 2940667117 ecr 3624192470], length 0
15.427128 IP mysite.https > theirsite.37999: Flags [.EW], seq 56819:58267, ack 627, win 506, options [nop,nop,TS val 3624192569 ecr 2940667117], length 1448
15.526297 IP theirsite.37999 > mysite.https: Flags [.E], ack 58267, win 730, options [nop,nop,TS val 2940667217 ecr 3624192569], length 0
15.526368 IP mysite.https > theirsite.37999: Flags [.EW], seq 58267:59715, ack 627, win 506, options [nop,nop,TS val 3624192668 ecr 2940667217], length 1448
15.625481 IP theirsite.37999 > mysite.https: Flags [.E], ack 59715, win 741, options [nop,nop,TS val 2940667316 ecr 3624192668], length 0
15.625552 IP mysite.https > theirsite.37999: Flags [.EW], seq 59715:61163, ack 627, win 506, options [nop,nop,TS val 3624192767 ecr 2940667316], length 1448
15.724607 IP theirsite.37999 > mysite.https: Flags [.E], ack 61163, win 752, options [nop,nop,TS val 2940667415 ecr 3624192767], length 0
15.724650 IP mysite.https > theirsite.37999: Flags [.EW], seq 61163:62611, ack 627, win 506, options [nop,nop,TS val 3624192866 ecr 2940667415], length 1448
15.823665 IP theirsite.37999 > mysite.https: Flags [.E], ack 62611, win 764, options [nop,nop,TS val 2940667514 ecr 3624192866], length 0
15.823717 IP mysite.https > theirsite.37999: Flags [.EW], seq 62611:64059, ack 627, win 506, options [nop,nop,TS val 3624192966 ecr 2940667514], length 1448
15.922871 IP theirsite.37999 > mysite.https: Flags [.E], ack 64059, win 775, options [nop,nop,TS val 2940667613 ecr 3624192966], length 0
15.922946 IP mysite.https > theirsite.37999: Flags [.EW], seq 64059:65507, ack 627, win 506, options [nop,nop,TS val 3624193065 ecr 2940667613], length 1448
16.022042 IP theirsite.37999 > mysite.https: Flags [.E], ack 65507, win 786, options [nop,nop,TS val 2940667712 ecr 3624193065], length 0
16.022113 IP mysite.https > theirsite.37999: Flags [.EW], seq 65507:66955, ack 627, win 506, options [nop,nop,TS val 3624193164 ecr 2940667712], length 1448
16.121191 IP theirsite.37999 > mysite.https: Flags [.E], ack 66955, win 798, options [nop,nop,TS val 2940667811 ecr 3624193164], length 0
16.121223 IP mysite.https > theirsite.37999: Flags [.EW], seq 66955:68403, ack 627, win 506, options [nop,nop,TS val 3624193263 ecr 2940667811], length 1448
16.220324 IP theirsite.37999 > mysite.https: Flags [.E], ack 68403, win 809, options [nop,nop,TS val 2940667911 ecr 3624193263], length 0
16.220371 IP mysite.https > theirsite.37999: Flags [.EW], seq 68403:69851, ack 627, win 506, options [nop,nop,TS val 3624193362 ecr 2940667911], length 1448
jp flag
log requests with `tcpdump`, check individual packets timestamps
cn flag
OK, I believe I've got tcpdump data for a sample request by twitter for the page and its 'share image', but I don't know how to interpret that.
Score:1
jp flag

Flags [E] and [W] in the tcpdump output are Explicit Congestion Notifications.

When you see theirsite.36652 > mysite.https: Flags [.E] it means that the receiver detected link congestion and asks the sender (your site) to slow down and the sender responds with Flags [.EW] indicating that it acknowledged the request and reduced TCP window.

You can disable ECN like so: sysctl -w net.ipv4.tcp_ecn=0

cn flag
So... Twitter is saying there's congestion on the route from mysite → theirs, asking mysite to slow down. So twitter is *asking* for the slow down. If that is the problem, presumably it's out of my hands to solve (except by moving hosts, which is not something I'm keen to do).
cn flag
Thanks, disabling ECN fixed the problem!
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.