Join Log in
Score:0
Ajay Singh avatar Server

Troubleshooting nginx requests not reaching server

us flag
Ajay Singh

When I browse from other servers using

wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" --spider http://server_ip/page --no-hsts

This is the response

Spider mode enabled. Check if remote file exists.
--2021-07-25 09:03:09--  http://server_ip/page
Connecting to 45.76.11.115:80... connected.
 HTTP request sent, awaiting response... 200 No headers, assuming HTTP/0.9
Length: unspecified
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.

Nginx default conf is

server {
   listen 80 default_server;
   server_name _;
   return 404;
   location ~ /\.ht {
      deny all; 
   }
}

There are no entries in both error or access log. However it's receiving requests from some unknown servers like

some_public_ip - - [25/Jul/2021:08:53:14 +0800] "GET / HTTP/1.1" 200 399 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36" "1.68"

By checking programs listening at 80 using

lsof -nP -iTCP -sTCP:LISTEN

I get only nginx though.

nginx     19182     root    6u  IPv4 19898878      0t0  TCP *:80 (LISTEN)
nginx     19182     root    7u  IPv4 19898879      0t0  TCP *:8443 (LISTEN)
nginx     19183 www-data    6u  IPv4 19898878      0t0  TCP *:80 (LISTEN)
nginx     19183 www-data    7u  IPv4 19898879      0t0  TCP *:8443 (LISTEN)

However tcpdump showing it's receiving request

09:13:06.520109 IP local_ip.42330 > server_ip.80: Flags [.], ack 1, win 21, options [nop,nop,TS val 3610668715 ecr 3536888037], length 0
09:13:06.523627 IP local_ip.42330 > server_ip.80: Flags [P.], seq 1:173, ack 1, win 21, options [nop,nop,TS val 3610668715 ecr 3536888037], length 172: HTTP: HEAD / HTTP/1.1
09:13:06.523668 IP server_ip.80 > local_ip.42330: Flags [.], ack 173, win 508, options [nop,nop,TS val 3536888310 ecr 3610668715], length 0
09:13:06.523741 IP server_ip.80 > local_ip.42330: Flags [P.], seq 1:58, ack 173, win 508, options [nop,nop,TS val 3536888310 ecr 3610668715], length 57: HTTP
09:13:06.523781 IP server_ip.80 > local_ip.42330: Flags [F.], seq 58, ack 173, win 508, options [nop,nop,TS val 3536888310 ecr 3610668715], length 0
09:13:06.790067 IP local_ip.42330 > server_ip.80: Flags [.], ack 58, win 21, options [nop,nop,TS val 3610668990 ecr 3536888310], length 0
09:13:06.795185 IP local_ip.42330 > server_ip.80: Flags [R.], seq 173, ack 59, win 21, options [nop,nop,TS val 3610668991 ecr 3536888310], length 0
09:13:06.800086 IP local_ip.42330 > server_ip.80: Flags [R], seq 1666043585, win 0, length 0

How to fix this?

Update #1

Tried tcp dump again with

tcpdump -vvXX -n port 80

and output

    local_ip.48914 > server_ip.80: Flags [S], cksum 0xf7e9 (correct), seq 2727175320, win 42340, options [mss 1318,sackOK,TS val 3613618415 ecr 0,nop,wscale 11], length 0
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  003c 5f00 0000 3806 875c 6adc f8c4 2d4c  .<_...8..\j...-L
        0x0020:  0b73 bf12 0050 a28d 6498 0000 0000 a002  .s...P..d.......
        0x0030:  a564 f7e9 0000 0204 0526 0402 080a d763  .d.......&.....c
        0x0040:  70ef 0000 0000 0103 030b                 p.........
    10:02:16.252990 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
        server_ip.80 > local_ip.48914: Flags [S.], cksum 0x9c8e (incorrect -> 0xbb3e), seq 3157700385, ack 2727175321, win 65160, options [mss 1460,sackOK,TS val 3539838101 ecr 3613618415,nop,wscale 7], length 0
        0x0000:  fe00 0302 ccd0 5600 0302 ccd0 0800 4500  ......V.......E.
        0x0010:  003c 0000 4000 4006 9e5c 2d4c 0b73 6adc  .<..@.@..\-L.sj.
        0x0020:  f8c4 0050 bf12 bc36 af21 a28d 6499 a012  ...P...6.!..d...
        0x0030:  fe88 9c8e 0000 0204 05b4 0402 080a d2fd  ................
        0x0040:  a495 d763 70ef 0103 0307                 ...cp.....
    10:02:16.615737 IP (tos 0x0, ttl 56, id 24419, offset 0, flags [none], proto TCP (6), length 52)
        local_ip.48914 > server_ip.80: Flags [.], cksum 0xe6f3 (correct), seq 1, ack 1, win 21, options [nop,nop,TS val 3613618810 ecr 3539838101], length 0
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  0034 5f63 0000 3806 8701 6adc f8c4 2d4c  .4_c..8...j...-L
        0x0020:  0b73 bf12 0050 a28d 6499 bc36 af22 8010  .s...P..d..6."..
        0x0030:  0015 e6f3 0000 0101 080a d763 727a d2fd  ...........crz..
        0x0040:  a495                                     ..
    10:02:16.629335 IP (tos 0x0, ttl 56, id 24423, offset 0, flags [none], proto TCP (6), length 227)
        local_ip.48914 > server_ip.80: Flags [P.], cksum 0x36e5 (correct), seq 1:176, ack 1, win 21, options [nop,nop,TS val 3613618810 ecr 3539838101], length 175: HTTP, length: 175
        HEAD /url HTTP/1.1
        User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
        Accept: */*
        Accept-Encoding: identity
        Host: server_ip
        Connection: Keep-Alive
        
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  00e3 5f67 0000 3806 864e 6adc f8c4 2d4c  .._g..8..Nj...-L
        0x0020:  0b73 bf12 0050 a28d 6499 bc36 af22 8018  .s...P..d..6."..
        0x0030:  0015 36e5 0000 0101 080a d763 727a d2fd  ..6........crz..
        0x0040:  a495 4845 4144 202f 7572 6c20 4854 5450  ..HEAD./url.HTTP
        0x0050:  2f31 2e31 0d0a 5573 6572 2d41 6765 6e74  /1.1..User-Agent
        0x0060:  3a20 4d6f 7a69 6c6c 612f 342e 3020 2863  :.Mozilla/4.0.(c
        0x0070:  6f6d 7061 7469 626c 653b 204d 5349 4520  ompatible;.MSIE.
        0x0080:  362e 303b 2057 696e 646f 7773 204e 5420  6.0;.Windows.NT.
        0x0090:  352e 313b 2053 5631 290d 0a41 6363 6570  5.1;.SV1)..Accep
        0x00a0:  743a 202a 2f2a 0d0a 4163 6365 7074 2d45  t:.*/*..Accept-E
        0x00b0:  6e63 6f64 696e 673a 2069 6465 6e74 6974  ncoding:.identit
        0x00c0:  790d 0a48 6f73 743a 2034 352e 3736 2e31  y..Host:.server_ip
        0x00d0:  312e 3131 350d 0a43 6f6e 6e65 6374 696f  Connectio
        0x00e0:  6e3a 204b 6565 702d 416c 6976 650d 0a0d  n:.Keep-Alive...
        0x00f0:  0a                                       .
    10:02:16.629408 IP (tos 0x0, ttl 64, id 24117, offset 0, flags [DF], proto TCP (6), length 52)
        server_ip.80 > local_ip.48914: Flags [.], cksum 0x9c86 (incorrect -> 0xe2e4), seq 1, ack 176, win 508, options [nop,nop,TS val 3539838478 ecr 3613618810], length 0
        0x0000:  fe00 0302 ccd0 5600 0302 ccd0 0800 4500  ......V.......E.
        0x0010:  0034 5e35 4000 4006 402f 2d4c 0b73 6adc  .4^5@.@.@/-L.sj.
        0x0020:  f8c4 0050 bf12 bc36 af22 a28d 6548 8010  ...P...6."..eH..
        0x0030:  01fc 9c86 0000 0101 080a d2fd a60e d763  ...............c
        0x0040:  727a                                     rz
    10:02:16.629513 IP (tos 0x0, ttl 64, id 24118, offset 0, flags [DF], proto TCP (6), length 109)
        server_ip.80 > local_ip.48914: Flags [P.], cksum 0x9cbf (incorrect -> 0xb393), seq 1:58, ack 176, win 508, options [nop,nop,TS val 3539838478 ecr 3613618810], length 57: HTTP
        0x0000:  fe00 0302 ccd0 5600 0302 ccd0 0800 4500  ......V.......E.
        0x0010:  006d 5e36 4000 4006 3ff5 2d4c 0b73 6adc  .m^6@.@.?.-L.sj.
        0x0020:  f8c4 0050 bf12 bc36 af22 a28d 6548 8018  ...P...6."..eH..
        0x0030:  01fc 9cbf 0000 0101 080a d2fd a60e d763  ...............c
        0x0040:  727a 0000 1204 0000 0000 0000 0300 0000  rz..............
        0x0050:  8000 0400 0100 0000 0500 ffff ff00 0004  ................
        0x0060:  0800 0000 0000 7fff 0000 0000 0807 0000  ................
        0x0070:  0000 0000 0000 0000 0000 01              ...........
    10:02:16.629551 IP (tos 0x0, ttl 64, id 24119, offset 0, flags [DF], proto TCP (6), length 52)
        server_ip.80 > local_ip.48914: Flags [F.], cksum 0x9c86 (incorrect -> 0xe2aa), seq 58, ack 176, win 508, options [nop,nop,TS val 3539838478 ecr 3613618810], length 0
        0x0000:  fe00 0302 ccd0 5600 0302 ccd0 0800 4500  ......V.......E.
        0x0010:  0034 5e37 4000 4006 402d 2d4c 0b73 6adc  .4^7@.@[email protected].
        0x0020:  f8c4 0050 bf12 bc36 af5b a28d 6548 8011  ...P...6.[..eH..
        0x0030:  01fc 9c86 0000 0101 080a d2fd a60e d763  ...............c
        0x0040:  727a                                     rz
    10:02:16.913890 IP (tos 0x0, ttl 56, id 24502, offset 0, flags [none], proto TCP (6), length 52)
        local_ip.48914 > server_ip.80: Flags [.], cksum 0xe361 (correct), seq 176, ack 58, win 21, options [nop,nop,TS val 3613619115 ecr 3539838478], length 0
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  0034 5fb6 0000 3806 86ae 6adc f8c4 2d4c  .4_...8...j...-L
        0x0020:  0b73 bf12 0050 a28d 6548 bc36 af5b 8010  .s...P..eH.6.[..
        0x0030:  0015 e361 0000 0101 080a d763 73ab d2fd  ...a.......cs...
        0x0040:  a60e                                     ..
    10:02:16.913950 IP (tos 0x0, ttl 56, id 24506, offset 0, flags [none], proto TCP (6), length 52)
        local_ip.48914 > server_ip.80: Flags [.], cksum 0xe360 (correct), seq 176, ack 59, win 21, options [nop,nop,TS val 3613619115 ecr 3539838478], length 0
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  0034 5fba 0000 3806 86aa 6adc f8c4 2d4c  .4_...8...j...-L
        0x0020:  0b73 bf12 0050 a28d 6548 bc36 af5c 8010  .s...P..eH.6.\..
        0x0030:  0015 e360 0000 0101 080a d763 73ab d2fd  ...`.......cs...
        0x0040:  a60e                                     ..
    10:02:16.913983 IP (tos 0x0, ttl 56, id 24508, offset 0, flags [none], proto TCP (6), length 52)
        local_ip.48914 > server_ip.80: Flags [R.], cksum 0xe35b (correct), seq 176, ack 59, win 21, options [nop,nop,TS val 3613619116 ecr 3539838478], length 0
        0x0000:  5600 0302 ccd0 fe00 0302 ccd0 0800 4500  V.............E.
        0x0010:  0034 5fbc 0000 3806 86a8 6adc f8c4 2d4c  .4_...8...j...-L
        0x0020:  0b73 bf12 0050 a28d 6548 bc36 af5c 8014  .s...P..eH.6.\..
        0x0030:  0015 e35b 0000 0101 080a d763 73ac d2fd  ...[.......cs...
        0x0040:  a60e

Could incorrect cksum relavant to the issue? Is there a way to check which application or process responded that request?

358
1 + 0
Score:0
Ajay Singh avatar Server
us flag
Ajay Singh

Found the reason for this error, which was caused by another config file for different server_name, having

listen 80 http2;

Removing http2 from the config fixed the issue. Unfortunate that Nginx config test doesn't report this as invalid.

Following resources helped me troubleshoot;

https://stackoverflow.com/questions/16021481/nginx-not-listening-to-port-80

HTTPS works, HTTP does not

0 + 2
djdomi avatar
za flag
djdomi
because http2 on port 80 is not a invalid option
0
Reply
Ajay Singh avatar
us flag
Ajay Singh
@djdomi Yeah, you're right. It would be invalid without no try_files? Either way, they could still test and show some error, rather than responding the request without logging.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.