Join Log in
Score:0
avatar Server

How to figure out what is bad about a 400 bad request, on an Apache-server

ar flag
Zeth

The overarching question

How do I see what is 'bad' about a 400 - bad request?

Info about the error

  • When I click around the WordPress-backend, then between 3 and 7 requests (out of 95-100) give me a: 400 - Bad request-error, upon every page load. But it varies which request that fails, from refresh to refresh?!
  • I've been battling this issue for 8 hours (ish). I feel like I've tried everything.
  • It happens both in Chrome and Firefox
  • It doesn't happen on another computer with an identical setup.
  • It's mainly static files, where the requests fail (but not exclusively).
  • There are 10-15 other developers also working on it, they haven't seen it either (same branch, almost identical database, same Vagrant setup).

System info

  • WordPress-installation
  • Apache-server
  • Running locally on a Vagrant box (Ubuntu VirtualBox)

Solution attempt 1: See tendencies between which files it fails to load

I can't see any tendencies there. Here are 4 random refreshes.

Refresh1

api-client-1740.js
backbone.min.js
styleGuide-1740.js
escape-html.js
admin-script.min.js
quicktags.js
admin-bar.js
hooks.js
debug-bar-js.dev.js
font-awesome.css
nav-menus.css
list-tables.css

Refresh2

nav-menus.css
about.css
admin-menu.css

Refresh3

tags-suggest.js
i18n.js
wp-polyfill.js
themes.css
list-tables.css

Refresh4

"Image loadingAnimation.gif"
flame.png
api-client-1740.js
element.min.js
admin-script.min.js
moment.min.js
mouse.min.js
lodash.min.js
underscore.min.js
hoverintent-js.min.js
adminbar-1740.css
admin-global-1740.css

And sometimes none of the requests fails.

Solution attempt 2: Inspect the request in the network-tab

I can find a reason for the 400 Bad request in there. Here is an example of a request that returned 400 Bad request (where I've redacted sensitive info).

Request

GET /wp/wp-admin/load-styles.php?c=0&dir=ltr&load%5Bchunk_0%5D=dashicons,admin-bar,site-health,common,forms,admin-menu,dashboard,list-tables,edit,revisions,media,themes,about,nav-menus,wp-poi&load%5Bchunk_1%5D=nter,widgets,site-icon,l10n,buttons,wp-auth-check&ver=5.8.3 HTTP/1.1
Host: mydomain.test
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://mydomain.test/wp/wp-admin/
DNT: 1
Connection: keep-alive
Cookie: wordpress_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2dfda29fb0e79974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cd16311d6028a3550a8777754a99a1e2d09bffb05f45f36854038387bc0fdce43; wp-settings-2=posts_list_mode%3Dlist%26ampampampamplibraryContent%3Dbrowse%26ampampampampeditor%3Dtinymce%26ampampampampmfold%3Do%26libraryContent%3Dbrowse; wp-settings-time-2=1652706157; CookieConsent={stamp:%27bs7eydGl1MNS0O2BJZiPnLkz4NSuHl+9jWJceT1L8gnBXxa3B+MlsA==%27%2Cnecessary:true%2Cpreferences:true%2Cstatistics:true%2Cmarketing:true%2Cver:1%2Cutc:1652701927276%2Ciab2:%27CPZD8gAPZD8gACGABBENCPCsAP_AAH_AAAAAIwtd_X__bX9j-_5_bft0eY1P9_r3_-QzjhfNs-8F3L_W_L0Xw2E7NF36pq4KuR4Eu3LBIQNlHMHUTUmwaokVrzHsak2cpyNKJ7LEknMZO2dYGH9Pn9lDuYKY7_5___bx3D-v_t_-39T378Xf3_d5_2_--vCfV599jbn9fV_7_9nP___9v-_8_________gjAASYal5AF2JY4MmkaRQogRhWEhVAoAKKAYWiKwAcHBTsrAJdQQsAEAqAjAiBBiCjBgEAAgEASERASAFggEQBEAgABAAiAQgAImAQWAFgYBAAKAaFiAFAAIEhBkQERymBARIlFBLZWIJQV7GmEAdZYAUCiMioAESAAAkBASFg5jgCQEuFkgSYoXyAEYIUAAAAA.YAAAAAAAAAAA%27%2Cgacm:%271~AAAAAAAAAAACAABAAAgAIAAABAAhAAAACAAAAAAAQAAQAAAAAAABBBAAIAAAAAAAAAAAAQAAAIBAAAAAIgMAAAAAAAgAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAgAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAQAAAAAAFAAAABAAAAAAAAAAAARBAAAAAAAAAACAAABAAAAAAAAAAEAAAAAAABAAAAAEAAAAAAAAAAAAAAAAAAACBAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAQAAAAAAAAAAAAAAAACAgBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAgAAAAAAAEAAAAAAAQAAgAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAkQAAAAAAAAAAAAAAAAQ=%27%2Cregion:%27dk%27}; modal_page_counter_cookie=1; utag_main=v_id:0180ccb62e2b002302377014532c05054001000f00fbe$_fd:2$_se:1$_ss:1$_st:1652709110390$dc_visit:2$ses_id:1652707310390%3Bexp-session$_pn:1%3Bexp-session$dc_event:1%3Bexp-session; user_obj=eyJ0b2tlbiI6ImQ1YWFhYWI5NzQyZTc2M2FlZjEyYzhjNDBhNGE3NWRhODA3MmZhOTc4MDNkZDdhMWJkNmRlNzYzODRhMjM4MDgiLCJzdWJzY3JpcHRpb25fYXJlYXMiOnsiMCI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWMdfiI6IjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWM3ZiIsIjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSI6IjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSIsIjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDkIJGkdf4jE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDk5OCIsIjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSI6IjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSIsIjc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiI6Ijc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiIsIjMxNjNiY2RhLTIxYWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSI6IjMxNjNiY2RhLT5YWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSIsIjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiI6IjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiIsImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiI6ImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiIsImUxY2ExNzc2LTU0NTUtNDc4MC04OTMzLTgxM2M0NjlmZmZhMSI6ImUxY2ExNzc2LTU0NTUtNDc4MC071TMzLTgxM2M0NjlmZmZhMSIsImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCI6ImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCIsImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSI6ImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSJ9LCJlbWFpbCI6InpvZCtzdWJzY3JpcHRpb250eXBlMkBwZXl0ei5kayIsInN1YnNjcmlwdGlvbl90eXBlIjoiMiJ9; user_token=d5aaaab9742e763aef12c8c40a4a75da8072fa97803dd7a1bd6de76384a23808; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f
Pragma: no-cache
Cache-Control: no-cache

Response headers

HTTP/1.1 400 Bad Request
Date: Mon, 16 May 2022 13:27:53 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Length: 299
Connection: close
Content-Type: text/html; charset=iso-8859-1

Response

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
</body></html>

I can't see out of that, why that request was 'bad'. Nor why the server couldn't understand it.

Solution attempt 3: Check the logs

I checked these:

/var/log/apache2/access.log
/var/log/apache2/error.log
/var/log/apache2/other_vhosts_access.log

I even tried only outputting lines containing the word 400, with this command:

tail -f access.log error.log other_vhosts_access.log | awk '/400/'

And I could get them (from the other_vhost_access.log-file). But I can't see any reason for it to be 'bad' either:

In Firefox

vagrant:80 123.123.123.1 - - [16/May/2022:15:27:53 +0200] "GET /wp/wp-admin/js/common.min.js HTTP/1.1" 400 481 "http://mydomain.test/wp/wp-admin/, http://mydomain.test/wp/wp-admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0, Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"

In Chrome

vagrant:80 123.123.123.1 - - [16/May/2022:15:45:24 +0200] "POST /es/query/ HTTP/1.1" 400 481 "http://mydomain.test/, http://mydomain.test/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36, Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36"

Solution attempt 4: Change LogLevel on the Apache-server

As advised here I tried Changing the LogLevel on the Apache-server.

But I didn't see any better logs in access.log, error.log nor other_vhost_access.log.

Solution attempt 5: Debug using tcpdump

I also tried running tcpdump from inside the box and output it all using the command:

13:01:42.102438 IP (tos 0x0, ttl 64, id 54239, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [.], cksum 0xc3db (incorrect -> 0xe5ed), seq 114948251, ack 2535382125, win 505, options [nop,nop,TS val 243067183 ecr 1641029207], length 0
13:01:42.102475 IP (tos 0x0, ttl 64, id 8630, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57334: Flags [.], cksum 0xc3db (incorrect -> 0xbf8a), seq 2046057008, ack 256413570, win 505, options [nop,nop,TS val 243067183 ecr 3257787508], length 0
13:01:42.106114 IP (tos 0x2,ECT(0), ttl 64, id 54240, offset 0, flags [DF], proto TCP (6), length 533)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [P.], cksum 0xc5bc (incorrect -> 0xe068), seq 114948251:114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067186 ecr 1641029207], length 481: HTTP, length: 481
    HTTP/1.1 400 Bad Request
    Date: Mon, 16 May 2022 11:01:42 GMT
    Server: Apache/2.4.41 (Ubuntu)
    Content-Length: 299
    Connection: close
    Content-Type: text/html; charset=iso-8859-1

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>400 Bad Request</title>
    </head><body>
    <h1>Bad Request</h1>
    <p>Your browser sent a request that this server could not understand.<br />
    </p>
    <hr>
    <address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
    </body></html>
13:01:42.106394 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.4.57340 > 123.123.123.12.80: Flags [.], cksum 0xddfb (correct), seq 2535382125, ack 114948732, win 2051, options [nop,nop,TS val 1641029211 ecr 243067186], length 0
13:01:42.109623 IP (tos 0x0, ttl 64, id 54241, offset 0, flags [DF], proto TCP (6), length 52)
    123.123.123.12.80 > 123.123.123.4.57340: Flags [F.], cksum 0xc3db (incorrect -> 0xe400), seq 114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067190 ecr 1641029211], length 0

And again, I can see the 400 Bad request-error. But I can't see what's bad about it.

Solution attempt 6: Look into headers / cookies

Since the errors only occurred when I was logged in, then I tried looking at the header and cookies. But I couldn't see anything spectacular there.

I even compared them to the headers/cookies on another machine, where this is running flawlessly, and nothing sprung in my eye.

... And still... Even if it was this, then why don't I get a better error message?

Solution attempt 7: Server performance - activity monitor

I checked the disk-space, CPU-usage and memory usage on the server (via bpytop) when I refreshed a page. But it looks normal.

Untried attempts

  • Downgrade and reinstall and/or upgrade WordPress core.
  • Stop and start apache
  • Restart the Ubuntu-server.

I've not done this, since I would really like to find out, how to see what is wrong here (instead of just trying to fix it).

42
0 + 0
400
us flag
Rob
Possibly the Apache virtualHost entry specifies (error) log files in a different location, or you configured something like: https://serverfault.com/q/997624/960939 because AFAIK otherwise 400 errors should be logged. When you're using wordpress it's very likely that you have Rewrite rules active, maybe those don't function correctly - https://httpd.apache.org/docs/2.4/mod/mod_rewrite.html#logging
0
Reply
djdomi avatar
za flag
djdomi
is the website for private purposes?
0
Reply
Tilman Schmidt avatar
bd flag
Tilman Schmidt
Look at the full unfiltered error.log at the time the error happens. Don't filter for "400" - there's no reason the message telling you what's going wrong has to contain that string.
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Don't split your efforts into "attempts". You should consider all factors at the same time: access log, error log, network (the actual "bytes" of the request and response), client presentation. You have to isolate the entries of all of these that correspond to the same single wrong request. Also, you may configure Apache to log more verbose. Also, you've seen the "bad request" itself — replicate that request with complete headers using curl or even "telnet webserver 80" or openssl s_client (in case of HTTPS), see for yourself if it's really bad request or transient problem on web server side.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.