Score:0

Ubuntu 23.04 Upgrade Leads to Apache2 Timing Out and Not Serving Pages on Two Servers

us flag

I upgraded Ubuntu Server from 22.10 to 23.04 that hosts some websites over Apache2 as well as a few .NET websites which was working normally but following the upgrade, requests for all the Apache2 websites just timeout where a WordPress one shows a 504 gateway timeout error and the .NET ones do load but then after a little while also then stop responding. Restarting Apache2 has no effect on the Apache2 websites but does make the Net ones work again briefly.

I set up a new blank 23.04 Ubuntu instance (downloading the iso from the Ubuntu website), installed Apache2, MariaDB and .NET 7 and moved everything across but after configuring Apache it is behaving exactly the same as the old server so it must be related to the way it is configured although this has not changed for over a year with only updates being applied.

The old server is hosted on DigitalOcean as a VPS whereas the new server is hosted on Hyper-V on a Windows Server over a TalkTalk business connection that is already successfully hosting a Postfix mailserver.

I would appreciate any ideas on this. I have tried steps such as disabling http2 connections back to http1.1 and also disabling a rewrite rule I added to direct http traffic to https but nothing I try makes any difference and the old server had been running fine for over a year.

This is a snippet from the error.log:

[Fri May 12 06:58:23.885767 2023] [proxy_fcgi:error] [pid 99667:tid 140182967776960] (70007)The timeout specified has expired: [client 198.204.231.122:50058] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:01:03.251102 2023] [proxy_fcgi:error] [pid 99668:tid 140183328556736] (70007)The timeout specified has expired: [remote 10.10.1.1:52874] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:01:08.525998 2023] [proxy_fcgi:error] [pid 99667:tid 140183126714048] (70007)The timeout specified has expired: [client 198.204.231.122:50156] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:09:02.501388 2023] [proxy_fcgi:error] [pid 99668:tid 140182581909184] (70007)The timeout specified has expired: [client 198.204.231.122:50354] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:11:45.345774 2023] [proxy_fcgi:error] [pid 99874:tid 140182288328384] (70007)The timeout specified has expired: [client 198.204.231.122:50451] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:17:07.597795 2023] [proxy_fcgi:error] [pid 99668:tid 140182892308160] (70007)The timeout specified has expired: [client 198.204.231.122:50702] AH01075: Error dispatching request to : (polling)
[Fri May 12 07:17:09.431865 2023] [proxy_fcgi:error] [pid 99667:tid 140182883849920] (70007)The timeout specified has expired: [client 198.204.231.122:50801] AH01075: Error dispatching request to : (polling)
[Fri May 12 08:05:05.369726 2023] [proxy_fcgi:error] [pid 99667:tid 140182909028032] (70007)The timeout specified has expired: [client 52.167.144.160:47424] AH01075: Error dispatching request to : (polling)
[Fri May 12 08:05:09.001276 2023] [proxy_fcgi:error] [pid 99874:tid 140182288328384] (70007)The timeout specified has expired: [client 66.249.73.196:62107] AH01075: Error dispatching request to : (polling)
[Fri May 12 08:50:36.883724 2023] [proxy_fcgi:error] [pid 99667:tid 140182959384256] (70007)The timeout specified has expired: [client 66.249.76.190:49970] AH01075: Error dispatching request to : (polling)
[Fri May 12 09:11:55.907282 2023] [proxy_fcgi:error] [pid 99874:tid 140182254757568] (70007)The timeout specified has expired: [client 66.249.76.65:57755] AH01075: Error dispatching request to : (polling)
[Fri May 12 09:14:36.037940 2023] [proxy_fcgi:error] [pid 100060:tid 140182632265408] (70007)The timeout specified has expired: [client 194.38.21.214:60500] AH01075: Error dispatching request to : (polling)

This is a snippet from the access log:

194.38.21.214 - - [12/May/2023:08:26:35 +0000] "GET /admin/elfinder/php/connector.minimal.php HTTP/1.1" 302 609 "-" "ALittle Client"
157.55.39.224 - - [12/May/2023:07:38:37 +0000] "GET /courthouse/86/9826/1110397 HTTP/2.0" 200 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36"
157.55.39.224 - - [12/May/2023:07:38:37 +0000] "GET /courthouse/86/9826/1110397 HTTP/2.0" 500 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36"
52.167.144.162 - - [12/May/2023:07:43:49 +0000] "GET /frayproof=39571&bRVjuhw=853552 HTTP/2.0" 200 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36"
52.167.144.162 - - [12/May/2023:07:43:49 +0000] "GET /frayproof=39571&bRVjuhw=853552 HTTP/2.0" 500 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36"
114.119.139.205 - - [12/May/2023:08:39:55 +0000] "GET /ebay/QL320?C=N%3BO%3DD HTTP/1.1" 302 585 "https://www.rwsbetas.com/ebay/QL320?C=M%3BO%3DA" "Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)"
66.249.76.190 - - [12/May/2023:08:02:28 +0000] "GET /s/W2204874 HTTP/1.1" 504 4912 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
114.119.152.221 - - [12/May/2023:08:56:01 +0000] "GET /WiiKey/ HTTP/1.1" 302 555 "https://www.rwsbetas.com" "Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)"
66.249.76.65 - - [12/May/2023:08:23:51 +0000] "GET /s/A400738 HTTP/1.1" 504 4916 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.5615.142 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
54.153.87.12 - - [12/May/2023:09:14:29 +0000] "GET /geeta HTTP/1.1" 301 4905 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
194.38.21.214 - - [12/May/2023:08:29:08 +0000] "GET /admin/elfinder/php/connector.minimal.php HTTP/1.1" 504 4929 "-" "ALittle Client"
10.10.1.1 - - [12/May/2023:09:25:50 +0000] "GET / HTTP/1.1" 302 541 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.0.0"

So it seems as if the request is just timing out when being handled by the PHP fast CGI module

Error dispatching request to : (polling)

The following steps were made in configuring apache2:

Installed packages:

apache2, php, php-curl, php-mbstring, php-imagick, php-intl

Enabled Apache2 modules:

suexec, authz_groupfile, headers, ssl, rewrite, proxy_http

Enabled PHP modules:

mbstring

PHP.ini changes:

upload_max_filesize = 32M
post_max_size = 32M
date.timezone = Europe/London

Lastly I switched over to HTTP2

sudo a2enmod http2
sudo apt install php-fpm
sudo a2dismod php8.1
sudo a2enconf php8.1-fpm
sudo a2enmod proxy_fcgi setenvif
sudo a2dismod php8.1

sudo a2dismod mpm_prefork
sudo a2enmod mpm_event

So looking

above

it would seem to be the proxy_fcgi module causing the issue. I will try reverting back to the older one.

Thanks Robin

in flag
The Apache logs should have *something* that's related to the issue you're facing. Could you [edit] your question to include some of the log file? This will make it easier to determine what the cause might be. Otherwise, it's just blind guessing, which does nobody any good
Robin Wilson avatar
us flag
Thanks sorry you're right. I have edited above. I will share edits I made to the apache2 setup too above.
Robin Wilson avatar
us flag
Reverting back to http1.1 prefork module has resolved the problem and the sites are back on so the question is how to get http2 support back since upgrading to 23.04. Thanks to matigo to get me to look at the logs again!
Robin Wilson avatar
us flag
Does anyone have any idea why reverting back to the http 1.1 prefork module is required in order to get apache working in Ubuntu 23.04. As I can replicate this on two servers with one being a fresh install this is definitely the case and I don't think my config is anything out of the ordinary. Would it be better to post a new question now I know the cause?
in flag
Creating a new question would make more sense. However, it may be that you’ve stumbled onto a bug. In that case, [reporting it on LaunchPad](https://help.ubuntu.com/community/ReportingBugs#Filing_bugs_manually_at_Launchpad.net) would have the development team look into the issue
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.