My webserver running Debian Bullseye has been running for years without any problems. Everything is up to date. The apache version is 2.4.54. Yesterday I noticed none of my websites were up so I logged in to my server and noticed Apache wasn't running anymore. I restarted Apache and checked its log file to see what happened. the log file showed this:
[Tue Dec 13 23:46:50.629635 2022] [mpm_prefork:emerg] [pid 2079339] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Tue Dec 13 23:46:51.397665 2022] [core:alert] [pid 2036789] AH00050: Child 2079339 returned a Fatal error... Apache is exiting!
[Tue Dec 13 23:46:51.397743 2022] [:emerg] [pid 2036789] AH02818: MPM run failed, exiting
Since apache never crashed on me before, I figured it was a one time deal and left it at that. But today I again noticed my websites were down and again Apache was no longer running. Checked error.log, and I saw this:
[Wed Dec 14 15:10:58.522933 2022] [mpm_prefork:emerg] [pid 2543516] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:58.600767 2022] [mpm_prefork:emerg] [pid 2543951] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:58.821819 2022] [mpm_prefork:emerg] [pid 2543852] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:58.985176 2022] [mpm_prefork:emerg] [pid 2543789] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:59.020098 2022] [mpm_prefork:emerg] [pid 2545055] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:59.054757 2022] [mpm_prefork:emerg] [pid 2543845] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:59.088885 2022] [mpm_prefork:emerg] [pid 2543953] (22)Invalid argument: AH00146: couldn't release the accept mutex
[Wed Dec 14 15:10:59.337919 2022] [core:alert] [pid 2080897] AH00050: Child 2543516 returned a Fatal error... Apache is exiting!
[Wed Dec 14 15:10:59.338011 2022] [:emerg] [pid 2080897] AH02818: MPM run failed, exiting
So now I'm guessing it wasn't a one time deal and something must be wrong. I searched the interwebs and although many have encountered this problem before, nowhere could I find what could be causing this. Even so, everywhere the same solution was offered. Add the following line to apache2.conf:
Mutex posixsem
I added the line so will see if that indeed fixes the problem. Still remains the question what could be causing this? The last update I did on the system before apache crashed was almost 2 weeks ago (snippet from my apt history.log):
Start-Date: 2022-12-01 01:14:02
Commandline: apt-get upgrade
Upgrade: php7.4-soap:amd64 (7.4.30-1+deb11u1, 7.4.33-1+deb11u1), php7.4-mbstring:amd64 (7.4.30-1+deb11u1, 7.4.33-1+deb11u1), krb5-locales:amd64 (1.18.3-6+deb11u2, 1.18.3-6+deb11u3), libgssapi-krb5-2:amd64 (1.18.3-6+deb11u2, 1.18.3-6+deb11u3
End-Date: 2022-12-01 01:16:00
After the first crash, I did another update:
Start-Date: 2022-12-13 23:58:36
Commandline: apt-get upgrade
Upgrade: libopenexr25:amd64 (2.5.4-2, 2.5.4-2+deb11u1), snapd:amd64 (2.49-1+deb11u1, 2.49-1+deb11u2)
End-Date: 2022-12-13 23:58:47
That (apparently) did not help. So again, what could be causing these crashes? Could it be something (attack/malformed request) from outside?
