Join Log in
Score:3
avatar Ubuntu

Ubuntu 20.04.3: nginx crashes with segfault

tk flag
Bernd Lentes

I have an Ubuntu box (20.04.3) on which nginx stopped already twice with a segfault:

Nov 16 00:01:23 documents-oo kernel: [2726316.050697] nginx[2579242]: segfault at 41 ip 00007fbf59548593 sp 00007ffc02313af0 error 4 in libperl.so.5.30.0[7fbf594e0000+166000]
Nov 16 00:01:23 documents-oo kernel: [2726316.050742] Code: 48 89 43 10 48 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 40 00 0f b6 7f 30 48 c1 e8 03 48 29 f8 48 89 c3 74 89 48 8b 02 <4c> 8b 68 10 4d 85 ed 0f 84 28 01 00 00 0f b6 40 30 49 c1 ed 03 49

root@documents-oo:~# systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: failed (Result: core-dump) since Tue 2021-11-16 00:03:17 CET; 8h ago
       Docs: man:nginx(8)
    Process: 2994244 ExecReload=/usr/sbin/nginx -g daemon on; master_process on; -s reload (code=exited, status=0/SUCCESS)
   Main PID: 2579242 (code=dumped, signal=SEGV)
      Tasks: 0 (limit: 19097)
     Memory: 19.7M
     CGroup: /system.slice/nginx.service

Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809495 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809496 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809497 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809498 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809499 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809500 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809501 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809502 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Killing process 2809503 (nginx) with signal SIGKILL.
Nov 16 00:03:17 documents-oo systemd[1]: nginx.service: Failed with result 'core-dump'.

Oct 31 00:02:06 documents-oo kernel: [1340433.341063] nginx[433202]: segfault at 10 ip 00007f6a15e08593 sp 00007ffeef04e220 error 4 in libperl.so.5.30.0[7f6a15da0000+166000]
Oct 31 00:02:06 documents-oo kernel: [1340433.355995] Code: 48 89 43 10 48 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 40 00 0f b6 7f 30 48 c1 e8 03 48 29 f8 48 89 c3 74 89 48 8b 02 <4c> 8b 68 10 4d 85 ed 0f 84 28 01 00 00 0f b6 40 30 49 c1 ed 03 49

root@documents-oo:~# systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: failed (Result: core-dump) since Sun 2021-10-31 00:04:57 CEST; 45min ago
       Docs: man:nginx(8)
    Process: 1503969 ExecReload=/usr/sbin/nginx -g daemon on; master_process on; -s reload (code=exited, status=0/SUCCESS)
   Main PID: 433202 (code=dumped, signal=SEGV)
      Tasks: 0 (limit: 19097)
     Memory: 6.1M
     CGroup: /system.slice/nginx.service

Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573437 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573438 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573439 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573440 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573441 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573442 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573443 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573444 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Killing process 573445 (nginx) with signal SIGKILL.
Okt 31 00:04:57 documents-oo systemd[1]: nginx.service: Failed with result 'core-dump'.

Both happened shortly after logrotate. I have a crashdump I can upload if desired. Btw: is there a good webpage which explains how to read a crashdump? I tried to file that as a bug to launchpad, but got this message:

root@documents-oo:/var/crash# ubuntu-bug /var/crash/_usr_sbin_nginx.0.crash

*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (1.6 MB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C): s

*** Problem in nginx-extras

The problem cannot be reported:

You have some obsolete package versions installed. Please upgrade the following packages and check if the problem still occurs:

libssl1.1, openssl

Press any key to continue...

But libssl1.1 and openssl are the most recent ones!?!

Thanks. Bernd

138
0 + 0
ru flag
Thomas Ward
When was the last time you did a `sudo apt update` and are you running without -updates and -security repos enabled?
0
Reply
lk flag
Dynalon
I have the same issues for quite a while in the same version of ubuntu. nginx just core-dumps, the symbol address are quite similar: `general protection fault ip:7f0f3e9dd593 sp:7ffe695de500 error:0 in libperl.so.5.30.0[7f0f3e975000+166000]`. I run an up-to-date system, all updates installed.
0
Reply
Score:0
avatar Ubuntu
lk flag
Dynalon

Check this answer on servfault which seems to address the problem: https://serverfault.com/a/1041052/105887

0 + 0
David avatar
cn flag
David
That may work today but what about later when that link is gone? Therefore link only answers are not answers. If there is something in that link that answers the question include it in this answer. You can still have the link but the info will be there link or no link.
0
Reply
lk flag
Dynalon
Ever heard of the web-archive?
0
Reply
cd flag
Timothy C. Quinn
Perl is now at 5.32.1 and nginx is 1.32.2 but its still segfaulting. ARGH! If it was not for zfs root, I'd be done with ubuntu desktop in a flash.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.