Score:0

Nginx stops after a while without registering any trackable log

cn flag

I have a server (Ubuntu 20.04) that uses Nginx as a web server. Also, several projects are on it. The problem is, sometimes the Nginx fails (stops). Like this:

enter image description here

I really have no idea why it crashes. Searched a lot for it and made some changes in the server blocks (port listening), but still it stops sometimes. The important part is, when I run systemctl restart nginx it back to be working as well, but after a while, fails again.

The result of 10 last lines inside the nginx error log:

2023/03/17 06:54:46 [emerg] 88675#88675: bind() to 0.0.0.0:443 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to 0.0.0.0:80 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to [::]:80 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to 0.0.0.0:443 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to 0.0.0.0:80 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to [::]:80 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: bind() to 0.0.0.0:443 failed (98: Address already in use)
2023/03/17 06:54:46 [emerg] 88675#88675: still could not bind()
2023/03/17 06:54:48 [error] 88631#88631: MEMSTORE:19: my procslot not found! I don't know what to do!
nginx: worker process: /build/nginx-lUTckl/nginx-1.18.0/debian/modules/nchan/src/store/memory/memstore.c:1616: nchan_store_exit_worker: Assertion `0' failed.

And this is the /etc/nginx/sites-enabled/default content:

server {    
    server_name api5.arshiyaniha.com;
    root /var/www/html/api/public/ ;

    error_log /var/log/nginx/api-new-error.log ;
    access_log /var/log/nginx/api-new.log ;

    index index.php index.html index.htm index.nginx-debian.html ;

    if ($request_uri ~* "^(.*/)index\.php$") {
       return 301 $1;
    }

    location / {
       try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php8.1-fpm.sock;
        fastcgi_read_timeout 2000;
    }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/api5.arshiyaniha.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/api5.arshiyaniha.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}


server {
        server_name arshiyanadmin.arshiyaniha.com;
        root /var/www/html/panel-front/build/ ;

        error_log /var/log/nginx/arshiyanadmin.log ;
        access_log /var/log/nginx/arshiyanadmin.log ;

        index index.html;

        location / {
           try_files $uri $uri/ /index.html?$query_string;
        }

    client_max_body_size 10M;

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/arshiyanadmin.arshiyaniha.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/arshiyanadmin.arshiyaniha.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server { 
     server_name api.arshiyaniha.com;
     location / {
          proxy_pass http://localhost:4020;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection 'upgrade';
          proxy_set_header Host $host;
          proxy_cache_bypass $http_upgrade;
     }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/api.arshiyaniha.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/api.arshiyaniha.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
     server_name dashboardapi.arshiyaniha.com;
     location / {
          proxy_pass http://localhost:4021;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection 'upgrade';
          proxy_set_header Host $host;
          proxy_cache_bypass $http_upgrade;
          proxy_connect_timeout 60s;
          proxy_read_timeout 5400s; 
          proxy_send_timeout 5400s;
    }

    listen 443 ssl; # managed by Certbot 
    ssl_certificate /etc/letsencrypt/live/dashboardapi.arshiyaniha.com/fullchain.pem; # managed by Certbot 
    ssl_certificate_key /etc/letsencrypt/live/dashboardapi.arshiyaniha.com/privkey.pem; # managed by Certbot 
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot 
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server { 
     server_name files.arshiyaniha.com;
     location / {
          proxy_pass http://localhost:4022;
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection 'upgrade';
          proxy_set_header Host $host;
          proxy_cache_bypass $http_upgrade;
     }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/files.arshiyaniha.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/files.arshiyaniha.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = api.arshiyaniha.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
 
    server_name api.arshiyaniha.com;
    listen 80;
    return 404; # managed by Certbot
}

server {
    if ($host = files.arshiyaniha.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    server_name files.arshiyaniha.com;
    listen 80;
    return 404; # managed by Certbot

}

server {
    if ($host = dashboardapi.arshiyaniha.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    server_name dashboardapi.arshiyaniha.com;
    listen 80;
    return 404; # managed by Certbot
}

server {
    if ($host = arshiyanadmin.arshiyaniha.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot
   
    server_name arshiyanadmin.arshiyaniha.com;
    listen 80;
    return 404; # managed by Certbot
}


server {
    if ($host = api5.arshiyaniha.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    listen [::]:80;    

    server_name api5.arshiyaniha.com;
    return 404; # managed by Certbot
}

Any idea what's the exact problem and how can I fix it?

Score:1
cn flag

This sounds like a 4 year old bug. Although it has been marked as resolved. It's possible it hasn't been backported into your OS.

And I know this isn't the solution, but as a workaround, you can configure systemd to restart nginx automatically on failure.

Create new directory,

mkdir /etc/systemd/system/nginx.service.d/

Create new file

nano /etc/systemd/system/nginx.service.d/override.conf

Paste following content

[Unit]
StartLimitIntervalSec=500
StartLimitBurst=5

[Service]
Restart=on-failure
RestartSec=5s

Now restart daemon service and nginx.

Try it.

Martin AJ avatar
cn flag
Thank you, I made that directory and that file inside, and also restarted Nginx (`systemctl restart nginx`), but since I have no idea what "daemon" is, I don't know how to restart it, could you please give me the command for restarting daemon?
Tharindu Kumara avatar
cn flag
Try `systemctl daemon-reload`
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.