Join Log in
Score:0
Miksterdam avatar Server

NGINX crashed and weird logs on access.log

ru flag
Miksterdam

When I came back home just now, I was checking my webserver if it was still running and I noticed that my NGINX server was not running anymore.

I tried to restart my NGINX a few times by running the command $ NGINX and these error's occured:

2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:01:54 [emerg] 2394#2394: still could not bind()
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: bind() to 0.0.0.0:80 failed (98: Unknown error)
2023/02/11 22:02:19 [emerg] 2454#2454: still could not bind()
2023/02/11 22:03:09 [emerg] 2484#2484: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:03:09 [emerg] 2484#2484: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:03:09 [emerg] 2484#2484: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:03:09 [emerg] 2484#2484: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:03:09 [emerg] 2484#2484: bind() to 0.0.0.0:443 failed (98: Unknown error)
2023/02/11 22:03:09 [emerg] 2484#2484: still could not bind()

Then I looked at my error.log file and saw that there were some errors around a certain time:

2023/02/11 20:21:56 [error] 1877#1877: *29 open() "/usr/share/nginx/html/.env" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /.env HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *30 open() "/usr/share/nginx/html/.env.prod" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /.env.prod HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *31 open() "/usr/share/nginx/html/.env.dev" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /.env.dev HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *32 open() "/usr/share/nginx/html/sendgrid.env" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /sendgrid.env HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *33 open() "/usr/share/nginx/html/.aws/credentials" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /.aws/credentials HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *34 open() "/usr/share/nginx/html/config.py" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /config.py HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *35 open() "/usr/share/nginx/html/docker-compose.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /docker-compose.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *36 open() "/usr/share/nginx/html/docker/docker-compose.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /docker/docker-compose.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *37 open() "/usr/share/nginx/html/config/settings.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /config/settings.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:56 [error] 1877#1877: *38 open() "/usr/share/nginx/html/app/config/settings.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /app/config/settings.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *39 open() "/usr/share/nginx/html/config/parameters.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /config/parameters.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *40 open() "/usr/share/nginx/html/app/config/parameters.yml" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /app/config/parameters.yml HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *41 open() "/usr/share/nginx/html/config.php-dist" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /config.php-dist HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *42 open() "/usr/share/nginx/html/application.ini" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /application.ini HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *43 open() "/usr/share/nginx/html/application/application.ini" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /application/application.ini HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *44 open() "/usr/share/nginx/html/configs/application.ini" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /configs/application.ini HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *45 open() "/usr/share/nginx/html/config/application.ini" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /config/application.ini HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:57 [error] 1877#1877: *46 open() "/usr/share/nginx/html/application/configs/application.ini" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /application/configs/application.ini HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *66 open() "/usr/share/nginx/html/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *67 open() "/usr/share/nginx/html/demo/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /demo/.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *68 open() "/usr/share/nginx/html/dev/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /dev/.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *69 open() "/usr/share/nginx/html/web/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /web/.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *70 open() "/usr/share/nginx/html/api/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /api/.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *71 open() "/usr/share/nginx/html/admin/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /admin/.git/config HTTP/1.1", host: "45.132.242.78"
2023/02/11 20:21:58 [error] 1877#1877: *72 open() "/usr/share/nginx/html/app/.git/config" failed (2: No such file or directory), client: 54.91.160.194, server: , request: "GET /app/.git/config HTTP/1.1", host: "45.132.242.78"

What was weird, because the server is not serving anything real yet, so the only one who would make request would be me and I was not home at that time. So I took a look to my access.log file and saw some strange Python requests:

54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /.env HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /.env.prod HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /.env.dev HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /sendgrid.env HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /.aws/credentials HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /config.py HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /docker-compose.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /docker/docker-compose.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /config/settings.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:56 +0000] "GET /app/config/settings.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config/parameters.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /app/config/parameters.yml HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config.php-dist HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /application.ini HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /application/application.ini HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /configs/application.ini HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config/application.ini HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /application/configs/application.ini HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /.env HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /.env.prod HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /.env.dev HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /sendgrid.env HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /.aws/credentials HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config.py HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /docker-compose.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /docker/docker-compose.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config/settings.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /app/config/settings.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config/parameters.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /app/config/parameters.yml HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /config.php-dist HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /application.ini HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:57 +0000] "GET /application/application.ini HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /configs/application.ini HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /config/application.ini HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /application/configs/application.ini HTTP/1.1" 400 264 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /demo/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /dev/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /web/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /api/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /admin/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"
54.91.160.194 - - [11/Feb/2023:20:21:58 +0000] "GET /app/.git/config HTTP/1.1" 404 134 "-" "python-requests/2.28.2"

Is this someone who was trying to break in? Or is this a bot or something else?

This is my server set-up: Is it setup well?

server {
    if ($host = www.backstrapp.nl) {
        return 301 https://www.backstr.app$request_uri;
    }

    if ($host = backstrapp.nl) {
        return 301 https://www.backstr.app$request_uri;
    }

     if ($host = backstr.app) {
         return 301 https://www.backstr.app$request_uri;
     }

     if ($host = www.backstr.app) {
         return 301 https://www.backstr.app$request_uri;
     }

     if ($host = api.backstr.app) {
         return 301 https://api.backstr.app$request_uri;
     }
}

server {
    server_name www.backstr.app;
    location / {
        root /var/www/backstrapp/client;
        index  index.html index.htm;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        try_files $uri $uri/ =404;
    }
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/backstr.app/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/backstr.app/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    server_name api.backstr.app;
    location / {
        proxy_pass http://localhost:8800;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/backstr.app/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/backstr.app/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
  listen        80 default_server;
  server_name   _;
  return        444; # "Connection closed without response"
}

Let me know! =)

468
2 + 0
Score:1
J_H avatar Server
in flag
J_H

is this a bot?

Yes.

All public IPv4 addresses on the internet get scanned by script kiddies all the time. Stay patched! apt upgrade is your friend.

tried to restart my NGINX a few times

bind() to 0.0.0.0:80 failed (98: Unknown error)

That is EADDRINUSE. You have another daemon process bound to port 80. Likely it is a leftover nginx that hasn't quite died yet. Kill it, or kill -9, or reboot if need be, and then your newly configured nginx will be happy to acquire the port.

Track it down with $ sudo lsof -i:80

Longer term, consider configuring a packet filter to reduce the noise level:

# apt install fail2ban
+ 1
Miksterdam avatar
ru flag
Miksterdam
Okay, clear. Thanks for the answer I will have a look to the fail2ban module
0
Reply
Score:0
avatar Server
us flag
Tero Kilkanen

J_H's answer is correct about the bot aspect. However, an additional comment:

This is my server set-up: Is it setup well?

It is best to not use if whenever possible. Therefore your redirects should be implemented like this:

server {
    server_name www.backstrapp.nl backstrapp.nl ...;

    listen 80;
    listen 443 ssl http2;

    ssl_certificate /path/to/certificate;
    ssl_certificate_key /path/to/key;

    return 301 https://www.backstr.app$request_uri;
}
+ 2
Miksterdam avatar
ru flag
Miksterdam
Ok thanks! I think Certbot implements this automatically. I will give it a go. Thanks for the tip.
0
Reply
us flag
Tero Kilkanen
Ah yes, certbot unfortunately has this odd structure.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.