How to handle requests to non-existent subdomains with Nginx

Here's my Nginx configuration for my domain example.com which serves a trivial static website.

server {
    root /var/www/example.com;

    index index.html index.htm index.nginx-debian.html;

    server_name example.com www.example.com;

    location / {
        try_files $uri $uri/ =404;
    }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri;
}

How do I configure Nginx to return 404 whenever I navigate to a subdomain such as asdf.example.com? Now I find myself looking at the same page as example.com.

I've set a wildcard A record for my domain so I'm expecting all requests ending with example.com to find my server. Do I have to remove the wildcard or can I handle these requests to subdomains with Nginx?

Nginx server_name configuration allows regular expressions, or even the simpler * wildcard, see https://nginx.org/en/docs/http/ngx_http_core_module.html#server_name

So either of these two should match any name (not better matched by another block)

server_name *.example.com

server_name ~^.+\.example\.com$

(The ~ is required to use a regular expression).

You can even capture parts of the name, and use it in subsequent configuration directives. The above documentation link has examples of that.

And you do need the DNS wildcard otherwise the traffic won't reach your webserver (or if no DNS wildcard, you need specific records for the names you want to resolve, no matter how many of them, and still keep the "wildcard" configuration of Nginx)