Block ChatGPT bot at an nginx level

I want to block access for ChatGPT for a whole website, but still allow it to see the robots.txt file located in the public root directory (my robots.txt basically also tells ChatGPT not to crawl the site).

I am blocking it as follows:

# Disallow chatGPT bot
location / {
    if ($http_user_agent ~* "gptbot") {
        return 401;
    }
}

But I want it to be able to access the robots.txt file.

I tried doing:

if ($http_user_agent ~* "GPTBot") {
    if ($request_uri != /robots.txt) {
        return 403;  # Forbidden
    }
}

But it fails.

I also tried:

location / {
    if ($http_user_agent ~* "gptbot") {
        return 401;
    }
}

location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;
}

And also a variation of this using map:

map $http_user_agent $block_gptbot {
    default         0;
    ~*gptbot        1;
}

server {

location / {
    if ($block_gptbot) {
        return 401;
    }
}

...

But in both cases, the whole site is blocked. Means, gptbot cannot acces anything (it gets the 401), not even robots.txt.