nginx proxy_pass for given user agent

I have a website with nginx that returns index.html (a single page react app) for almost all requests. Any request for almost any path (e.g. /some/arcticle) will also return index.html, and the react app (react router) will translate the path into API calls. This is how it works, and it cannot be changed (the website is huge, it would be too much work, and I'm in no position to change this anyway).

There should be two exceptions for these requests:

1. All requests with starting with /api/* path are passed to an upstream (application server). So a different backend will handle all actual API requests.
2. The other exception should be facebook external hits. There is a different endpoint under /api/open_graph on the application server for that. E.g. /api/open_graph should be preprended to the original path. That endpoint returns actual content (and not a common single page react app that has no real content). The format is also different - normal API calls usually return JSON data, but the open_graph endpoint returns simple HTML.

Example nginx config:

upstream www {
    # ...
}

server {
    # ...

    # Use /api/open_graph on the upstream for facebook external hits
    if ($http_user_agent ~* "^facebookexternalhit.*$") {
        rewrite ^/(.*)$ /api/open_graph/$1 permanent;
    }

    # API requests will go to upstream
    location ~ ^/api/ {
        proxy_pass          http://www;
        proxy_read_timeout  90;
        proxy_redirect      http://www https://example.com;
    }

    # All other requests use a react app,
    # react router handles all further requests
    location / {
        index index.html;
        error_page 404 =200 /index.html;
    }

}

The above config works like this:

# curl -A "facebookexternalhit" -s -D - -o /dev/null https://example.com/article1
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 20 Oct 2021 08:14:13 GMT
Content-Type: text/html
Content-Length: 162
Location: https://example.com/api/open_graph/article1
Connection: keep-alive
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN

This is almost good, except that facebook graph api does not handle redirects. (Who knows why?). So instead of redirecting to /api/open_graph/* the server should directly connect the upstream and forward the request.

But I don't know how? The naive solution would be:

    # Use /api/open_graph on the upstream for facebook external hits
    if ($http_user_agent ~* "^facebookexternalhit.*$") {
        rewrite ^/(.*)$ /api/open_graph/$1 permanent;
        proxy_pass          http://www;
        proxy_read_timeout  90;
        proxy_redirect      http://www https://example.com;
    }

But it does not work, because proxy_pass can only be used inside a location. It cannot be used inside an if. If I try that config then I get:

nginx: [emerg] "proxy_pass" directive is not allowed here in /etc/nginx/sites-enabled/example-www:62
nginx: configuration file /etc/nginx/nginx.conf test failed

I might be possible to change the program code in the upstream (e.g. add a filter for the user agent there), but it would be quite difficult.

Is there a workaround for this within nginx?

This question is related to: nginx - proxy_pass on user_agent but that wasn't working for me either (threw an error)