Nginx reverse proxy returning internal address to client on page refresh

I have a reactjs app being served by a container running NGINX with following configuration:

server {
listen       ${NGINX_PORT};
access_log  /var/log/nginx/access.log  main;
location / {
    root   /usr/share/nginx/html;
    try_files $uri $uri/ /index.html?$args;        
}
}

Then I put a reverse proxy in front of it with the following configuration:

upstream internal {
    server X.X.X.X:3100;
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name mysite.com;
    port_in_redirect off;

    ssl_certificate /opt/nginx/certs/cert.cer;
    ssl_certificate_key /opt/nginx/certs/cert.key;

    location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_pass http://internal;
            proxy_http_version 1.1;
            proxy_redirect off;
    }
  }

It works fine for navigation, I can jump to one page to another without any issue. However, when I hit refresh, it redirects the browser to the internal address.

http://internal:3100/docs/intro/

EDIT: The root address, or mysite.com can be refreshed with no issues. The refresh issue occurs only when inside some topic tree like example above.

How can I prevent this ?