How do I set up an nginx reverse proxy for Synology drive?

tl;dr

I tried setting up nginx to forward a TCP connection over 6690 to Synology Drive. But when using listen 6690 ssl; the Synology Drive Client cannot find the NAS, and when using listen 6690; (without SSL) the NAS is found, but its unsigned certificate is used instead of the certificate stored with nginx. Any idea how to enable SSL and use an nginx reverse proxy with Synology Drive?

Intro

I already found several posts/questions explaining how to set up a SSL-connection to a Synology NAS using nginx as a reverse proxy on a separate computer. Thereby, when accessing (https://)my-domain.com I can (already) access my Synology Drive's settings website (i.e. the DSM reachable via port 5000/5001). However, I also want to set up Synology Drive to use the same SSL certificate from Let's Encrypt, but here I'm having difficulties setting up nginx.

Question

So, I only want Synology Drive configured and I set up the DNS of drive.my-domain.com to reach my router. Also, I acquired an SSL certificate for this domain and have adjusted the Synology Drive HTTPS port to be 10003. In my router, I forwarded ports 80, 443 and 6690 to my reverse proxy as recommended here. For nginx, I configured /etc/nginx/conf.d/synology.conf:

include /etc/letsencrypt/options-ssl-nginx.conf;

server {
    listen      80;
    server_name drive.my-domain.com default_server;

    # For Let's Encrypt verification
    location ^~ /.well-known {
        allow all;
        root /data/letsencrypt/;
    }
}

server {
    listen 80;
    listen 443 ssl;
    server_name drive.my-domain.com;

    location / {
        proxy_set_header    Host $host;
        proxy_set_header    X-Real_IP $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto $scheme;
        proxy_pass          https://<IP-OF-SYNOLOGY-NAS>:10003;
        proxy_read_timeout  90;
    }

    ssl_certificate         /etc/letsencrypt/live/drive.my-domain.com/fullchain.pem;
    ssl_certificate_key     /etc/letsencrypt/live/drive.my-domain.com/privkey.pem;
    
    include /etc/letsencrypt/options-ssl-nginx.conf;
    if ($scheme != "https") {
        return 301 https://$host$request_uri;
    }
}

Further, I added the following line to /etc/nginx/nginx.conf:

...

include /etc/nginx/fallthrough.conf;

And /etc/nginx/fallthrough.conf contains the following:

stream {

    upstream synology_drive {
         server <IP-OF-SYNOLOGY-NAS>:6690;
    }

    server {
        listen     6690 ssl;
        proxy_pass synology_drive;
        
        ssl_certificate     /etc/letsencrypt/live/drive.my-domain.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/drive.my-domain.com/privkey.pem;
    }
}

However, when trying to set up the Synology Drive Client, it cannot connect/find the Synology NAS. When changing the /etc/nginx/fallthrough.conf to not use SSL, i.e.:

stream {

    upstream synology_drive {
         server <IP-OF-SYNOLOGY-NAS>:6690;
    }

    server {
        listen     6690;
        proxy_pass synology_drive;
    }
}

The Synology NAS is found, but the self-signed certificate stored on the NAS is used, instead of the certificate from Let's Encrypt stored on the reverse proxy computer.

Does anybody know what I am or may be doing wrong here?

The stream module is a layer 4 proxying module, that is, it only forwards TCP packets between the connecting client and proxied server.

As TLS happens on a higher level, nginx takes no part to the SSL encryption. I am actually surprised that nginx gives no errors on the ssl_certificate directives that are in your configuration.

Your only option is to copy the private key and certificate from nginx to the Synology device, as that is the entity that terminates TLS connection in this case.

I, too, recently ran into this domain of problems while trying to provide a LetsEncrypt-powered SSL proxy in front of Synology Drive for desktop clients.

I believe the core issue is that Synology Drive TCP 6690 is a proprietary protocol, one that begins unencrypted and then steps up to its own SSL handshake as requested.

Querying my NAS directly shows there is no SSL information initially available:

$ openssl s_client -connect localhost:6690 < /dev/null 
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 293 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

So when I configure socat to sit in front of my NAS and use a LetsEncrypt certificate:

$ socat -ddd -v OPENSSL-LISTEN:6690,reuseaddr,fork,su=nobody,key=/data/cert/key.pem,cert=/data/cert/cert.pem,cafile=/data/cert/fullchain.pem TCP:synology.local:6690
2023/08/28 04:44:41 socat[7] I setting option "so-reuseaddr" to 1
2023/08/28 04:44:41 socat[7] I setting option "fork" to 1
2023/08/28 04:44:41 socat[7] I setting option "substuser" to 65534
2023/08/28 04:44:41 socat[7] I setting option "openssl-key" to "/data/cert/key.pem"
2023/08/28 04:44:41 socat[7] I setting option "openssl-certificate" to "/data/cert/cert.pem"
2023/08/28 04:44:41 socat[7] I setting option "openssl-cafile" to "/data/cert/fullchain.pem"
2023/08/28 04:44:41 socat[7] I setting option "openssl-min-proto-version" to "TLS1.3"
2023/08/28 04:44:41 socat[7] I SSL_CTX mode has SSL_MODE_AUTO_RETRY set. Correcting..
2023/08/28 04:44:41 socat[7] I PEM_read_bio_DHparams(0x7f70f108bd90, NULL, NULL, NULL): error
2023/08/28 04:44:41 socat[7] I socket(2, 1, 6) -> 6
2023/08/28 04:44:41 socat[7] I starting accept loop
2023/08/28 04:44:41 socat[7] N listening on AF=2 0.0.0.0:6690

Which I can successfully interact with via openssl:

$ openssl s_client -connect drive.example.com:6690 < /dev/null
CONNECTED(00000005)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = drive.example.com
verify return:1
---
Certificate chain
 0 s:CN = drive.example.com
   i:C = US, O = Let's Encrypt, CN = R3
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug 27 21:41:59 2023 GMT; NotAfter: Nov 25 21:41:58 2023 GMT
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  4 00:00:00 2020 GMT; NotAfter: Sep 15 16:00:00 2025 GMT
 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
   i:O = Digital Signature Trust Co., CN = DST Root CA X3
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jan 20 19:14:03 2021 GMT; NotAfter: Sep 30 18:14:03 2024 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
REDACTED
-----END CERTIFICATE-----
subject=CN = drive.example.com
issuer=C = US, O = Let's Encrypt, CN = R3
---
No client certificate CA names sent
Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 5241 bytes and written 797 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
DONE

I get version number errors when connecting from the Synology Drive desktop client:

2023/08/28 04:48:51 socat[7] I accept(6, {2, AF=2 192.168.10.210:58099}, 16) -> 7
2023/08/28 04:48:51 socat[7] N accepting connection from AF=2 192.168.10.210:58099 on AF=2 172.17.0.2:6690
2023/08/28 04:48:51 socat[7] I permitting connection from AF=2 192.168.10.210:58099
2023/08/28 04:48:51 socat[7] N forked off child process 9
2023/08/28 04:48:51 socat[7] I close(7)
2023/08/28 04:48:51 socat[7] I still listening
2023/08/28 04:48:51 socat[7] N listening on AF=2 0.0.0.0:6690
2023/08/28 04:48:51 socat[9] I just born: child process 9
2023/08/28 04:48:51 socat[9] I close(4)
2023/08/28 04:48:51 socat[9] I close(3)
2023/08/28 04:48:51 socat[9] I just born: child process 9
2023/08/28 04:48:51 socat[9] I close(6)
2023/08/28 04:48:51 socat[9] E SSL_accept(): error:0A00010B:SSL routines::wrong version number
2023/08/28 04:48:51 socat[9] N exit(1)
2023/08/28 04:48:51 socat[9] I close(7)
2023/08/28 04:48:51 socat[7] N childdied(): handling signal 17
2023/08/28 04:48:51 socat[7] I childdied(signum=17)
2023/08/28 04:48:51 socat[7] I childdied(17): cannot identify child 9
2023/08/28 04:48:51 socat[7] I waitpid(): child 9 exited with status 1
2023/08/28 04:48:51 socat[7] I waitpid(-1, {}, WNOHANG): No child process
2023/08/28 04:48:51 socat[7] I childdied() finished

Assuming I used socat incorrectly, I tried another generic SSL-enabled socket tool (ghostunnel) and it, too, reports an invalid handshake from the Synology Drive client:

[183] 2023/08/28 05:15:56.268946 error on TLS handshake from 192.168.10.210:58297: tls: first record does not look like a TLS handshake

I think it's that the Synology Drive client sends some messages initially as non-SSL, and then later can step up to SSL, which breaks the standard SSL-enabled socket handshake process.

If this is true, then it seems like the LetsEncrypt certificate & key ought to exist on the Synology NAS itself, so it can step up connections with signed files.