I have a Domain Controller(Windows server 2016) and installed ADFS on another machine(Windows Server 2016) on the same domain. I can access the IDP signon page in the domain machines internally. Now, I am trying to provide a reverse proxy to the adfs server by using a web application proxy which is a standalone server(2016). I tried whether this machine is able to connect to ADFS machine by pinging the IP and using hostname and I got response bytes successfully to WAP server. I have used the same SSL certificate I created using Certificate Authority based on this site. But on trying to establish the connection between the WAP server and the ADFS server I am getting errors like
The underlying Connection was closed: Could not establish a trust relationship for the SSL/TLS secure channel.
So, I have reinstalled the web application again and this time I have binded the certificate to port 443 using IIS manager in WAP and to ports 443 and 49443 on the ADFS server. But this time I got error like this
An error occurred when attempting to establish a trust relationship with the federation service. Error: Service unavailable
I have checked whether the proxy endpoint('/adfs/Proxy/EstablishTrust/') is disabled in ADFS but it is enabled though. I have a doubt this may be due to certificate-based issues. Need help on this issue and steps to followed while creating ADFS service externally accessible.
Edit: I have checked the packets through wireshark for the 'service unavailable' message, it occurs because the DNS entry for the federation server was not found in Proxy server. But I have created a entry in Hosts file of proxy server and I am able to ping the ADFS server with FQDN name.