Join Log in
Score:0
avatar Server

Upgrading websocket using Nginx and Daphne. Only works if I put port number in address bar

in flag
VIDesignz

To start, I am not well versed in server proxies but I can get along a bit.

Here is the scenario.

I am running an app within a docker container, using django, nginx, daphne, redis on an apache server.

  1. Visit the website run by apache using example.com.
  2. Apache does a proxy pass to Nginx (running in a container) 80:8080 and 443:8443
  3. Then Nginx passes off to Daphne to run the actual app which is running on port 8000 in the container

The site runs perfect except websocket connection return a 404 when visiting example.com But if I visit example.com:8443 the websockets work as expected.

I tried passing the port number along with the proxy pass but it never seems to make it in the request headers.

I just really need to know where to look to figure this out. Is it an Apache, Nginx or Daphne issue?

I have tried so many things it's not worth listing at the moment. I am hoping the fact that the websockets work when I put the port number in the address bar is a clue.

Thank you in advance for your help!

Update

Apache directive to pass to Nginx

    SSLProxyEngine On
    SSLProxyVerify none
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off
    ProxyPass        "/" "https://localhost:8443/"
    ProxyPassReverse "/" "https://localhost:8443/"

Nginx config

upstream app {
    server app:8000;
}

server {
    listen 80;
    server_name secret.com;

    location / {
        proxy_pass http://app;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
    }

    location /static/ {
        alias /static/;
    }
}

server {
    listen 443 ssl;
    server_name secret.com;
    ssl_certificate /etc/nginx/certs/srv1.videsignz.com.crt;
    ssl_certificate_key /etc/nginx/private/srv1.videsignz.com.key;

    location / {
        try_files $uri @proxy_to_app;
    }

    location @proxy_to_app {
        proxy_pass http://app;

        proxy_ssl_certificate /etc/nginx/certs/srv1.videsignz.com.crt;
        proxy_ssl_certificate_key /etc/nginx/private/srv1.videsignz.com.key;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";

        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Forwarded-Port $server_port;
    }

    location /static/ {
        alias /static/;
    }
}

Docker Compose File

version: '3.7'

services:
  app:
    volumes:
      - assets:/assets
    image: videsignz/ktech:latest
    ports:
      - "8000:8000"
  nginx:
    build: ./nginx
    volumes:
      - assets:/assets
      - /etc/ssl/certs:/etc/nginx/certs
      - /etc/ssl/private:/etc/nginx/private
    ports:
      - "8080:80"
      - "8443:443"
    depends_on:
      - app
  redis:
    image: redis:latest
    ports:
      - "6379:6379"
    depends_on:
      - app

volumes:
  assets:

Entry Point File

#!/bin/sh

python manage.py collectstatic --no-input

daphne -b 0.0.0.0 -p 8000 core.asgi:application

Full Apache Virtual Host Config port 80 and Port 443

<VirtualHost 198.46.134.221:80>
  ServerName connect.ktechonline.com
    ServerAlias mail.connect.ktechonline.com www.connect.ktechonline.com
  DocumentRoot /home/connktechonline/public_html
  ServerAdmin [email protected]
  UseCanonicalName Off

  ## User connktechonline # Needed for Cpanel::ApacheConf
  <IfModule userdir_module>
    <IfModule !mpm_itk.c>
      <IfModule !ruid2_module>
        <IfModule !mod_passenger.c>
          UserDir disabled
          UserDir enabled connktechonline 
        </IfModule>
      </IfModule>
    </IfModule>
  </IfModule>

  # Enable backwards compatible Server Side Include expression parser for Apache versions >= 2.4.
  # To selectively use the newer Apache 2.4 expression parser, disable SSILegacyExprParser in
  # the user's .htaccess file.  For more information, please read:
  #    http://httpd.apache.org/docs/2.4/mod/mod_include.html#ssilegacyexprparser
  <IfModule include_module>
    <Directory "/home/connktechonline/public_html">
      SSILegacyExprParser On
    </Directory>
  </IfModule>

  

  <IfModule suphp_module>
    suPHP_UserGroup connktechonline connktechonline
  </IfModule>
  <IfModule suexec_module>
    <IfModule !mod_ruid2.c>
      SuexecUserGroup connktechonline connktechonline
    </IfModule>
  </IfModule>
  <IfModule ruid2_module>
    RMode config
    RUidGid connktechonline connktechonline
  </IfModule>
  <IfModule mpm_itk.c>
    # For more information on MPM ITK, please read:
    #   http://mpm-itk.sesse.net/
    AssignUserID connktechonline connktechonline
  </IfModule>
  <IfModule mod_passenger.c>
    PassengerUser connktechonline
    PassengerGroup connktechonline
  </IfModule>

  <IfModule alias_module>
    ScriptAlias /cgi-bin/ /home/connktechonline/public_html/cgi-bin/
  </IfModule>


    # Global DCV Rewrite Exclude
    <IfModule rewrite_module>
        RewriteOptions Inherit
    </IfModule>

        
        
            
                
                
  Include "/etc/apache2/conf.d/userdata/std/2_4/connktechonline/connect.ktechonline.com/*.conf"


  # To customize this VirtualHost use an include file at the following location
  # Include "/etc/apache2/conf.d/userdata/std/2_4/connktechonline/connect.ktechonline.com/*.conf"
</VirtualHost>

<VirtualHost 198.46.134.221:443>
  ServerName connect.ktechonline.com
  ServerAlias mail.connect.ktechonline.com www.connect.ktechonline.com webmail.connect.ktechonline.com cpcontacts.connect.ktechonline.com autodiscover.connect.ktechonline.com cpcalendars.connect.ktechonline.com cpanel.connect.ktechonline.com webdisk.connect.ktechonline.com
  DocumentRoot /home/connktechonline/public_html
  ServerAdmin [email protected]
  UseCanonicalName Off

  ## User connktechonline # Needed for Cpanel::ApacheConf
  <IfModule userdir_module>
    <IfModule !mpm_itk.c>
      <IfModule !ruid2_module>
        <IfModule !mod_passenger.c>
          UserDir disabled
          UserDir enabled connktechonline 
        </IfModule>
      </IfModule>
    </IfModule>
  </IfModule>

  # Enable backwards compatible Server Side Include expression parser for Apache versions >= 2.4.
  # To selectively use the newer Apache 2.4 expression parser, disable SSILegacyExprParser in
  # the user's .htaccess file.  For more information, please read:
  #    http://httpd.apache.org/docs/2.4/mod/mod_include.html#ssilegacyexprparser
  <IfModule mod_include.c>
    <Directory "/home/connktechonline/public_html">
      SSILegacyExprParser On
    </Directory>
  </IfModule>

  
  <Proxymatch ^https?://127\.0\.0\.1:(2082|2083|2077|2078|2079|2080|2086|2087|2095|2096)/>
       <IfModule security2_module>
          SecRuleEngine Off
       </IfModule>
  </Proxymatch>

  <IfModule mod_suphp.c>
    suPHP_UserGroup connktechonline connktechonline
  </IfModule>
  <IfModule suexec_module>
    <IfModule !mod_ruid2.c>
      SuexecUserGroup connktechonline connktechonline
    </IfModule>
  </IfModule>
  <IfModule ruid2_module>
    RMode config
    RUidGid connktechonline connktechonline
  </IfModule>
  <IfModule mpm_itk.c>
    # For more information on MPM ITK, please read:
    #   http://mpm-itk.sesse.net/
    AssignUserID connktechonline connktechonline
  </IfModule>
  <IfModule mod_passenger.c>
    PassengerUser connktechonline
    PassengerGroup connktechonline
  </IfModule>

  <IfModule alias_module>
    ScriptAlias /cgi-bin/ /home/connktechonline/public_html/cgi-bin/
  </IfModule>
  <IfModule ssl_module>
    SSLEngine on
    
    SSLCertificateFile /var/cpanel/ssl/apache_tls/connect.ktechonline.com/combined

    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
    <Directory "/home/connktechonline/public_html/cgi-bin">
      SSLOptions +StdEnvVars
    </Directory>
  </IfModule>
        
        
            
                
    SSLProxyEngine On
    SSLProxyVerify none
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off
    ProxyPass        "/" "https://localhost:8443/"
    ProxyPassReverse "/" "https://localhost:8443/"




  # To customize this VirtualHost use an include file at the following location
  # Include "/etc/apache2/conf.d/userdata/ssl/2_4/connktechonline/connect.ktechonline.com/*.conf"

    <IfModule headers_module>
    RequestHeader set X-HTTPS 1
    </IfModule>
    RewriteEngine On
                RewriteCond %{HTTP_HOST} =autodiscover.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =autodiscover.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^ http://127.0.0.1/cgi-sys/autodiscover.cgi [P]
                RewriteCond %{HTTP_HOST} =cpanel.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =cpanel.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^/(.*) /___proxy_subdomain_cpanel/$1 [PT]
        ProxyPass "/___proxy_subdomain_cpanel" "http://127.0.0.1:2082" max=1 retry=0
                RewriteCond %{HTTP_HOST} =cpcalendars.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =cpcalendars.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^/(.*) /___proxy_subdomain_cpcalendars/$1 [PT]
        ProxyPass "/___proxy_subdomain_cpcalendars" "http://127.0.0.1:2079" max=1 retry=0
                RewriteCond %{HTTP_HOST} =cpcontacts.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =cpcontacts.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^/(.*) /___proxy_subdomain_cpcontacts/$1 [PT]
        ProxyPass "/___proxy_subdomain_cpcontacts" "http://127.0.0.1:2079" max=1 retry=0
                RewriteCond %{HTTP_HOST} =webdisk.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =webdisk.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^/(.*) /___proxy_subdomain_webdisk/$1 [PT]
        ProxyPass "/___proxy_subdomain_webdisk" "http://127.0.0.1:2077" max=1 retry=0
                RewriteCond %{HTTP_HOST} =webmail.connect.ktechonline.com [OR]
                RewriteCond %{HTTP_HOST} =webmail.connect.ktechonline.com:443
            RewriteCond %{HTTP:Upgrade} !websocket   [nocase]

            RewriteRule ^/(.*) /___proxy_subdomain_webmail/$1 [PT]
        ProxyPass "/___proxy_subdomain_webmail" "http://127.0.0.1:2095" max=1 retry=0

                RewriteCond %{HTTP:Upgrade} websocket   [nocase]
                    RewriteCond %{HTTP_HOST} =cpanel.connect.ktechonline.com [OR]
                    RewriteCond %{HTTP_HOST} =cpanel.connect.ktechonline.com:443

                RewriteRule ^/(.*) /___proxy_subdomain_ws_cpanel/$1 [PT]
                RewriteCond %{HTTP:Upgrade} websocket   [nocase]
                    RewriteCond %{HTTP_HOST} =webmail.connect.ktechonline.com [OR]
                    RewriteCond %{HTTP_HOST} =webmail.connect.ktechonline.com:443

                RewriteRule ^/(.*) /___proxy_subdomain_ws_webmail/$1 [PT]

    RewriteRule ^/Microsoft-Server-ActiveSync /___proxy_activesync/$1 [PT]
    ProxyPass "/___proxy_activesync" "http://127.0.0.1:2090/Microsoft-Server-ActiveSync" max=1 retry=0
</VirtualHost>
344
0 + 7
in flag
VIDesignz
@MichaelHampton Updated. Like I said, the site works perfect in regards to being served. Just the websocket connections fail unless I add the port numbers to the url.
0
Reply
Gerard H. Pille avatar
in flag
Gerard H. Pille
You don't show the Apache config responsible for 80 -> 8080.
0
Reply
in flag
VIDesignz
@GerardH.Pille Added it to the end. It is an auto generated file.
0
Reply
Michael Hampton avatar
cz flag
Michael Hampton
Sorry, we cannot support cPanel systems. You can try to get help from cPanel support.
1
Reply
Gerard H. Pille avatar
in flag
Gerard H. Pille
I still don't see where Apache is handling connections on port 80.
0
Reply
in flag
VIDesignz
Here you go @GerardH.Pille Added it. My biggest problem is that when I perform the proxy pass, it breaks webmail.connect.ktechonline.com and mail.connect.ktechonline.com. I am not sure how to perform the proxypass only on the connect.ktechonline.com.
0
Reply
Gerard H. Pille avatar
in flag
Gerard H. Pille
There is no proxypass in what you added. A selective proxypass could be a rewriterule with P-flag preceded by rewritecond testing the FQDN. Plenty of those in the 443 config.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.