Score:0

Can't connect to open secure port on Ubuntu

tk flag

I opened the 8443 port on which I run Clickhouse server. I can connect to SSH on 22 port, I can also connect to 8443 via SSH tunnel, however I can't connect normally to that host. I'm trying to connect from the Windows machine, if that is related anyhow. I even opened outbound port (pretty sure that it is redundant).

I tried to disable firewall and then I was able to connect. What can be wrong?

user@myhost:~/d/clickhouse$ sudo ufw status
To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere                  
9440/tcp                   ALLOW       Anywhere                  
8443/tcp                   ALLOW       Anywhere                  
8443                       ALLOW       Anywhere                  
22/tcp (v6)                ALLOW       Anywhere (v6)             
9440/tcp (v6)              ALLOW       Anywhere (v6)             
8443 (v6)                  ALLOW       Anywhere (v6)             
8443/tcp (v6)              ALLOW       Anywhere (v6)

user@myhost:~/d/clickhouse$ sudo lsof -iTCP -sTCP:LISTEN -P
COMMAND      PID            USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
systemd-r    841 systemd-resolve   13u  IPv4   26021      0t0  TCP localhost:53 (LISTEN)
vsftpd       901            root    3u  IPv6   26299      0t0  TCP *:21 (LISTEN)
sshd        1037            root    3u  IPv4   29181      0t0  TCP *:22 (LISTEN)
sshd        1037            root    4u  IPv6   29183      0t0  TCP *:22 (LISTEN)
docker-pr  86081            root    4u  IPv6  520074      0t0  TCP *:8088 (LISTEN)
docker-pr 287023            root    4u  IPv6 1831110      0t0  TCP *:8086 (LISTEN)
docker-pr 318522            root    4u  IPv6 2109586      0t0  TCP *:9440 (LISTEN)
docker-pr 318537            root    4u  IPv6 2110806      0t0  TCP *:8443 (LISTEN)
node      354955           user   18u  IPv4 2274703      0t0  TCP localhost:34575 (LISTEN)

user@myhost:~/d/clickhouse$ netstat -an | grep "LISTEN "
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:34575         0.0.0.0:*               LISTEN     
tcp6       0      0 :::21                   :::*                    LISTEN     
tcp6       0      0 :::8086                 :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 :::8088                 :::*                    LISTEN     
tcp6       0      0 :::8443                 :::*                    LISTEN     
tcp6       0      0 :::9440                 :::*                    LISTEN 

UPDATE:

on the server I ran sudo tcpdump -ni eth0 port 8443 and then on client machine I ran nc -zv 192.168.1.58 8443:

user@myhost:~$ sudo tcpdump -ni eth0 port 8443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:05:51.368952 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434934937 ecr 0,nop,wscale 7], length 0
15:05:52.380268 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434935948 ecr 0,nop,wscale 7], length 0
15:05:54.460280 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434938028 ecr 0,nop,wscale 7], length 0
15:05:58.540705 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434942109 ecr 0,nop,wscale 7], length 0
15:06:06.940802 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434950509 ecr 0,nop,wscale 7], length 0
15:06:23.581056 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434967149 ecr 0,nop,wscale 7], length 0
15:06:56.221198 IP 192.168.1.70.59364 > 192.168.1.58.8443: Flags [S], seq 2263747478, win 64240, options [mss 1460,sackOK,TS val 1434999788 ecr 0,nop,wscale 7], length 0

and nc failed with the message nc: connect to 192.168.1.58 port 8443 (tcp) failed: Connection timed out

The output of sudo ufw status verbose

user@myhost:~$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere                  
9440/tcp                   ALLOW IN    Anywhere                  
8443/tcp                   ALLOW IN    Anywhere                  
8443                       ALLOW IN    Anywhere                  
22/tcp (v6)                ALLOW IN    Anywhere (v6)             
9440/tcp (v6)              ALLOW IN    Anywhere (v6)             
8443 (v6)                  ALLOW IN    Anywhere (v6)             
8443/tcp (v6)              ALLOW IN    Anywhere (v6)  

I can connect to service if firewall is disabled:

nc -zv 192.168.1.58 8443 
Connection to 192.168.1.58 8443 port [tcp/*] succeeded!

I can connect to service with IPv4 address if firewall is disabled: enter image description here

Wayne Vosberg avatar
bd flag
At first glance it appears you have opened 8443 (ufw) on IPv4 but it is listening only on IPv6 (lsof/netstat output). Try `sudo ufw disable` and see if it works then.
Wayne Vosberg avatar
bd flag
just to be clear, `ufw disable` is just for a quick test. If it then works do `sudo ufw enable` and open 8443 on IPv6 as well (something like `sudo ufw allow from any to any port 8443`
Dmitrij Kultasev avatar
tk flag
@WayneVosberg it worked with disabled firewall. After that I ran your command and still can't connect. I updated the question with the new results for netstat/lsof
Dmitrij Kultasev avatar
tk flag
I also tried to run `sudo ufw allow proto tcp from any to any port 8443`. Updated `ufw status` part as well
raj avatar
cn flag
raj
Your `netstat` output shows that some service is listening on port 8443 **only on IPv6 address**. Nothing at all is listening on port 8443 on IPv4 address. This has nothing to do with firewall, but with the service configuration itself. For some reason the service binds only to IPv6 address. You must check the configuration.
Dmitrij Kultasev avatar
tk flag
@raj then how can I connect to it by specifying IPv4 address when firewall is disabled? (added screenshot)
raj avatar
cn flag
raj
I can only guess - is there something configured on your server that redirects connections from IPv4 to IPv6 ? I see your service is running in the Docker container, then that "something" can sit somewhere on the Docker level, between your host OS and the service itself. And maybe firewall interferes with that "something". I think it will be better if the service listened directly on IPv4.
Dmitrij Kultasev avatar
tk flag
that's weird as I see from the `docker ps` that the port is forwarded to the IPv4 port, as well from all the configurations I can see the same: `2229b0ce11e7 test/clickhouse "/entrypoint.sh" 2 days ago Up 2 days 8123/tcp, 9000/tcp, 0.0.0.0:8443->8443/tcp, 9009/tcp, 0.0.0.0:9440->9440/tcp clickhouse`
Score:1
tk flag

Finally I fixed that by running following command: sudo ufw route allow proto tcp from any to any port 8443

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.