cURL hangs unless using IPv6 with HTTPS

eberts

5/31/24, 8:27 PM

I need to make some cURL requests to a server that doesn't have an ipv6 address, but my machine only seems able to connect when 1) using https, and 2) using ipv6. Other requests hang indefinitely after DNS resolution, before any connection is established.

Would appreciate any guidance on how to resolve this.

The machine is running Ubuntu 18.04.4 with curl 7.58.0 and ufw 0.36.

This happens with any domain, but here are some examples with google.com:

me@myserver:/$ curl -I4 http://www.google.com --trace-ascii /dev/stdout --no-tcp-nodelay
== Info: Rebuilt URL to: http://www.google.com/
== Info:   Trying 142.251.40.100...
*hangs*

me@myserver:/$ curl -I4 https://www.google.com --trace-ascii /dev/stdout --no-tcp-nodelay
== Info: Rebuilt URL to: https://www.google.com/
== Info:   Trying 142.251.40.100...
*hangs*

me@myserver:/$ curl -I6 http://www.google.com --trace-ascii /dev/stdout --no-tcp-nodelay
== Info: Rebuilt URL to: http://www.google.com/
== Info:   Trying 2607:f8b0:4006:81f::2004...
*hangs*

me@myserver:/$ curl -I6 https://www.google.com --trace-ascii /dev/stdout --no-tcp-nodelay
== Info: Rebuilt URL to: https://www.google.com/
== Info:   Trying 2607:f8b0:4006:81f::2004...
== Info: Connected to www.google.com (2607:f8b0:4006:81f::2004) port 443 (#0)
*no problems*

I've tried IP whitelisting in UFW, but it doesn't help:

me@myserver:/$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22                         ALLOW IN    Anywhere
80                         ALLOW IN    Anywhere
443                        ALLOW IN    Anywhere
Anywhere                   ALLOW IN    142.251.40.100
22 (v6)                    ALLOW IN    Anywhere (v6)
80 (v6)                    ALLOW IN    Anywhere (v6)
443 (v6)                   ALLOW IN    Anywhere (v6)

The machine is running Ubuntu 18.04.4.

Here's the (gently reformatted) output of curl -V:

curl 7.58.0 (x86_64-pc-linux-gnu) 
    libcurl/7.58.0 
    OpenSSL/1.1.1d 
    zlib/1.2.11 
    libidn2/2.3.0 
    libpsl/0.19.1 (+libidn2/2.0.4) 
    nghttp2/1.30.0 
    librtmp/2.3
Release-Date: 2018-01-24
Protocols: 
    dict file ftp ftps gopher http https 
    imap imaps ldap ldaps pop3 pop3s 
    rtmp rtsp smb smbs smtp smtps telnet tftp
Features: 
    AsynchDNS GSS-API HTTP2 HTTPS-proxy
    IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

235

0 + 6

ubuntu

ipv6

ufw

curl

Jaromanda X

6/1/24, 12:23 AM

your UFW rules are all incoming rules, your UFW allows all outgoing anyway - so, it's nothing to do with UFW - perhaps some other firewall between you machine and the internet is responsible

hardillb

6/1/24, 9:48 AM

Can you `traceroute` to the IPv4 address?

eberts

6/1/24, 9:18 PM

@JaromandaX Good point. Looking through iptables for anything suspicious now.

eberts

6/1/24, 9:24 PM

@hardillb The box don't have traceroute installed (and can't get apt working with the current network situation). `mtr -6b --tcp google.com` displays nothing, same using `-4`, but `--udp` and icmp packets both go through.

Jaromanda X

6/1/24, 11:19 PM

I think you misunderstood - I meant a physical machine ... like a router, or a firewall

eberts

6/2/24, 2:41 AM

@JaromandaX Ah, rereading your comment that does make sense. iptables on my machine look clear; I think you're right.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: cURL hangs unless using IPv6 with HTTPS

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.