Score:2

Two ipv6 tunnels from different providers on the same host

in flag

My question is related to this question but with little difference:

I have an Ubuntu vps with a singe IPv4 on the main interface. I would like to have two separate IPv6 tunnels on this machine that are from two different providers with different endpoint address. When I try one of them alone. Every thing is okey. But when I add the second tunnel the first one still works but the second one cannot be pinged from outside. It can ping itself, however, and the gateway.

I add these line to my /etc/network/interfaces :

auto he-ipv6
iface he-ipv6 inet6 v4tunnel
        address 2001:xxx:xxx:xxxx::2
        netmask 64
        endpoint xx.xx.xx.xx
        local yy.yy.yyy.yy
        ttl 255
        gateway 2001:xxx:xxx:xxxx::1

auto sbtb-ipv6
iface sbtb-ipv6 inet6 v4tunnel
        address 2a09:xxx:xxx:xxxx::2
        netmask 64
        endpoint zz.zz.zz.zz
        local yy.yy.yyy.yy
        ttl 255
        gateway 2a09:xxx:xxx:xxxx::1

Does anybody know where the problem is or how can I further troubleshoot this?

A.B avatar
cl flag
A.B
It's a multi-homing problem (you attempt to have two default routes). This usually requires policy routing. There are plenty of examples for IPv4, maybe not so much for IPv6. Routing behavior with IPv6 will probably have very small subtle differences compared to IPv4 if you try to adapt something.
PouJa avatar
in flag
can you guide me on this please?
PouJa avatar
in flag
@A.B would you please help. I really need a solution on this and I don't know where to look for it.
A.B avatar
cl flag
A.B
And here you are.
fr flag
This scenario is not different from the one discussed in linked question. The only difference is administrative - two distinct tunnel providers versus one. From the network and configuration perspective they are the same.
Score:3
cl flag
A.B

A few remarks

For network configuration tools other than ifupdown, v4tunnel is really a layer 3 SIT tunnel which can be created depending on tools using the equivalent of ip tunnel add FOO mode sit ... or of ip link add name FOO type sit ....

Layer 3 interfaces don't have a layer 2 address (no Ethernet MAC address) and thus don't have to resolve the address of underlying layer 2: ARP (IPv4) and NDP (IPv6) don't happen with a layer 3 interface. For the same reason there's no gateway to resolve: a gateway is not needed for its gateway property but the syntax can still allow to guess the interface to use when creating a routing table entry without specifying the interface and that's usually what is done, including the tunnel broker's own configuration examples. In the end using /64 netmask and a gateway are not needed anywhere for OP's setup. But this answer will still stick to these settings, so it becomes easier for a reader to adapt it to a layer 2 interface tunnel where all these settings then become mandatory. Likewise, usually only settings about the non-default interface (here sbtb-ipv6) are needed, but anyway I'm providing a setup for the two interfaces. The only difference left is which interface gets the default route in the main routing table.

Multihoming and policy routing

This is now a case of a multihoming system using multiple addresses each with its own path to reach (IPv6) Internet. Upstream routers most certainly implement Strict Reverse Path Forwarding (SRPF) for anti-spoofing: each source address must transit through its dedicated path. With simple routing only one default route is used (the first displayed). It's possible to add a 2nd default route with longer metric, but it will never be chosen until the route with shorter metric disappears.

To overcome this, one can use policy routing. This allows the system to choose other routing tables (populated with different routes) and routing rules that will choose these routing tables with additional selectors rather than just use the destination to find the route. Here what's useful is the source address (and for a few cases also the outgoing interface when this interface is forced with SO_BINDTODEVICE).

So the goal is to create multiple alternate routing tables with a partial view of routes, based on entries from the main routing table: each routing table will consider that only one path to Internet exists and will have its own default route. Then a routing rule based on source address will select the adequate routing table for outgoing packets. In OP's case nothing special is needed for incoming packets, because the local routing table already handle this case.

Implementation

First the interfaces settings should be adjusting by adding a different metric in each, or the sbtb-ipv6 setup will probably fail because there can't be two default routes with same metric.

Then manually here's how to get this working. Add two routing tables with (arbitrary) values 2000 for HE and 2001 for SBTB and copy only related routes to each one. Metric doesn't matter here as there's no possible conflict.

ip -6 route add 2001:xxx:xxx:xxxx::/64 dev he-ipv6             table 2000
ip -6 route add default via 2001:xxx:xxx:xxxx::1 dev he-ipv6   table 2000

ip -6 route add 2a09:xxx:xxx:xxxx::/64 dev sbtb-ipv6           table 2001
ip -6 route add default via 2a09:xxx:xxx:xxxx::1 dev sbtb-ipv6 table 2001

And select the adequate route based on source address with routing rules (and to be explicit, state a fixed priority and state it's a locally initiated packet with iif lo):

ip -6 rule add pref 20000 from 2001:xxx:xxx:xxxx::2 iif lo lookup 2000
ip -6 rule add pref 20010 from 2a09:xxx:xxx:xxxx::2 iif lo lookup 2001

For the case where one binds to the interface (eg ping -I sbtb-ipv6 ...) rather than "simply" to the interface's address (ping -I 2a09:xxx:xxx:xxxx::2) also use this for proper results (really useful for layer 2 cases where no 2nd default route exists in the main routing table, not really needed in OP's case with layer 3 interfaces):

ip -6 rule add pref 20001 oif he-ipv6 lookup 2000
ip -6 rule add pref 20011 oif sbtb-ipv6 lookup 2001

Opposite direction (incoming traffic aka ingress) is already handled properly in the local routing table when addresses are added, so there's nothing special to do for ingress.

Final configuration

Above should be tried manually first, with a backup remote console access (though IPv4 being unaffected, IPv4 access should be enough). Then it can be integrated in the interfaces configuration file, using up/down commands wherever ifupdown doesn't support the feature (ie: all of policy routing). Be sure to remove previous manually tests, or a few commands in the interface setup will clash with errors such as RTNETLINK answers: File exists and the interfaces configuration will fail.

UPDATE: this also includes emitting three pings to the remote end of the SBTB tunnel, since from comments it appears this tunnel must see traffic from the server before it can accept traffic to the server. If the SBTB (tunneled) peer address is not enough to trigger accepting traffic to the server, replace it with some "well known" IPv6 address. Same could be added on HE interface but this doesn't appear needed.

auto he-ipv6
iface he-ipv6 inet6 v4tunnel
        address 2001:xxx:xxx:xxxx::2
        netmask 64
        endpoint xx.xx.xx.xx
        local yy.yy.yyy.yy
        ttl 255
        gateway 2001:xxx:xxx:xxxx::1
        metric 1000
        up   ip -6 route add 2001:xxx:xxx:xxxx::/64 dev he-ipv6 table 2000
        up   ip -6 route add default via 2001:xxx:xxx:xxxx::1 dev he-ipv6 table 2000
        up   ip -6 rule add pref 20000 from 2001:xxx:xxx:xxxx::2 iif lo lookup 2000
        up   ip -6 rule add pref 20001 oif he-ipv6 lookup 2000
        down ip -6 rule del pref 20001
        down ip -6 rule del pref 20000

auto sbtb-ipv6
iface sbtb-ipv6 inet6 v4tunnel
        address 2a09:xxx:xxx:xxxx::2
        netmask 64
        endpoint zz.zz.zz.zz
        local yy.yy.yyy.yy
        ttl 255
        gateway 2a09:xxx:xxx:xxxx::1
        metric 1001
        up   ip -6 route add 2a09:xxx:xxx:xxxx::/64 dev sbtb-ipv6 table 2001
        up   ip -6 route add default via 2a09:xxx:xxx:xxxx::1 dev sbtb-ipv6 table 2001
        up   ip -6 rule add pref 20010 from 2a09:xxx:xxx:xxxx::2 iif lo lookup 2001
        up   ip -6 rule add pref 20011 oif sbtb-ipv6 lookup 2001
        up ping -q -c3 -I sbtb-ipv6 2a09:xxx:xxx:xxxx::1
        down ip -6 rule del pref 20011
        down ip -6 rule del pref 20010

How to use?

Now one can bring up one or both interfaces, it will always work properly, with priority given to he-ipv6 when both are up. To use the sbtb-ipv6 interface when both are up, one should bind to its address or bind to the interface itself. Here are examples to display the route the kernel is expected to use:

# ip route get to 2001:db8:f00:ba7::1
2001:db8:f00:ba7::1 from :: via 2001:xxx:xxx:xxxx::1 dev he-ipv6 src 2001:xxx:xxx:xxxx::2 metric 1000 pref medium

# ip route get from 2001:xxx:xxx:xxxx::2 to 2001:db8:f00:ba7::1
2001:db8:f00:ba7::1 from 2001:xxx:xxx:xxxx::2 via 2001:xxx:xxx:xxxx::1 dev he-ipv6 table 2000 src 2001:xxx:xxx:xxxx::2 metric 1024 pref medium

# ip route get oif he-ipv6 to 2001:db8:f00:ba7::1
2001:db8:f00:ba7::1 from :: via 2001:xxx:xxx:xxxx::1 dev he-ipv6 table 2000 src 2001:xxx:xxx:xxxx::2 metric 1024 pref medium

# ip route get from 2a09:xxx:xxx:xxxx::2 to 2001:db8:f00:ba7::1
2001:db8:f00:ba7::1 from 2a09:xxx:xxx:xxxx::2 via 2a09:xxx:xxx:xxxx::1 dev sbtb-ipv6 table 2001 src 2a09:xxx:xxx:xxxx::2 metric 1024 pref medium

# ip route get oif sbtb-ipv6 to 2001:db8:f00:ba7::1
2001:db8:f00:ba7::1 from :: via 2a09:xxx:xxx:xxxx::1 dev sbtb-ipv6 table 2001 src 2a09:xxx:xxx:xxxx::2 metric 1024 pref medium

Various commands, daemons or tools require various options to bind to an other source address or device. Eg: ping -I, traceroute -i, curl --interface, ssh -b/ssh -B, etc.

One can check by comparing the result of one of these commands:

curl -6 https://ifconfig.co
curl -6 --interface 2001:xxx:xxx:xxxx::2 https://ifconfig.co
curl -6 --interface he-ipv6 https://ifconfig.co

versus the result of one of these:

curl -6 --interface 2a09:xxx:xxx:xxxx::2 https://ifconfig.co
curl -6 --interface sbtb-ipv6 https://ifconfig.co
PouJa avatar
in flag
This is a hilarious answer. Thanks a lot.
PouJa avatar
in flag
I learned a lot from your answer and tried to implement it. made the interfaces configuration file as you suggested above. Both interfaces are up. Internally every thing is good. But I cannot ping both IPs from a remote machine. It will only respond to he-ipv6. As soon as I bring down he-ipv6, sbtb will start to respond to pings. I need them both be pingable without having to bring down one of them.
PouJa avatar
in flag
Wow It works. I just needed to initiate a little traffic through sbtb before it becomes pingable. Maybe this is a condition from my tunnel broker. No each time I reboot my machine I have to initiate a connection from sbtb-ipv6. Can we tell ping6 command what interface to use?
PouJa avatar
in flag
One last question. What if I want to assign the whole subnet IP range to my machine. For example I want to be able to ping the whole 2001:xxx:xxx:xx::/64 from remote machines.
A.B avatar
cl flag
A.B
for ping it's in my answer: " Eg: `ping -I`" (followed by the interface name). For the whole subnet that's something else that can't fit with this answer. You have to understand what you manipulate. whole subnet is a problem in itself. Plus policy routing at the same time it must be even more difficult. Then you will think about routing (containers, vms, tunnel): that's yet an other thing that isn't answered in this answer
fr flag
Are you sure it is not your firewall which is blocking incoming traffic from Securebit tunnel until you ping it? Do you have rule which allows NEW IPv6-in-IPv4 traffic from them unconditionally? If not - you should. Ditto for HE tunnel.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.