Clients have no Internet access while Ubuntu 20.04 router does

I am recently trying to configure my Ubuntu 20.04 as a router. A weird problem occurred when I thought I finished all steps, that the clients connecting to the Ubuntu router just have no Internet access while the Ubuntu router itself does.

Before putting configurations here, I will put a picture of how the router is associating with other devices:

First Level Router >----<[WAN] Ubuntu Router(Dynamic IP) [LAN]>----< Client (DHCP)

Following the guide, I made these configurations on the Ubuntu router. /etc/netplan/00-installer-config.yaml:

network:
  ethernets:
    wan1:
      match:
        macaddress: xx:xx:xx:xx:xx:01
      set-name: wan1
    dhcp4: yes
  eth1:
    match:
      macaddress: xx:xx:xx:xx:xx:02
    set-name: eth1
    dhcp4: no
bridges:
  br:
    interfaces:
      - eth1
    addresses:
      - 192.168.3.1/24

The ifconfig output is here:

br: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    inet 192.168.3.1  netmask 255.255.255.0  broadcast 192.168.3.255
    inet6 fe80::f0de:2aff:fe06:98ad  prefixlen 64  scopeid 0x20<link>
    ether a4:1a:3a:b6:08:b5  txqueuelen 1000  (Ethernet)
    RX packets 7283  bytes 456695 (456.6 KB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 295  bytes 36494 (36.4 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    ether a4:1a:3a:b6:08:b5  txqueuelen 1000  (Ethernet)
    RX packets 7283  bytes 558657 (558.6 KB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 287  bytes 35750 (35.7 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
    inet 127.0.0.1  netmask 255.0.0.0
    inet6 ::1  prefixlen 128  scopeid 0x10<host>
    loop  txqueuelen 1000  (Local Loopback)
    RX packets 7941154  bytes 604164948 (604.1 MB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 7941154  bytes 604164948 (604.1 MB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wan1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.4  netmask 255.255.255.0  broadcast 192.168.1.255
    inet6 fe80::da50:e6ff:fe3f:fdcb  prefixlen 64  scopeid 0x20<link>
    inet6 2409:8a50:1873:70b3:da50:e6ff:fe3f:fdcb  prefixlen 64  scopeid 0x0                                                                                        <global>
    ether d8:50:e6:3f:fd:cb  txqueuelen 1000  (Ethernet)
    RX packets 39031  bytes 3017742 (3.0 MB)
    RX errors 0  dropped 2229  overruns 0  frame 0
    TX packets 35636  bytes 36600911 (36.6 MB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Also, I have enabled ipv4 forwarding by adding this line to /etc/sysctl.conf:

net.ipv4.ip forward=1

Iptables were set as the following line below, and maintained by making systemd run iptables-restore on-boot:

sudo iptables -t nat -A POSTROUTING -o wan1 -j MASQUERADE

I installed dnsmasq and edited /etc/dnsmasq.conf into the following below: (systemd-resolved was stopped and disabled)

listen-address=127.0.0.1,192.168.3.1
port=53
interface=eth1
dhcp-range=192.168.3.100,192.168.3.199,255.255.255.0,24h
dhcp-option=option:router,192.168.3.1
dhcp-option=option:dns-server,192.168.3.1

After I configured them, it seems like it was working. I can confirm DHCP is working as I can see the logs of dnsmasq, and the Ubuntu Router itself have Internet access.

But something was wrong. From the Client device, I tried to ping a domain. The domain was resolved to the correct IPV4 address, but the client cannot access it. Then I pinged from client to router and vice versa, both worked. The guide I followed never mentioned about this. Please let me know if you have any idea why it isn't working, thanks in advance!

Try add this rule:

iptables -A FORWARD --in-interface br -j ACCEPT

and eventually replace sudo iptables -t nat -A POSTROUTING -o wan1 -j MASQUERADE by

sudo iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o wan1 -j MASQUERADE

A Wireshark (or tshark, tcpdump) can be used for diagnose on both interfaces simultaneously.

Extended analysis

There can be many reasons why your router is not working:

• IP forwarding is not enabled
• DNS server is unreachable for LAN hosts
• NAT (network address translation) setting does not work for LAN hosts
• ...

Reboot your PC to get all services to their normal state.

Do not use names but numeric IP addresses to ping test until you verify that all other tests listed below are OK. You eliminate possible simultaneous DNS problem by this way.

Test 1

Check IP routing (forwarding) status (result 1 means forwarding is enabled):

cat /proc/sys/net/ipv4/ip_forward

Test 2

Check the correct NAT (masquerade) setting for your LAN interface:

ping -c 3 8.8.8.8
ping -c 3 -I 192.168.3.1   8.8.8.8

First command above is an accessibility test of destination IP address. You must get 3 valid ping responses. Second command tests whether the NAT is working for your LAN IP addresses. If you get no response, it means masquerade cannot translate your source address 192.168.3.1. and your other addresses in LAN (192.168.3.0/24) have the same problem too. Verify your NAT section of the iptables setting very carefully. Continue to next tests if the NAT setting looks good, but you get no ping response.

If ping response is OK but other network nodes in the LAN still cannot ping to 8.8.8.8, then possible solution is to specify complete network address (not only own br interface address 192.168.3.1) as source for NAT:

sudo iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -o wan1 -j MASQUERADE

Test 3

Outgoing WAN packets and their IP addresses:

Verify the tcpdump is installed and install it, if it is not present.

which tcpdump
sudo apt-get install tcpdump

Described process will guide you to check the tcpdump functionality and a network data capturing. You can stop the capturing process by pressing the Ctrl+C in running tcpdump instance window. Open second terminal window (T2). First (T1) will be used for traffic generation and response checking. You will use the second terminal (T2) to start and stop capture tasks using tcpdump. It is possible use only one terminal for both tasks, but if you are not familiar with the foreground and background task switching, it is more safe to use two terminals.

T2

sudo tcpdump -i wan1 -n icmp and host 8.8.8.8

T1

ping -c 2 8.8.8.8

You will see the two ICMP outgoing packets (echo request) and two incoming echo responses. Watch their source and destination IP addresses. Outgoing packet has destination IP 8.8.8.8 and source is your public WAN IP address. You will compare them with addresses displayed in next step.

Press Enter key in T2 terminal to create new line space there in output of running tcpdump. It helps you to separate old and new captured data in window.

T1

ping -c 2 -I 192.198.3.1  8.8.8.8

What do you see? There are these possibilities:

• no packets are captured
• only outgoing packets are displayed and they have wrong source IP adress (e.g. 192.168.3.1 instead of your public IP address of WAN interface.)
• both outgoing and incoming packets are visible.

Break the tcpdump in T2 window (Ctrl+C). Please, copy the response from T2 and paste it to your answer here. Add the response of ip route and eventually these commands:

ip neigh
sudo iptables-save

Describe results of other tests.