I'm working on a project right now where we have an embedded Linux system running some sensor/motor control functionality, and we're creating a GUI application to allow the user to control the system. In the future we may run both applications on a single system, but for the time being, we want to be able to install this as an upgrade to existing machines, and the existing controller boards don't have the capability to run the GUI as it's designed, so it will be running on a separate SBC (currently we're prototyping with an ODroid C4 running Android 9)
We have worked out the communication API between the two systems and I've been doing initial testing by running a direct ethernet connection between the two systems with static IP addresses, however due to deployment requirements and limitations of Android I need to figure out a different connection method for actual deployment of the product. We need the customer to be able to connect to the internet on the GUI system via either WiFi or ethernet, so the ODroid's onboard ethernet port can't be used for that purpose, and on top of that, the Android kernel has some internal behavior that only allows a single network connection to be active at any given time.
The exception to Android's network prioritization rules is USB network interfaces. Both the Linux controller system and the Android device itself have the ability to appear as a network interface to a remote host - on the controller system we have a USB gadget port available, and I enabled the g_ether kernel module in our system image, and on the GUI system we have a USB OTG port and I can enable USB tethering in the Android system settings.
I have been able to get communication between either of these systems and my Windows development system using their respective interfaces, but I have so far not figured out how to get either solution to talk to the other.
Linux system as remote interface(USB gadget with g_ether driver)
- Loading the g_ether kernel module on the linux system creates a
usb0 interface:
# modprobe g_ether
# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.42.128.107 netmask 255.255.255.0 broadcast 10.42.128.255
inet6 fe80::206:cff:fe01:1027 prefixlen 64 scopeid 0x20<link>
ether 00:06:0c:01:10:27 txqueuelen 1000 (Ethernet)
RX packets 500 bytes 54062 (52.7 KiB)
RX errors 0 dropped 46 overruns 0 frame 0
TX packets 0 bytes 24423 (23.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::24b:5bfc:8a9e:b148 prefixlen 64 scopeid 0x20<link>
ether 9a:8c:fa:85:ec:a1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 96 (96.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
- Connecting a USB cable from the gadget port on the linux system to a host port on the Android system immediately creates a
usb0 interface on Android and generates an IP address for the usb0 interface on Linux:
Android:
odroidc4:/ # ifconfig usb0
usb0 Link encap:Ethernet HWaddr fa:4c:95:c0:8d:18 Driver cdc_subset
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 TX bytes:0
Linux:
# ifconfig usb0
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.119.94 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::24b:5bfc:8a9e:b148 prefixlen 64 scopeid 0x20<link>
ether 9a:8c:fa:85:ec:a1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 96 (96.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
- Bringing up the interface on the Android system does not assign an IP address so I have to assign one manually, using the 169.254/16 space that the Linux system self-assigns
odroidc4:/ # ifconfig usb0 up
odroidc4:/ # ifconfig usb0
usb0 Link encap:Ethernet HWaddr fa:4c:95:c0:8d:18 Driver cdc_subset
inet6 addr: fe80::f84c:95ff:fec0:8d18/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:117 errors:4 dropped:75 overruns:0 frame:4
TX packets:27 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5080 TX bytes:4897
odroidc4:/ # ip addr add 169.254.1.1/16 broadcast 169.254.255.255 dev usb0
odroidc4:/ # ifconfig usb0
usb0 Link encap:Ethernet HWaddr fa:4c:95:c0:8d:18 Driver cdc_subset
inet addr:169.254.1.1 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::f84c:95ff:fec0:8d18/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:117 errors:4 dropped:75 overruns:0 frame:4
TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5080 TX bytes:8528
I have 2 problems at this point: First is that pinging either device from the other results in no response. Looking at the packet counts on the Android side makes it appear as though it is receiving the pings from the Linux side but is not responding, which may be caused by the second problem, which is that the routing tables do not appear to work as expected on the Android side. Given that I need this to work while there is also an active internet connection on the Android system, I have the ethernet connected at the same time, and this is what I see when looking at routing:
odroidc4:/ # ip route list
169.254.0.0/16 dev usb0 proto kernel scope link src 169.254.1.1
10.42.128.0/24 dev eth0 proto kernel scope link src 10.42.128.166
odroidc4:/ # ip route get 169.254.119.94
169.254.119.94 via 10.42.128.2 dev eth0 table eth0 src 10.42.128.166 uid 0
cache
odroidc4:/ # ip route show table 0
default via 10.42.128.2 dev eth0 table eth0 proto static
...
Android system as remote interface (USB tethering with OTG port)
This scenario seems to get me a little bit closer as I can get one-way communication working.
- After adding the necessary drivers in my Linux image, connecting a cable from the Android OTG port to a Linux host port shows the system recognizing the USB device in
dmesg:
[ 235.710937] usb 1-2.3: new full-speed USB device number 7 using at91_ohci
[ 235.862304] rndis_host 1-2.3:1.0 usb0: register 'rndis_host' at usb-at91-2.3, RNDIS device, 32:1d:e8:fc:dd:8
- A
usb0 interface is immediately created and assigned an IP address on the Linux end:
# ifconfig usb0
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 169.254.249.80 netmask 255.255.0.0 broadcast 169.254.255.255
inet6 fe80::e341:f9b1:b676:99b7 prefixlen 64 scopeid 0x20<link>
ether 32:1d:e8:fc:dd:80 txqueuelen 1000 (Ethernet)
RX packets 69 bytes 13572 (13.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 73 bytes 18251 (17.8 KiB)
TX errors 8 dropped 0 overruns 0 carrier 0 collisions 0
- On the Android end with USB tethering enabled in the settings, the usb0 interface appears but
starts out down. Bringing it up does not give it an IP address, so I assigned one manually, but as with the first scenario the routing table does not appear to function as expected:
odroidc4:/ # ifconfig usb0
usb0 Link encap:Ethernet HWaddr de:75:5c:41:2a:e6
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 TX bytes:0
odroidc4:/ # ifconfig usb0 up
odroidc4:/ # ifconfig usb0
usb0 Link encap:Ethernet HWaddr de:75:5c:41:2a:e6
inet6 addr: fe80::dc75:5cff:fe41:2ae6/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1090 TX bytes:2751
odroidc4:/ # ip addr add 169.254.1.1/16 broadcast 169.254.255.255 dev usb0
odroidc4:/ # ip route
10.42.128.0/24 dev eth0 proto kernel scope link src 10.42.128.166
169.254.0.0/16 dev usb0 proto kernel scope link src 169.254.1.1
odroidc4:/ # ip route get 169.254.249.80
169.254.249.80 via 10.42.128.2 dev eth0 table eth0 src 10.42.128.166 uid 0
cache
- However, in this scenario I can get successful communication originating from the Android end by manually specifying the interface:
odroidc4:/ # ping -c 5 -I usb0 169.254.249.80
PING 169.254.249.80 (169.254.249.80) from 169.254.1.1 usb0: 56(84) bytes of data.
64 bytes from 169.254.249.80: icmp_seq=1 ttl=64 time=1.56 ms
64 bytes from 169.254.249.80: icmp_seq=2 ttl=64 time=1.89 ms
64 bytes from 169.254.249.80: icmp_seq=3 ttl=64 time=1.71 ms
64 bytes from 169.254.249.80: icmp_seq=4 ttl=64 time=1.75 ms
64 bytes from 169.254.249.80: icmp_seq=5 ttl=64 time=1.85 ms
--- 169.254.249.80 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
2|odroidc4:/ # ping6 -c5 ff02::1%usb0
PING ff02::1%usb0(ff02::1) 56 data bytes
64 bytes from fe80::dc75:5cff:fe41:2ae6: icmp_seq=1 ttl=64 time=0.202 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=1 ttl=64 time=1.91 ms (DUP!)
64 bytes from fe80::dc75:5cff:fe41:2ae6: icmp_seq=2 ttl=64 time=0.196 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=2 ttl=64 time=1.80 ms (DUP!)
64 bytes from fe80::dc75:5cff:fe41:2ae6: icmp_seq=3 ttl=64 time=0.199 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=3 ttl=64 time=1.37 ms (DUP!)
64 bytes from fe80::dc75:5cff:fe41:2ae6: icmp_seq=4 ttl=64 time=0.199 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=4 ttl=64 time=1.85 ms (DUP!)
64 bytes from fe80::dc75:5cff:fe41:2ae6: icmp_seq=5 ttl=64 time=0.205 ms
--- ff02::1%usb0 ping statistics ---
5 packets transmitted, 5 received, +4 duplicates, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 0.196/0.883/1.914/0.776 ms
- I still can't get any traffic originating from the Linux side to work, presumably because of the routing problem on the Android end preventing return traffic from being routed correctly:
# ping -c 5 -I usb0 169.254.1.1
PING 169.254.1.1 (169.254.1.1) from 169.254.249.80 usb0: 56(84) bytes of data.
--- 169.254.1.1 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4295ms
# ping6 -c 5 ff02::1%usb0
PING ff02::1%usb0(ff02::1) 56 data bytes
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=1 ttl=64 time=0.522 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=2 ttl=64 time=0.302 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=3 ttl=64 time=0.376 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=4 ttl=64 time=0.383 ms
64 bytes from fe80::e341:f9b1:b676:99b7: icmp_seq=5 ttl=64 time=0.385 ms
--- ff02::1%usb0 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4009ms
rtt min/avg/max/mdev = 0.302/0.393/0.522/0.074 ms
In both scenarios it looks like the root of the problem is the routing table on the Android end not correctly routing traffic through the USB interface, and the Android end does not automatically assign an IP address while the Linux end does. With the Linux gadget interface/Android host I can somewhat understand these problems but with the Android OTG port tethering/Linux host I would expect it to route correctly since the Android system is in control of the tethering connection.
