Enable UART communication (ttyS0 or serial0) on Pi4 Ubuntu Mate 20.04

I am using Raspberry Pi 4 (8G) version. It works well when communicating to Pixhawk via serial0 before. But several month ago, I changed the OS from Ubuntu 16 Ubiquity to Ubuntu Mate 20. Other functions work well, but when I try to use serial0 to connect to Pixhawk, I found serial0 doesn't exist. I did some research online and found some users have similar issues of serial0 disabled when using RPi3 and 4. I have tried to use some scripts they mentioned and some steps to change the config.txt related files, but none of them works for me. The main steps I followed are in Enable UART communication on Pi4 Ubuntu 20.04

I notice that the dmesg info has some differences: It should have three lines:

$ dmesg | grep tty
[    0.001061] printk: console [tty0] enabled
[    1.919367] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 29, base_baud = 0) is a PL011 rev2
[    1.924677] fe215040.serial: ttyS0 at MMIO 0x0 (irq = 31, base_baud = 62500000) is a 16550

But my RPi only has two lines:

ubuntu@ubuntu-desktop:~$ dmesg | grep tty
[    0.000245] printk: console [tty0] enabled
[    1.412273] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 14, base_baud = 0) is a PL011 rev2

ubuntu@ubuntu-desktop:~$ mavproxy.py --master=/dev/ttyS0,921600
Connect /dev/ttyS0,921600 source_system=255
Failed to connect to /dev/ttyS0,921600 : [Errno 2] could not open port /dev/ttyS0: [Errno 2] No such file or directory: '/dev/ttyS0'

So, it looks like ttyS0 is disabled. Do you think it has some relationship with the Kernel command line "8250.nr_uarts = 0"?

ubuntu@ubuntu-desktop:~$ dmesg | grep uart
[    0.000000] Kernel command line:  coherent_pool=1M 8250.nr_uarts=0 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=DC:A6:32:B2:AD:BF vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  net.ifnames=0 dwc_otg.lpm_enable=0 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc quiet splash quiet splash

Here're related files:

ubuntu@ubuntu-desktop:~$ cat /boot/firmware/config.txt
# Please DO NOT modify this file; if you need to modify the boot config, the
# "usercfg.txt" file is the place to include user changes. Please refer to
# the README file for a description of the various configuration files on
# the boot partition.

# The unusual ordering below is deliberate; older firmwares (in particular the
# version initially shipped with bionic) don't understand the conditional
# [sections] below and simply ignore them. The Pi4 doesn't boot at all with
# firmwares this old so it's safe to place at the top. Of the Pi2 and Pi3, the
# Pi3 uboot happens to work happily on the Pi2, so it needs to go at the bottom
# to support old firmwares.

[pi4]
kernel=uboot_rpi_4.bin
max_framebuffers=2

[pi2]
kernel=uboot_rpi_2.bin

[pi3]
kernel=uboot_rpi_3.bin

[all]
arm_64bit=1
device_tree_address=0x03000000

# The following settings are "defaults" expected to be overridden by the
# included configuration. The only reason they are included is, again, to
# support old firmwares which don't understand the "include" command.

# enable_uart=1
cmdline=cmdline.txt

include syscfg.txt
include usercfg.txt

# Enable the FKMS ("Fake" KMS) graphics overlay, enable the camera firmware
# and allocate 128Mb to the GPU memory
dtoverlay=vc4-fkms-v3d
gpu_mem=128
start_x=1

# Comment out the following line if the edges of the desktop appear outside
# the edges of your display
disable_overscan=1

# If you have issues with audio, you may try uncommenting the following line
# which forces the HDMI output into HDMI mode instead of DVI (which doesn't
# support audio output)
#hdmi_drive=2

# If you have a CM4, uncomment the following line to enable the USB2 outputs
# on the IO board (assuming your CM4 is plugged into such a board)
#dtoverlay=dwc2,dr_mode=host

ubuntu@ubuntu-desktop:~$ cat /boot/firmware/usercfg.txt 
# Place "config.txt" changes (dtparam, dtoverlay, disable_overscan, etc.) in
# this file. Please refer to the README file for a description of the various
# configuration files on the boot partition.

ubuntu@ubuntu-desktop:~$ cat /boot/firmware/README 
An overview of the files on the /boot/firmware partition (the 1st partition
on the SD card) used by the Ubuntu boot process (roughly in order) is as
follows:

* bootcode.bin   - this is the second stage bootloader loaded by all pis with
                   the exception of the pi4 (where this is replaced by flash
                   memory)
* config.txt     - the first configuration file read by the boot process
* syscfg.txt     - the file in which system modified configuration will be
                   placed, included by config.txt
* usercfg.txt    - the file in which user modified configuration should be
                   placed, included by config.txt
* start*.elf     - the third stage bootloader, which handles device-tree
                   modification and which loads...
* uboot*.bin     - various u-boot binaries for different pi platforms; these
                   are launched as the "kernel" by config.txt
* boot.scr       - the boot script executed by uboot*.bin which in turn
                   loads...
* vmlinuz        - the Linux kernel, executed by boot.scr
* initrd.img     - the initramfs, executed by boot.scr

ubuntu@ubuntu-desktop:~$ cat /boot/firmware/syscfg.txt 
# This file is intended to be modified by the pibootctl utility. User
# configuration changes should be placed in "usercfg.txt". Please refer to the
# README file for a description of the various configuration files on the boot
# partition.

enable_uart=0
dtparam=audio=on
dtparam=i2c_arm=on
dtparam=spi=on
cmdline=cmdline.txt

I also tried to change enable_uart=0 to 1 in /boot/firmware/syscfg.txt. After that, the system cannot boot which shows that file does influence the system. Then I changed the file back and system boots normally.

ubuntu@ubuntu-desktop:~$ cat /boot/firmware/cmdline.txt
net.ifnames=0 dwc_otg.lpm_enable=0 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc quiet splash

ubuntu@ubuntu-desktop:~$ sudo systemctl status serial-getty@ttyS0.service
● serial-getty@ttyS0.service
     Loaded: masked (Reason: Unit serial-getty@ttyS0.service is masked.)
     Active: inactive (dead)

ubuntu@ubuntu-desktop:~$ groups
ubuntu adm tty dialout cdrom sudo dip video plugdev input lxd lpadmin sambashare spi i2c gpio

This problem was solved by installing Ubuntu 20 Ubiquity. Even it is a beta version, it worked for me. It seems they did some work in Linux kernel configuration which made those physical ports work well.