ssh: port 22 Connection refuses

I have a question regarding remote access a machine via ssh. I'm trying to ssh login to my linux machine from my mac and I already setup the ssh-server on my ubuntu, opened the 22 port and firewall. However, I got these error message while connecting

[@Ges-MacBook ~]->ssh <username>@<remote machine's ipv4> -vvv 

OpenSSH_8.6p1, LibreSSL 2.7.3 

debug1: Reading configuration data /etc/ssh/ssh_config 

debug1: /etc/ssh/ssh_config line 47: Applying options for * 

debug2: resolve_canonicalize: hostname <remote machine's ipv4> is address 

debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/geyan/.ssh/known_hosts' 

debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/geyan/.ssh/known_hosts2' 

debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling 

debug3: ssh_connect_direct: entering 

debug1: Connecting to <remote machine's ipv4> [<remote machine's ipv4>] port 22. 

debug3: set_sock_tos: set socket 3 IP_TOS 0x48 

debug1: connect to address <remote machine's ipv4> port 22: Connection refused 

ssh: connect to host <remote machine's ipv4> port 22: Connection refused

Does anyone know what's the problem and how fix it?

Here are some configuration check results on the ubuntu I want to connect to:

ge@ge-ubuntu:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       Anywhere                  
22/tcp                     ALLOW       Anywhere                  
22 (v6)                    ALLOW       Anywhere (v6)             
22/tcp (v6)                ALLOW       Anywhere (v6) 

ge@ge-ubuntu:~$ sudo service ssh status 
[sudo] password for ge: 
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-10-28 12:37:53 EDT; 2h 52min ago
       Docs: man:sshd(8)
             man:sshd_config(5)
    Process: 19004 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
   Main PID: 19005 (sshd)
      Tasks: 1 (limit: 38085)
     Memory: 2.6M
     CGroup: /system.slice/ssh.service
             └─19005 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups

Oct 28 12:37:53 ge-ubuntu systemd[1]: Starting OpenBSD Secure Shell server...
Oct 28 12:37:53 ge-ubuntu sshd[19005]: Server listening on 0.0.0.0 port 22.
Oct 28 12:37:53 ge-ubuntu sshd[19005]: Server listening on :: port 22.
Oct 28 12:37:53 ge-ubuntu systemd[1]: Started OpenBSD Secure Shell server.
Oct 28 12:38:22 ge-ubuntu sshd[19034]: Accepted password for geyan from 127.0.0.1 port 40692 ssh2
Oct 28 12:38:22 ge-ubuntu sshd[19034]: pam_unix(sshd:session): session opened for user geyan by (uid=0)