ssh: port 22 Connection refuses

I have a question regarding remote access a machine via ssh. I'm trying to ssh login to my linux machine from my mac and I already setup the ssh-server on my ubuntu, opened the 22 port and firewall. However, I got these error message while connecting

[@Ges-MacBook ~]->ssh <username>@<remote machine's ipv4> -vvv
OpenSSH_8.6p1, LibreSSL 2.7.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug2: resolve_canonicalize: hostname <remote machine's ipv4> is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/geyan/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/geyan/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: ssh_connect_direct: entering
debug1: Connecting to <remote machine's ipv4> [<remote machine's ipv4>] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: connect to address <remote machine's ipv4> port 22: Connection refused
ssh: connect to host <remote machine's ipv4> port 22: Connection refused

Does anyone know what's the problem and how fix it?

Here are some configuration check results on the ubuntu I want to connect to:

ge@ge-ubuntu:~$ sudo ufw status
Status: active
To Action From
-- ------ ----
22 ALLOW Anywhere
22/tcp ALLOW Anywhere
22 (v6) ALLOW Anywhere (v6)
22/tcp (v6) ALLOW Anywhere (v6) 

ge@ge-ubuntu:~$ sudo service ssh status
[sudo] password for ge:
● ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-10-28 12:37:53 EDT; 2h 52min ago Docs: man:sshd(8) man:sshd_config(5) Process: 19004 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS) Main PID: 19005 (sshd) Tasks: 1 (limit: 38085) Memory: 2.6M CGroup: / └─19005 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
Oct 28 12:37:53 ge-ubuntu systemd[1]: Starting OpenBSD Secure Shell server...
Oct 28 12:37:53 ge-ubuntu sshd[19005]: Server listening on 0.0.0.0 port 22.
Oct 28 12:37:53 ge-ubuntu sshd[19005]: Server listening on :: port 22.
Oct 28 12:37:53 ge-ubuntu systemd[1]: Started OpenBSD Secure Shell server.
Oct 28 12:38:22 ge-ubuntu sshd[19034]: Accepted password for geyan from 127.0.0.1 port 40692 ssh2
Oct 28 12:38:22 ge-ubuntu sshd[19034]: pam_unix(sshd:session): session opened for user geyan by (uid=0)

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.