2 Interfaces with different metric, and a single default gateway - losing connection

I have a weird issue. I have two interfaces: One is my WiFi, The other is a 4G USB

I am trying to set up the default GW through the 4G USB (which is NATed by the 4G provider, hence, I can't SSH through this interface into the device) because I need all the traffic to go out through the 4G interface.

I also need that all traffic from my local LAN to be funnelled through my WiFi interface, and, I need to be able to SSH (with port forwarding from my WiFi router) into the device.

The problem I am having is that when I configure everything with my WiFi as defaultGW, it works, and when I reverse the configuration the opposite, it doesn't.

let me show my configuration, which is just a simple netplan:

network:
  version: 2
  renderer: networkd
  wifis:
    wlx5:
      dhcp4: true
      dhcp4-overrides:
        route-metric: 200
        use-routes: false
      access-points:
        "myaccesspoint":
          password: "mypassword"
      routes:
        - to: 192.168.1.0/24
          via: 192.168.1.1
          table: 101
      routing-policy:
        - from: 192.168.1.0/24
          table: 101
  ethernets:
    enx9:
      dhcp4: true
      dhcp4-overrides:
        route-metric: 100
        use-routes: false
      routes:
        - to: default
          via: 192.168.225.1
        - to: 192.168.225.0/24
          via: 192.168.225.1
          table: 102
      routing-policy:
        - from: 192.168.225.0/24
          table: 102

enx9 is my 4G dongle.