Can't ping Openstack guest VM from another machine in the provider network.

I have a small setup of Openstack Zed on Ubuntu 22.04. A control host, a compute host and one host "external" in the same networks used by Openstack (management and provider). The 3 hosts are VMs in Oracle Virtual Box (Network bridged, Promiscuous allowed for all, Nested VMs allowed)

----+-------------------+-----provider-net ---+--------------
    |                   |                     |
|---------------|  |----+------------|   |----+-------------|
| eth1          |  |   eth1          |   |  eth1            |
| 172.30.0.101  |  |   172.30.0.102  |   |  172.30.0.109    |
|               |  |                 |   |                  |
|               |  | |-------------| |   |                  |
|               |  | | guestVM     | |   |                  |
|               |  | | FIP         | |   |                  |
|               |  | | 172.30.0.77 | |   |                  |
|               |  | |-------------| |   |                  |
|               |  |                 |   |  EXTERNAL        |
| OS CONTROL    |  |  OS COMPUTE     |   |  no OS           |
| "zoscontrol"  |  |  "zoscompute1"  |   |  "zostmpl"       |
|               |  |                 |   |                  |
| 192.168.2.101 |  |  192.168.2.102  |   |   192.168.2.109  |
| eth0          |  |  eth0           |   |   eth0           |
|---------------|  |---+-------------|   |----+-------------|
    |                  |                     |
----+------------------+------managementnet--+--------------

I CAN reach (ping/ssh) the guest VM from the control node using its floating IP. However - I CANNOT reach the guest VM from the external host.

The IP connect says:

root@external:~# ip neigh
...
172.30.0.77 dev eth1  FAILED
...
root@external:~#


root@control:~# openstack security group rule list default
+-------------+-----------+-----------+------------+-----------+--------------------------------------+
| IP Protocol | Ethertype | IP Range  | Port Range | Direction | Remote Security Group                | 
+--------------------------------------+-------------+-----------+-----------+------------+-----------+
| None        | IPv4      | 0.0.0.0/0 |            | ingress   | a6021c94-6638-423b-b243-514df718e07b | 
| None        | IPv6      | ::/0      |            | egress    | None                                 | 
| icmp        | IPv4      | 0.0.0.0/0 |            | ingress   | None                                 | 
| tcp         | IPv4      | 0.0.0.0/0 | 22:22      | ingress   | None                                 | 
| None        | IPv4      | 0.0.0.0/0 |            | egress    | None                                 | 
| None        | IPv6      | ::/0      |            | ingress   | a6021c94-6638-423b-b243-514df718e07b | 
+--------------------------------------+-------------+-----------+-----------+------------+-----------+-
root@control:~#

Although I followed the standard documentation, I guess I miss some routing or security settings?? Any hints are appreciated!

========== config on control

root@zoscontrol:/etc/neutron# cat l3_agent.ini
[DEFAULT]
interface_driver = linuxbridge
[agent]
[network_log]
[ovs]

root@zoscontrol:/etc/neutron# cat neutron.conf
[DEFAULT]
core_plugin = ml2
service_plugins = router
transport_url = rabbit://openstack:****@zoscontrol
auth_strategy = keystone
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
[agent]
root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"
[cache]
[cors]
[database]
#connection = sqlite:////var/lib/neutron/neutron.sqlite
connection = mysql+pymysql://neutron:*****@zoscontrol/neutron
[experimental]
# https://stackoverflow.com/questions/74133695/feature-linuxbridge-is-experimental
# https://docs.openstack.org/neutron/latest//admin/config-experimental-framework.html
linuxbridge = true
[healthcheck]
[ironic]
[keystone_authtoken]
www_authenticate_uri = http://zoscontrol:5000
auth_url = http://zoscontrol:5000
memcached_servers = zoscontrol:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = *****
[nova]
auth_url = http://zoscontrol:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password = *****
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
...
[ssl]
root@zoscontrol:/etc/neutron#


root@zoscontrol:/etc/neutron/plugins/ml2# cat linuxbridge_agent.ini
[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:eth1
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = true
local_ip = 192.168.2.101
l2_population = true

root@zoscontrol:/etc/neutron/plugins/ml2# cat ml2_conf.ini
[DEFAULT]
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
[ml2_type_vxlan]
vni_ranges = 1:1000
[ovs_driver]
[securitygroup]
enable_ipset = true
[sriov_driver]
root@zoscontrol:/etc/neutron/plugins/ml2#

========== config on compute1

root@zoscompute1:/etc/neutron# cat neutron.conf
[DEFAULT]
core_plugin = ml2
transport_url = rabbit://openstack:****@zoscontrol
auth_strategy = keystone
[agent]
root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"
[cache]
[cors]
[database]
connection = sqlite:////var/lib/neutron/neutron.sqlite
[healthcheck]
[ironic]
[keystone_authtoken]
www_authenticate_uri = http://zoscontrol:5000
auth_url = http://zoscontrol:5000
memcached_servers = zoscontrol:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = *******
[nova]
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[placement]
[privsep]
[quotas]
[ssl]
root@zoscompute1:/etc/neutron#

root@zoscompute1:/etc/neutron/plugins/ml2# cat linuxbridge_agent.ini
[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:eth1
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = true
local_ip = 192.168.2.102
l2_population = true

========== config of VM and selfservice network

root@zoscontrol:/etc/neutron/plugins/ml2# openstack subnet show 062b9969-8d2d-4a02-aadc-0b18c6b2f180
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 10.10.10.2-10.10.10.99               |
| cidr                 | 10.10.10.0/24                        |
| created_at           | 2022-11-06T12:17:40Z                 |
| description          |                                      |
| dns_nameservers      |                                      |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 10.10.10.1                           |
| host_routes          |                                      |
| id                   | 062b9969-8d2d-4a02-aadc-0b18c6b2f180 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | doznetsub                            |
| network_id           | b6b682b3-2b43-42db-90fe-9edd3722d716 |
| project_id           | 587e458aa2cf49aea5d13e4a0f0c899c     |
| revision_number      | 1                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2022-11-06T19:44:06Z                 |
+----------------------+--------------------------------------+

 root@zoscontrol:~# openstack subnet show 0501c11f-36f2-4738-80ff-017232596de1
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 172.30.0.1-172.30.0.99               |
| cidr                 | 172.30.0.0/24                        |
| created_at           | 2022-11-06T12:14:11Z                 |
| description          |                                      |
| dns_nameservers      | 172.30.0.254                         |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 172.30.0.254                         |
| host_routes          |                                      |
| id                   | 0501c11f-36f2-4738-80ff-017232596de1 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | provider                             |
| network_id           | 3543a56b-a743-4bc7-b0ec-0811b1678ca0 |
| project_id           | fe07028a3944415ca0022c7082a5b4f9     |
| revision_number      | 1                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2022-11-06T19:52:19Z                 |
+----------------------+--------------------------------------+