We have set up OpenStack servers on two server hosts connected to a router and switch. We've divided the switch into two VLANs:
- VLAN2: 10.2.xx
- VLAN3: 10.3.xx
Our network architecture includes VMware-created nodes:
- Controller: Connected to:
- placement
- nova-API
- neutron-server
- glance
- keystone
- cinder.
- Five Compute Nodes: Each communicates with nova-api and neutron-server and also connects to VLAN2 and VLAN3.
The compute nodes have necessary services such as openstack-nova-compute and can connect to both VLANs.
Issue
After successfully creating instances in the dashboard, the instances are running fine but can't connect to the network. In the "Network" -> "Port" section, the status remains "Build". Upon querying logs, it's evident that the neutron-linuxbridge-agent on the compute node fails to identify the instance NIC.
Relevant Logs
[user@controller ~]$ . admin-openrc
[user@controller ~]$ openstack port list
+--------------------------------------+------+-------------------+---------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+---------------------------------------------+--------+
| 268d042d-b3c1-4a81-961c-1f4bf1e12d2f | | fa:16:3e:43:77:7a | ip_address='10.3.4.26', subnet_id='89e7de44-166d-4bd9-8d63-edb1120f1e0d' | BUILD |
| a79ce3a6-4af3-46d6-895e-842e5d54bf70 | | fa:16:3e:7d:f3:4d | ip_address='10.3.4.0', subnet_id='89e7de44-166d-4bd9-8d63-edb1120f1e0d' | BUILD |
+--------------------------------------+------+-------------------+---------------------------------------------+--------+
[user@Compute-Node-3 ~]$ sudo nmcli d
DEVICE TYPE STATE CONNECTION
ens34 ethernet Connected ens34
ens35 ethernet Connected ens35
brqa3d10201-4c bridge Connected(ext) brqa3d10201-4c
tap268d042d-b3 tun Connected(ext) tap268d042d-b3
lo loopback Unmanaged --
[user@Compute-Node-3 ~]$
Error Logs
2023-08-29 21:08:30.582 1231542 INFO neutron.plugins.ml2.drivers.agent._common_agent [req-04dd55a0-0501-4afa-86ac-b6f7cbed9bb7 - - - - -] Linux bridge agent Agent out of sync with plugin!
2023-08-29 21:08:30.583 1231542 INFO neutron.agent.securitygroups_rpc [req-04dd55a0-0501-4afa-86ac-b6f7cbed9bb7 - - - - -] Preparing filters for devices {'tap268d042d-b3'}
2023-08-29 21:08:31.107 1231542 INFO neutron.plugins.ml2.drivers.agent._common_agent [req-04dd55a0-0501-4afa-86ac-b6f7cbed9bb7 - - - - -] Port tap268d042d-b3 updated. Details: {'device': 'tap268d042d-b3', 'network_id': 'a3d10201-4c86-4db6-8aa5-f4d4747bc52a', 'port_id': '268d042d-b3c1-4a81-961c-1f4bf1e12d2f', 'mac_address': 'fa:16:3e:43:77:7a', 'admin_state_up': True, 'network_type': 'flat', 'segmentation_id': None, 'physical_network': 'provider', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '89e7de44-166d-4bd9-8d63-edb1120f1e0d', 'ip_address': '10.3.4.26'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False}
2023-08-29 21:08:31.453 1231542 ERROR neutron.plugins.ml2.drivers.agent._common_agent [req-04dd55a0-0501-4afa-86ac-b6f7cbed9bb7 - - - - -] Error in agent loop. Devices info: {'current': {'tap268d042d-b3'}, 'timestamps': {'tap268d042d-b3': 9}, 'added': {'tap268d042d-b3'}, 'removed': set(), 'updated': set()}: pr2modules.netlink.exceptions.NetlinkError: (101, 'Network is unreachable')
2023-08-29 21:08:31.453 1231542 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
2023-08-29 21:08:31.453 1231542 ERROR neutron.plugins.ml2.drivers.agent._common_agent File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 465, in daemon_loop
2023-08-29 21:08:31.453 1231542 ERROR neutron.plugins.ml2.drivers.agent._common_agent sync = self.process_network_devices(device_info)
2023-08-29 21:08:31.453 1231542 ERROR neutron.plugins.ml2.drivers.agent._common_ag
Network Agent List
[user@controller ~]$ . admin-openrc
[user@controller ~]$ openstack network agent list
+--------------------------------------+--------------------+-----------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+-----------------+-------------------+-------+-------+---------------------------+
| 18b075eb-d309-40f2-971a-000242a65025 | Linux bridge agent | network-service | None | :-) | UP | neutron-linuxbridge-agent |
| 4c347266-16a0-4aa3-a9df-830224736185 | Metadata agent | network-service | None | :-) | UP | neutron-metadata-agent |
| 62e12e3e-fcd4-4b78-be16-a60d881cc5f2 | Linux bridge agent | Compute-Node-4 | None | :-) | UP | neutron-linuxbridge-agent |
| 95cbadbd-ec54-42f4-96c7-81ce1f0ddbd4 | Linux bridge agent | Compute-Node-5 | None | :-) | UP | neutron-linuxbridge-agent |
| b4430f05-80d7-49ea-83fe-2e14db97da67 | DHCP agent | network-service | nova | :-) | UP | neutron-dhcp-agent |
| d3fe4062-30bb-4fb5-9618-0d07877142c3 | Linux bridge agent | Compute-Node-3 | None | :-) | UP | neutron-linuxbridge-agent |
| e90f054c-e2a8-4c87-a986-7f01a365912b | Linux bridge agent | Compute-Node-2 | None | :-) | UP | neutron-linuxbridge-agent |
| f3cc749d-5e40-42ad-b103-11b9657aee16 | Linux bridge agent | Compute-Node-1 | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+-----------------+-------------------+-------+-------+---------------------------+
neutron.conf in Network Node
[database]
connection = mysql+pymysql://neutron:********@10.2.3.1/neutron
[DEFAULT]
core_plugin = ml2
service_plugins =
transport_url = rabbit://openstack:********@10.2.3.1
auth_strategy = keystone
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
[keystone_authtoken]
www_authenticate_uri = http://10.2.3.4:5000
auth_url = http://10.2.3.4:5000
memcached_servers = 10.2.3.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = *******
[nova]
auth_url = http://10.2.3.4:5000
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = admin
username = nova
password = *******
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
neutron.conf in Compute Node
[DEFAULT]
transport_url = rabbit://openstack:********@10.2.3.1
auth_strategy = keystone
[cache]
[cors]
[database]
[healthcheck]
[keystone_authtoken]
www_authenticate_uri = http://10.2.3.4:5000
auth_url = http://10.2.3.4:5000
memcached_servers = 10.2.3.1:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = ********
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_policy]
[privsep]
[profiler]
[ssl]
linuxbridge_agent.ini: Provider Network (Option 1 in official guiance)
[DEFAULT]
[linux_bridge]
physical_interface_mappings = provider:ens34
[vxlan]
enable_vxlan = false
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
neutron-server.log in Network Node
OneDrive link:
/var/log/neutron/server.log
