The ability to attach Synology volumes has for some reason stopped working. This was working, but after the latest zed update from Ubuntu appears to have stopped working.
Running Openstack Zed on Ubuntu 22.04.
I am able to create volumes, but it appears Nova will not attach the volumes. I can see in the Synology Logs that the volume LUN has been created, that a target is mapped and attached to the LUN, but the iscsi attach fails.
Here is the output from nova-compute.log:
2023-04-24 15:16:00.287 5503 INFO nova.compute.manager [None req-c1c4fcc3-aac5-403f-9db0-a9a19d17b481 78b851184ddc4480b7a0b599e4379949 dde7c0000b9b4a5d9927d070f9794f1c - - default default] [instance: bd58eb24-556d-4c0c-926e-0fb860db81de] Attaching volume 707548c5-fe6b-41ef-8d78-b857e7a1a9a9 to /dev/vdc
2023-04-24 15:16:03.506 5503 WARNING os_brick.initiator.connectors.base [None req-c1c4fcc3-aac5-403f-9db0-a9a19d17b481 78b851184ddc4480b7a0b599e4379949 dde7c0000b9b4a5d9927d070f9794f1c - - default default] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi [None req-c1c4fcc3-aac5-403f-9db0-a9a19d17b481 78b851184ddc4480b7a0b599e4379949 dde7c0000b9b4a5d9927d070f9794f1c - - default default] Exception encountered during portal discovery: KeyError: 'target_lun'
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi Traceback (most recent call last):
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 206, in _get_ips_iqns_luns
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi ips_iqns_luns = method(connection_properties)
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 455, in _discover_iscsi_portals
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi luns = self._get_luns(connection_properties, iqns)
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/base_iscsi.py", line 41, in _get_luns
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi return luns or [con_props['target_lun']] * num_luns
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi KeyError: 'target_lun'
2023-04-24 15:16:03.573 5503 ERROR os_brick.initiator.connectors.iscsi
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi [None req-c1c4fcc3-aac5-403f-9db0-a9a19d17b481 78b851184ddc4480b7a0b599e4379949 dde7c0000b9b4a5d9927d070f9794f1c - - default default] Exception encountered during portal discovery: KeyError: 'target_lun'
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi Traceback (most recent call last):
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 206, in _get_ips_iqns_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi ips_iqns_luns = method(connection_properties)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 455, in _discover_iscsi_portals
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi luns = self._get_luns(connection_properties, iqns)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/base_iscsi.py", line 41, in _get_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return luns or [con_props['target_lun']] * num_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi KeyError: 'target_lun'
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi During handling of the above exception, another exception occurred:
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi Traceback (most recent call last):
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 523, in connect_volume
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return self._connect_multipath_volume(connection_properties)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/utils.py", line 95, in _wrapper
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return r(f, *args, **kwargs)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 423, in __call__
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi do = self.iter(retry_state=retry_state)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 360, in iter
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return fut.result()
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return self.__get_result()
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi raise self._exception
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 426, in __call__
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi result = fn(*args, **kwargs)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 706, in _connect_multipath_volume
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi ips_iqns_luns = self._get_ips_iqns_luns(connection_properties)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 220, in _get_ips_iqns_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi raise exception.TargetPortalNotFound(
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi os_brick.exception.TargetPortalNotFound: Unable to find target portal 192.168.3.14:3260.
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi During handling of the above exception, another exception occurred:
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi Traceback (most recent call last):
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 206, in _get_ips_iqns_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi ips_iqns_luns = method(connection_properties)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/iscsi.py", line 400, in _get_discoverydb_portals
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi luns = self._get_luns(connection_properties, iqns)
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi File "/usr/lib/python3/dist-packages/os_brick/initiator/connectors/base_iscsi.py", line 41, in _get_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi return luns or [con_props['target_lun']] * num_luns
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi KeyError: 'target_lun'
2023-04-24 15:16:03.589 5503 ERROR os_brick.initiator.connectors.iscsi
Here is the output from the synology logs:
info,2023/04/24 10:16:16,cinder,Delete iSCSI Target [Cinder-Target-707548c5-fe6b-41ef-8d78-b857e7a1a9a9].
info,2023/04/24 10:16:15,cinder,Unmap LUN [volume-707548c5-fe6b-41ef-8d78-b857e7a1a9a9] from Target [Cinder-Target-707548c5-fe6b-41ef-8d78-b857e7a1a9a9].
info,2023/04/24 10:16:13,cinder,Map LUN [volume-707548c5-fe6b-41ef-8d78-b857e7a1a9a9] to Target [Cinder-Target-707548c5-fe6b-41ef-8d78-b857e7a1a9a9].
info,2023/04/24 10:16:12,cinder,Create iSCSI Target [Cinder-Target-707548c5-fe6b-41ef-8d78-b857e7a1a9a9].
This is feeling like a race condition between cinder creating the target and nova trying to attach the target, but I am at a loss on what to change to fix the problem.
