I believe this behaviour is by design and correct according to UEFI specs. Some UEFI firmwares act differently regarding removable media and removed HDD. This is totally ok, because this is not defined in specs.
In short, you can try to enter your BIOS setting to see if you can manually create an entry with partition uuid. But the fireware can still decide to remove boot-entries reside on missing disks (I can confirm some dell firmware do this)
When boot into ubuntu use efibootmgr -v
to check the EFI boot entries. You should see something like:
Boot0004* ubuntu HD(1,GPT,uuid,0xnnnn,0xmmmmm)/File(\EFI\UBUNTU\SHIMX64.EFI)
Check if the uuid is the partition where the efi application located. Sometime the uuid can be something else, then it is wrong.
In long:
The removable devices should have /EFI/BOOT/BOOTX64.efi, and when the firmware see no boot-entry for that disk, it will take the /EFI/BOOT/BOOTX64.efi
The /EFI/BOOT/BOOTX64.efi
then call the fallback efi application: fbx64.efi
the fallback application then generate a boot entry according to BOOTX64.CSV and make it default and reset the machine. This is the ubuntu entry you saw.
Then the next boot will directly boot into ubuntu
Things will go south when you unplug the USB disk. The next boot the firmware will try to boot from the default option ubuntu, then the disk is missing. The firmware decide to remove the ubuntu entry or remove the default flag. This depends on how your firmware provider decide to do this.
Thus you can try to manually create ubuntu boot entry in bios or efimanager and do NOT mark it default. Then the entry may stay. Still depends on the firmware implementation, it is pretty ok if they decide to remove entries pointed to missing disk.