Kolla-Ansible fails to deploy Openstack because of MariaDB liveness step failing

Samuel Lourenço

4/21/24, 4:06 PM

I'm having issues when deploying Openstack via kolla-ansible on virtual machines that are running inside Proxmox, across three nodes. Both "kolla-ansible -i multinode bootstrap-servers" and "kolla-ansible -i multinode prechecks" run successfully without errors, but "kolla-ansible -i multinode deploy" fails during the steps where MariaDB service port liveness is being checked, on all three controller MV nodes (not to be confused with the three Proxmox that run on real machines).

Summary

Error

The nodes are as follows:

node1 (Proxmox HW) 1.1. controller1 (Ubuntu Server VM - 10.0.1.1/16 (eth1)) 1.2. compute1 (Ubuntu Server VM - 10.0.1.2/16 (eth1)) 1.3. block1 (Ubuntu Server VM - 10.0.1.3/16 (eth1))
node2 (Proxmox HW) 2.1. controller2 (Ubuntu Server VM - 10.0.2.1/16 (eth1)) 2.2. compute2 (Ubuntu Server VM - 10.0.2.2/16 (eth1)) 2.3. block2 (Ubuntu Server VM - 10.0.2.3/16 (eth1))
node3 (Proxmox HW) 3.1. controller3 (Ubuntu Server VM - 10.0.3.1/16 (eth1)) 3.2. compute3 (Ubuntu Server VM - 10.0.3.2/16 (eth1)) 3.3. block3 (Ubuntu Server VM - 10.0.3.3/16 (eth1)) 3.4. router (VyOS VM - 10.0.0.1/16 (eth1)) 3.5. management (Ubuntu Desktop VM - 10.0.0.2/16 (eth1))

Each VM has two interfaces: eth0 and eth1. It is important to note that eth0 is for the provider network, so the individual machines can get updates. IPs are assigned via DHCP and the network address is 192.168.1.0/24 on the provider. Conversely, eth1 on all machines if for the management network and the network address is 10.0.0.0/16. All IPs are static.

Tried to deploy multiple times. Also tried "kolla-ansible -i multinode mariadb_recovery" after the failed deployment, and "kolla-ansible -i multinode destroy --yes-i-really-mean-it", after which I ran the commands again without improvements.

I would like to attach the logs, and the configuration files used as well, but I don't have that option.

Kind regards, Samuel Lourenço

Edit: I think it can be due to this, but I don't know how to "bootstrap" nodes in order.

Edit 2: globals.yml: https://pastebin.com/jZ199HUw multinode: https://pastebin.com/eLAqH9sT controller1 MariaDB log: https://pastebin.com/mNXzSKiP controller2 MariaDB log: https://pastebin.com/FUxrLwji controller3 MariaDB log: https://pastebin.com/r2tpHQnM

293

0 + 4

ubuntu

mariadb

openstack

kolla-ansible

eblock

5/4/24, 6:52 AM

To me it looks like `safe_to_bootstrap` in `/var/lib/mysql//grastate.dat` is 1 for multiple nodes, that should usually not be possible and could prevent galera from starting. Can you verify if `safe_to_bootstrap: 1` is only on one node true and maybe set it manually to 0 on the other two nodes?

Samuel Lourenço

5/5/24, 9:25 AM

Thanks, eblock. I have three controller nodes, but I have 9 nodes in total. So, should I disable safe_to_bootstrap for the 3 controller nodes, or for all 9 nodes? Also, I don't have that file, perhaps because kolla-ansible installed MariaDB instead, but I'll have to investigate. Meanwhile, if you can help me further, I would be very grateful. However, thanks for the lead.

eblock

5/5/24, 9:34 AM

This is only relevant for the control nodes, do you see anything galera related in `/var/lib/mysql`? I'm not familiar with kolla so I can't really comment what to do to get out of this situation. But if the grastate file is not present (did you check inside the containers?) something seems really off.

Samuel Lourenço

5/5/24, 2:47 PM

Thank you. Unfortunately, I can't even see a mysql directory inside /var/lib. I'm yet to see if there is any equivalent configuration for MariaDB, or something of that order.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Kolla-Ansible fails to deploy Openstack because of MariaDB liveness step failing

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.