I am running Debian Buster 10.12. I have galera configured with 3 nodes. I'm currently trying to process the following upgrade:
libmariadb-dev (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
libmariadb-dev-compat (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
libmariadb3 (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-client (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-client-10.5 (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-client-core-10.5 (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-common (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-server (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-server-10.5 (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mariadb-server-core-10.5 (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
mysql-common (1:10.5.10+maria~buster => 1:10.5.15+maria~buster)
After the update runs, I'm unable to get the node to start. The error being: 2022-05-01 20:24:22 0 [ERROR] WSREP: /home/buildbot/buildbot/build/gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():1170: Will never receive state. Need to abort.
Here is a snippet of the logs:
2022-05-01 20:24:22 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2022-05-01 20:24:22 0 [Note] WSREP: Opened channel 'db-cluster-mariacluster'
2022-05-01 20:24:22 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
2022-05-01 20:24:22 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2022-05-01 20:24:22 1 [Note] WSREP: Starting applier thread 1
2022-05-01 20:24:22 0 [Note] WSREP: STATE EXCHANGE: sent state msg: af98cb7e-c98c-11ec-8036-4bdaa86e34fe
2022-05-01 20:24:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: af98cb7e-c98c-11ec-8036-4bdaa86e34fe from 0 (mariacluster-db2)
2022-05-01 20:24:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: af98cb7e-c98c-11ec-8036-4bdaa86e34fe from 2 (mariacluster-db3)
2022-05-01 20:24:22 2 [Note] WSREP: Starting rollbacker thread 2
2022-05-01 20:24:22 0 [Note] WSREP: STATE EXCHANGE: got state msg: af98cb7e-c98c-11ec-8036-4bdaa86e34fe from 1 (mariacluster-db1)
2022-05-01 20:24:22 0 [Note] WSREP: Quorum results:
version = 6,
component = PRIMARY,
conf_id = 689,
members = 2/3 (joined/total),
act_id = 17942450,
last_appl. = 17942346,
protocols = 2/10/4 (gcs/repl/appl),
vote policy= 0,
group UUID = b61a8e41-7939-11eb-a472-c255843a33c7
2022-05-01 20:24:22 0 [Note] WSREP: Flow-control interval: [28, 28]
2022-05-01 20:24:22 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 17942451)
2022-05-01 20:24:22 1 [Note] WSREP: ####### processing CC 17942451, local, ordered
2022-05-01 20:24:22 1 [Note] WSREP: Process first view: b61a8e41-7939-11eb-a472-c255843a33c7 my uuid: af4b3fad-c98c-11ec-a283-bf212d4e4d7b
2022-05-01 20:24:22 1 [Note] WSREP: Server mariacluster-db1 connected to cluster at position b61a8e41-7939-11eb-a472-c255843a33c7:17942451 with ID af4b3fad-c98c-11ec-a283-bf212d4e4d7b
2022-05-01 20:24:22 1 [Note] WSREP: Server status change disconnected -> connected
2022-05-01 20:24:22 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-01 20:24:22 1 [Note] WSREP: ####### My UUID: af4b3fad-c98c-11ec-a283-bf212d4e4d7b
2022-05-01 20:24:22 1 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes
2022-05-01 20:24:22 0 [Note] WSREP: Service thread queue flushed.
2022-05-01 20:24:22 1 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1
2022-05-01 20:24:22 1 [Note] WSREP: State transfer required:
Group state: b61a8e41-7939-11eb-a472-c255843a33c7:17942451
Local state: 00000000-0000-0000-0000-000000000000:-1
2022-05-01 20:24:22 1 [Note] WSREP: Server status change connected -> joiner
2022-05-01 20:24:22 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-01 20:24:22 0 [Note] WSREP: Joiner monitor thread started to monitor
2022-05-01 20:24:22 0 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '192.168.216.139' --datadir '/var/lib/mysql/' --parent '17935' --mysqld-args --wsrep_start_position=b61a8e41-7939-11eb-a472-c255843a33c7:17942404'
WSREP_SST: [INFO] Using stunnel for SSL encryption: CA: '/etc/mysql/ssl/ca.crt', CAPATH='', ssl-mode='VERIFY_CA' (20220501 20:24:22.659)
2022.05.01 20:24:22 LOG4[ui]: Insecure file permissions on /etc/mysql/ssl/mariacluster.key
2022-05-01 20:24:22 1 [Note] WSREP: ####### IST uuid:00000000-0000-0000-0000-000000000000 f: 0, l: 17942451, STRv: 3
2022-05-01 20:24:22 1 [Note] WSREP: IST receiver addr using ssl://192.168.216.139:4568
2022-05-01 20:24:22 1 [Note] WSREP: IST receiver using ssl
2022-05-01 20:24:22 1 [Note] WSREP: Prepared IST receiver for 0-17942451, listening at: ssl://192.168.216.139:4568
2022-05-01 20:24:22 0 [Note] WSREP: Member 1.0 (mariacluster-db1) requested state transfer from '*any*'. Selected 0.0 (mariacluster-db2)(SYNCED) as donor.
2022-05-01 20:24:22 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 17942451)
2022-05-01 20:24:22 1 [Note] WSREP: Requesting state transfer: success, donor: 0
2022-05-01 20:24:22 1 [Note] WSREP: Resetting GCache seqno map due to different histories.
2022-05-01 20:24:22 1 [Note] WSREP: GCache history reset: b61a8e41-7939-11eb-a472-c255843a33c7:0 -> b61a8e41-7939-11eb-a472-c255843a33c7:17942451
2022-05-01 20:24:22 0 [Warning] WSREP: 0.0 (mariacluster-db2): State transfer to 1.0 (mariacluster-db1) failed: -125 (Operation canceled)
2022-05-01 20:24:22 0 [ERROR] WSREP: /home/buildbot/buildbot/build/gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():1170: Will never receive state. Need to abort.
2022-05-01 20:24:22 0 [Note] WSREP: gcomm: terminating thread
2022-05-01 20:24:22 0 [Note] WSREP: gcomm: joining thread
2022-05-01 20:24:22 0 [Note] WSREP: gcomm: closing backend
2022-05-01 20:24:22 0 [Note] WSREP: view(view_id(NON_PRIM,aaca8b78-b91d,711) memb {
af4b3fad-a283,0
} joined {
} left {
} partitioned {
aaca8b78-b91d,0
c4cb7478-ac36,0
})
2022-05-01 20:24:22 0 [Note] WSREP: PC protocol downgrade 1 -> 0
2022-05-01 20:24:22 0 [Note] WSREP: view((empty))
2022-05-01 20:24:22 0 [Note] WSREP: gcomm: closed
2022-05-01 20:24:22 0 [Note] WSREP: /usr/sbin/mariadbd: Terminated.
Terminated
Any ideas what I'm missing? This is a fully functional cluster on the current version. I ended up restoring a backup while I do some research before trying again.
