How to restart MariaDB Galera cluster?

How to restart MariaDB Galera cluster?

After all node have been crashed I try to recover the cluster but without success.
I have only 2 nodes.

As documentation says I set a parameter on one of the node:

set global wsrep_provider_options="pc.bootstrap=true";

And then try to start first node:

systemctl start mariadb

After that I get an error:

```
Oct 11 16:11:12 proxy1 sh[2367]: 2016-10-11 16:11:12 140291677038720 [Note] /usr/sbin/mysqld (mysqld 10.1.18-MariaDB) starting as process 2402 ...
Oct 11 16:11:15 proxy1 sh[2367]: WSREP: Recovered position b6c1dc93-8fa7-11e6-933e-e64cd44e3be0:141
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] /usr/sbin/mysqld (mysqld 10.1.18-MariaDB) starting as process 2434 ...
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: Read nil XID from storage engines, skipping position init
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: wsrep_load(): Galera 25.3.18(r3632) by Codership Oy loaded successfully.
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: CRC-32C: using hardware acceleration.
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: Found saved state: b6c1dc93-8fa7-11e6-933e-e64cd44e3be0:-1
Oct 11 16:11:15 proxy1 mysqld[2434]: 2016-10-11 16:11:15 140047023368320 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 192.168.0.41; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_t

MariaDB Galera Cluster:

Solution 1:

1) I've changed safe_to_bootstrap parameter to 1 on one of the node in the file /var/lib/mysql/grastate.dat:

safe_to_bootstrap: 1

2) After that I killed all mysql processes:

killall -KILL mysql mysqld_safe mysqld mysql-systemd

3) And started a new cluster:

galera_new_cluster

4) All other nodes I reconnected to the new one:

systemctl restart mariadb

P.S. to install killall on CentOS use psmisc:

sudo yum install psmisc

Solution 2:

Another way to restart a MariaDB Galera Cluster is to use --wsrep-new-cluster parameter.

1) Kill all mysql processes:

killall -KILL mysql mysqld_safe mysqld mysql-systemd

2) On the most up to date node start a new cluster:

/etc/init.d/mysql start --wsrep-new-cluster

3) Now other nodes can be connected:

service mysql start --wsrep_cluster_address="gcomm://192.168.0.101,192.168.0.102,192.168.0.103" \
--wsrep_cluster_name="my_cluster"

Percona XtraDB Cluster:

Solution 1:

In case you can connect to the most up to date node then you can setup the node to bootstrap with the next SQL:

SET GLOBAL wsrep_provider_options='pc.bootstrap=true';

Solution 2:

In case if all your nodes are dead and can not be started, you can stop the old one cluster and run a new one. You must stop all the cluster nodes because they have an information about old nodes in the old cluster.

1) Kill all the mysql processes on all nodes:

killall -KILL mysql mysqld_safe mysqld mysql-systemd

2) Start a new cluster on the most up to date node:

systemctl start mysql@bootstrap.service

3) Start other nodes:

systemctl start mysql

StackExchange Database Administrators Q#151941, answer score: 14