HiveBrain v1.2.0
Get Started
← Back to all entries
patternsqlMinor

what could cause a mysql process to close (clean)

Submitted by: @import:stackexchange-dba··
0
Viewed 0 times
closewhatprocesscouldmysqlcauseclean

Problem

We are using mysql master-master (version 5.5.32) for an API manager called Layer7.
The 2 hosts holding the instances are in the same ESX

From time to time we have mysql doing a clean shutdown for now reason

We check the cronjobs and they are none
We checked the logins to the hosts or the instance during this time and there were none

other than manually shutdown, shutdown from a script or app or shutdown due to a kill what could cause clean shutdown in a master-master config

here is the log of the host where the instance stopped

Thank you

Regards

```
150605 18:59:32 [Note] /usr/sbin/mysqld: Normal shutdown

150605 18:59:32 [Note] Event Scheduler: Purging the queue. 0 events
150605 18:59:32 [Note] Error reading relay log event: slave SQL thread was killed
150605 18:59:32 [ERROR] Error reading packet from server: Lost connection to MySQL server during query ( server_errno=2013)
150605 18:59:32 [Note] Slave I/O thread killed while reading event
150605 18:59:32 [Note] Slave I/O thread exiting, read up to log 'ssgbin-log.000351', position 91210654
150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360878 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360872 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360871 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360864 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360860 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360859 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360858 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360857 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360855 user: 'gateway'

150605 18:59:34 [Warning] /usr/sbin/mysqld: Forcing close of thread 21360854 u

Solution

The SHUTDOWN privilege can only be used when running mysqladmin

  • Apr 22, 2013 : /usr/libexec/mysqld: Normal shutdown, but my team doesn't do that?



  • Mar 06, 2013 : How to properly kill MySQL?



  • Feb 28, 2013 : mysql restart won't kill child processes on CentOS



Taking away SHUTDOWN grant will stop users from running mysqladmin shutdown.
Your Actual Problem

The error log you are presenting in the question is a restart of mysqld.

150605 18:59:32 [Note] /usr/sbin/mysqld: Normal shutdown
.
.
.
150605 18:59:36  InnoDB: Starting shutdown...
150605 18:59:36  InnoDB: Waiting for 76 pages to be flushed
150605 18:59:37  InnoDB: Shutdown completed; log sequence number 5077358548454
150605 18:59:37 [Note] /usr/sbin/mysqld: Shutdown complete

150605 18:59:37 mysqld_safe mysqld from pid file /var/lib/mysql/mtllnl7.mtl.cbc.ca.pid ended
150605 18:59:38 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql


mysqladmin cannot start or restart mysqld.

Someone or something is running service mysql restart

First run the history

# history | grep restart


If nothing comes up in conjunction with the restarts you see, you need to locate a cronjob on the DB Server that is performing service mysql restart. In some cases, it might be a cronjob on a remote server that no one remembers.

Code Snippets

150605 18:59:32 [Note] /usr/sbin/mysqld: Normal shutdown
.
.
.
150605 18:59:36  InnoDB: Starting shutdown...
150605 18:59:36  InnoDB: Waiting for 76 pages to be flushed
150605 18:59:37  InnoDB: Shutdown completed; log sequence number 5077358548454
150605 18:59:37 [Note] /usr/sbin/mysqld: Shutdown complete

150605 18:59:37 mysqld_safe mysqld from pid file /var/lib/mysql/mtllnl7.mtl.cbc.ca.pid ended
150605 18:59:38 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
# history | grep restart

Context

StackExchange Database Administrators Q#104343, answer score: 2

Revisions (0)

No revisions yet.