debugsqlMinor
MySql: Frequent crashes on server with substantial resources
Viewed 0 times
withcrashesmysqlsubstantialserverresourcesfrequent
Problem
My MySQL server running the latest version of MySql on Ubuntu 16.x keeps crashing once or twice a day. Sometimes it repairs itself fairly quickly (10 minutes). Sometimes I have to reboot and do an fsck to get things running again.
What would be causing this?
Things I've tried so far:
Gen Core I5).
Here is a typical mysql error log:
2017-04-20T18:43:46.958430Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 11791ms. The settings might not be optimal. (flu shed=92 and evicted=0, during the time.)
2017-04-20T18:44:11.989905Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 6822ms. The settings might not be optimal. (flushed=8 and evicted=0, during the time.)
2017-04-20T18:44:49.145162Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 5021ms. The settings might not be optimal. (flus hed=0 and evicted=0, during the time.)
2017
What would be causing this?
Things I've tried so far:
- increased RAM from 1.5GB up to 5GB.
- Hardware Upgrades: MotherBoard, processor, RAM (DDR4) but that didn't help (it was running a 7 year old processor, now running 7th
Gen Core I5).
- Setup UFW firewall to ensure it wasn't being caused by bots attacking MySQL or other services.
- In my.cnf, changed innodb_buffer_pool_size from 128MB to 500MB. didn't help but still in place
- I've run: mysqlcheck -u root -p --auto-repair --optimize --all-databases multiple times. didn't help
- In my.cnf, Decreased mysql max_connections from 151 to 80 and restarted mysql. didn't help
- Decreased apache MaxRequestWorkers from 150 to 100. Didn't help. Still crashing.
- I already had a 1GB Swap file. Left it.
- Scoured through Apache2 logs, SysLog, any other log that seemed appropriate but haven't found anything that caught my eye.
- Shutdown the server and tried to move the VM to another drive but it fails with File Error.
- My latest suspicion is this is being caused by a bad block but running badblocks seemed to trigger a crash at 25% complete. During the fsck I see this: fsck critical medium error, dev sda, sector 147306432
Here is a typical mysql error log:
2017-04-20T18:43:46.958430Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 11791ms. The settings might not be optimal. (flu shed=92 and evicted=0, during the time.)
2017-04-20T18:44:11.989905Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 6822ms. The settings might not be optimal. (flushed=8 and evicted=0, during the time.)
2017-04-20T18:44:49.145162Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 5021ms. The settings might not be optimal. (flus hed=0 and evicted=0, during the time.)
2017
Solution
- Shutdown the server and tried to move the VM to another drive but it
fails with File Error.
- My latest suspicion is this is being caused by
a bad block but running badblocks seemed to trigger a crash at 25%
complete. During the fsck I see this: fsck critical medium error, dev
sda, sector 147306432
Don't waste your time on DB tuning & others things, your drive seem definitely guilty for me...
Try to make a dump of the DB with as much as you can, try a low level drive formatting and start a clean VM.
Context
StackExchange Database Administrators Q#171558, answer score: 4
Revisions (0)
No revisions yet.