Corrupted InnoDB table, mysqlcheck and mysqldump crash server

I manage a Percona XtraDB Cluster that uses a network storage with a flapping connection. Periodically we experience a high iowait with crashes and remount of the fs in read-only. Replacing the storage is, unfortunately, not an option for now.

Recently I noticed that when mysqldump or mysqlcheck are run, they crash the MySQL server on the node, with an error mysqlcheck: Got error: 2013: Lost connection to MySQL server during query when executing 'CHECK TABLE ... '

Here's the content of mysqld.log during the crash:

```
InnoDB: Error in pages 9479 and 9480 of index "PRIMARY" of table "foobar"."quux"
InnoDB: broken FIL_PAGE_NEXT or FIL_PAGE_PREV links
2015-09-28 14:39:45 7f015813b700 InnoDB: Page dump in ascii and hex (16384 bytes):

(...)

InnoDB: End of page dump
2015-09-28 14:39:45 7f015813b700 InnoDB: uncompressed page, stored checksum in field1 4038097986, calculated checksums for field1: crc32 2787032309, innodb 4038097986, none 3735928559, stored checksum in field2 1190336748, calculated checksums for field2: crc32 2787032309, innodb 1190336748, none 3735928559, page LSN 4 3652646491, low 4 bytes of LSN at page end 3652646491, page number (if stored to page already) 9479, space id (if created with >= MySQL-4.1.1 and stored already) 18
InnoDB: Page may be an index page where index id is 67
InnoDB: (index "PRIMARY" of table "foobar"."quux")
2015-09-28 14:39:45 7f015813b700 InnoDB: Page dump in ascii and hex (16384 bytes):

(...)

InnoDB: End of page dump
2015-09-28 14:39:46 7f015813b700 InnoDB: uncompressed page, stored checksum in field1 554678569, calculated checksums for field1: crc32 2178598661, innodb 554678569, none 3735928559, stored checksum in field2 1065260512, calculated checksums for field2: crc32 2178598661, innodb 1065260512, none 3735928559, page LSN 10 202985777, low 4 bytes of LSN at page end 202985777, page number (if stored to page already) 6792, space id (if created with >= MySQL-4.1.1 and stored already) 18
InnoDB: Page may be an index

mysqlcheck won't fix corruption in InnoDB table. You need to dump data from the table and re-create it.

Start MySQL with innodb_force_recovery option. Try values from 1 to 6 until MySQL starts.

Dump the table with mysqldump.

Drop the table.

Restart MySQL w/o innodb_force_recovery.

Reload the dump.

StackExchange Database Administrators Q#116730, answer score: 9