MySql Gap Lock Deadlock on Inserts

I'm getting Deadlocks from gap locks on a table when inserting into it frequently from multiple sources. Here is an overview of my processes.

START TRANSACTION
  UPDATE vehicle_image
  SET active = 0
  WHERE vehicleID = SOMEID AND active = 1

  Loop:
    INSERT INTO vehicle_image (vehicleID, vehicleImageFilePath, vehicleImageSplashFilePath
      ,vehicleImageThumbnailFilePath, vehicleImageMiniFilePath, mainVehicleImage, active)
    VALUES (%s, %s, %s, %s, %s, %s, 1);
END TRANSACTION

The output of SHOW Create table vehicle_image; is:

CREATE TABLE `vehicle_image` (
  `vehicleImageID` int(11) NOT NULL AUTO_INCREMENT,
  `vehicleID` int(11) DEFAULT NULL,
  `vehicleImageFilePath` varchar(200) DEFAULT NULL,
  `vehicleImageSplashFilePath` varchar(200) DEFAULT NULL,
  `vehicleImageThumbnailFilePath` varchar(200) DEFAULT NULL,
  `vehicleImageMiniFilePath` varchar(200) DEFAULT NULL,
  `mainVehicleImage` bit(1) DEFAULT NULL,
  `active` bit(1) DEFAULT b'1',
  `userCreated` int(11) DEFAULT NULL,
  `dateCreated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `userModified` int(11) DEFAULT NULL,
  `dateModified` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  PRIMARY KEY (`vehicleImageID`),
  KEY `active` (`active`),
  KEY `mainvehicleimage` (`mainVehicleImage`),
  KEY `vehicleid` (`vehicleID`)
) ENGINE=InnoDB AUTO_INCREMENT=22878102 DEFAULT CHARSET=latin1

And the last Deadlock given by SHOW engine innodb status:

```
LATEST DETECTED DEADLOCK
------------------------
2018-03-27 12:31:15 11a58
*** (1) TRANSACTION:
TRANSACTION 5897678083, ACTIVE 2 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1248, 2 row lock(s), undo log entries 1
MySQL thread id 873570, OS thread handle 0x124bc, query id 198983754 ec2-34-239-240-179.compute-1.amazonaws.com 34.239.240.179 image_processor update
INSERT INTO vehicle_image (vehicleID, vehicleImageFilePath, vehicleImageSplashFilePath, vehicleImageThumbnailFilePath, vehicleImageMiniFile

I'm not a MySQL expert, but by the look of your Deadlock logs, even though you are INSERTing different vehicle IDs per statement, those require the whole datapage (238326) of the VehicleID non-clustered index to be locked.

The fact you are occasionally getting deadlocks means that within 1 page you have multiple vehicle IDs, so there is a small chance that 2 different processes will need a lock for the same page.

Best thing to advise is to keep your transactions as small as possible.

If there is some way you can do the following, it'll help lessen the chance of a deadlock:

START TRANSACTION;
  UPDATE vehicle_image SET active = 0 WHERE vehicleID = SOMEID and active = 1;
END TRANSACTION;
Loop:
  START TRANSACTION;
  INSERT INTO vehicle_image (vehicleID, vehicleImageFilePath,
    vehicleImageSplashFilePath, vehicleImageThumbnailFilePath,
    vehicleImageMiniFilePath, mainVehicleImage, active)
  VALUES (%s, %s, %s, %s, %s, %s, 1);  
  END TRANSACTION;
--EndLoop here

If you can, try to change that index's fill factor to 95%, and test to see if you get fewer deadlocks.

A more extreme test would be to remove that index completely while INSERTing, then recreate it when done.

StackExchange Database Administrators Q#202455, answer score: 7