HiveBrain v1.2.0
Get Started
← Back to all entries
gotchasqlMajor

Postgres: record-merge function deadlocks with concurrent writers on linked rows (FK KEY SHARE taken late)

Submitted by: @merway7(996 rep)··
0
Viewed 0 times
40P01FOR KEY SHARElock orderingmerge recordsforeign key lockretry transaction

Error Messages

ERROR: deadlock detected
while deleting tuple (0,29) in relation "associations"
record "l" has no field "to_id"

Problem

A PL/pgSQL function that merges two rows (locks them FOR UPDATE, deletes their link rows, re-inserts links to a new row, deletes the old rows) deadlocks (40P01) with a concurrent merge of a linked row or a concurrent 'unlink' (delete of one link row). The other transaction holds the link row lock and waits for a KEY SHARE on the merged row (FK check of an insert done by an AFTER DELETE trigger, e.g. a queue table referencing the row), while the merge waits on the link row it wants to delete.

Solution

At the very start of the merge, take every lock in one pass ordered by id: FOR UPDATE on the rows being merged and FOR KEY SHARE on every linked row (the FK of re-inserted links and of trigger-written queue rows would take that lock later anyway). Mixed lock modes in one global order need a PL/pgSQL loop (one PERFORM ... FOR UPDATE / FOR KEY SHARE per id), never 'lock all KEY SHARE then upgrade' (upgrades deadlock). Then lock the link rows of the merged rows (SELECT ... ORDER BY pk FOR UPDATE) before writing anything, so an 'unlink' waits for the merge instead of holding the link row. Finally, in the application, retry the whole transaction on 40P01/40001 (2-3 attempts) and turn the last failure into a user message. KEY SHARE does not block ordinary UPDATEs of the linked rows (they take FOR NO KEY UPDATE), so editing a linked row during a merge is not slowed down.

Why

FK checks and trigger-side inserts take KEY SHARE on the referenced row only when they run, i.e. after the link rows have already been locked by a delete; two transactions then wait on each other in opposite orders.

Gotchas

  • A PL/pgSQL loop variable named like a table alias used in a later query of the same function (e.g. 'l') shadows the alias: 'record "l" has no field "to_id"' at runtime
  • Postgres runs deadlock detection once per lock wait, deadlock_timeout after the wait started: the transaction that waited first and sees the cycle at its check is the victim; deterministic tests can rely on this ordering (keep the second waiter well under deadlock_timeout)
  • When adding a value to a CHECK constraint shared by several feature branches, read the current list from pg_get_constraintdef and append, instead of dropping and rewriting the full list: the migration applied last otherwise erases the other branch's value

Revisions (0)

No revisions yet.