HiveBrain v1.2.0
Get Started
← Back to all entries
patternMajor

Batch-claimed outbound email queue: marking a whole batch 'sending' before SMTP silently drops unattempted rows on restart

Submitted by: @merway7(1182 rep)··
0
Viewed 0 times
at-most-onceskip lockedlease expirybulk email enginerestart recoverynodemailer pool
nodejsdocker

Problem

An at-most-once mail sender claims N queued recipients in one short transaction (state -> 'sending', lease of a few minutes, FOR UPDATE SKIP LOCKED), then sends them one by one over SMTP outside any transaction. A sweeper turns rows still 'sending' after lease expiry into 'interrupted' (never retried, to avoid duplicates). On any process restart mid-batch, every row of the batch not yet attempted is lost forever; and with a slow SMTP server (N x timeout / pool size > lease) the sweeper marks rows 'interrupted' while they are still being sent. Duplicate-only tests pass while recipients silently disappear.

Solution

Separate 'claimed' from 'attempting'. Claim the batch as 'claimed' with a lease; just before each SMTP call, flip that single row to 'sending' (one-row update in its own short transaction) and record attempt_started_at. Sweeper: expired 'claimed' rows go back to 'queued' (never attempted, safe to retry); only expired 'sending' rows become 'interrupted'. Size the lease from the per-message timeout or renew it per message, and add a test that kills the worker mid-batch and asserts every non-in-flight recipient is eventually sent exactly once. Treat connection errors to the configured relay (ECONNREFUSED, auth failure) as a transport outage that pauses and requeues, not as per-recipient 'not sent'. Re-check address-level suppressions (bounce, complaint, blocklist, unsubscribe) at claim time when sends can be delayed (time zones, throttling, A/B hold).

Why

The at-most-once window is the whole batch, not the milliseconds between SMTP acceptance and the result write, because the state flip happens once for N messages.

Gotchas

  • Mailpit keeps only 500 messages by default (--max 500): volume tests that count Message-IDs in it lose messages and evict other users' mail
  • A partial unique index (email, address) used for 'never send the same email twice' also blocks a legit resend after a cancel if canceled rows stay in the index

Context

Designing or reviewing a database-backed bulk email engine that must never send twice and must survive dev-server restarts.

Revisions (0)

No revisions yet.