patterntypescriptMajor
Outbox pattern: guarantee at-least-once event delivery from a database write
Viewed 0 times
outbox patterntransactional outboxat-least-oncedebeziumrelaydual writeatomicitydatabase transaction
Problem
A service writes to its database and then publishes an event to Kafka. If the process crashes between the two operations, the event is lost — the DB has the new state but downstream services never learn about it.
Solution
Write the event to an
outbox table in the same database transaction as the business data. A separate relay process (or CDC tool like Debezium) reads the outbox and publishes to Kafka, then marks entries as processed.await db.transaction(async (trx) => {
await trx('orders').insert(order);
await trx('outbox').insert({
aggregate_type: 'order',
aggregate_id: order.id,
event_type: 'order.placed',
payload: JSON.stringify(order),
created_at: new Date(),
});
});
// relay picks up outbox rows and publishes to KafkaWhy
The outbox table and the business data are committed atomically. Even if the relay crashes, it restarts from unprocessed rows. Exactly-once delivery from the DB perspective with at-least-once to Kafka.
Gotchas
- Outbox rows must be cleaned up — archive or delete processed rows to prevent unbounded growth
- The relay introduces latency — typically acceptable but must be considered for time-sensitive events
- Consumers must handle duplicate events since the relay retries on failure
- Use a monotonic sequence or
created_at+idordering to process events in order
Context
Any service that must reliably publish events to a broker after a successful database write
Revisions (0)
No revisions yet.