patternMajorpending
Kafka message ordering guarantees — partition key strategy
Viewed 0 times
partition keymessage orderingconsumer groupkey strategyrepartitioning
linuxdockernodejs
Problem
Messages arrive out of order at the consumer. Events for the same entity (user, order) are processed in wrong sequence, causing state corruption or conflicts.
Solution
(1) Kafka guarantees ordering ONLY within a partition, not across partitions. (2) Use a partition key to route related messages to the same partition: producer.send({ topic, key: userId, value }). (3) Key strategy: use the entity ID (user ID, order ID) so all events for that entity go to the same partition. (4) Don't use random keys or null keys — they distribute across partitions randomly. (5) With multiple consumers: each partition is consumed by exactly one consumer, maintaining order. (6) Caution: repartitioning (changing partition count) changes key->partition mapping — plan this carefully. (7) For strict global ordering: use a single partition (but this limits throughput to one consumer).
Why
Kafka partitions are the unit of parallelism and ordering. Within a partition, messages are appended sequentially and consumed in order. Across partitions, there's no ordering guarantee.
Revisions (0)
No revisions yet.