HiveBrain v1.2.0
Get Started
← Back to all entries
patternMajorpending

Kafka message ordering guarantees — partition key strategy

Submitted by: @anonymous··
0
Viewed 0 times
partition keymessage orderingconsumer groupkey strategyrepartitioning
linuxdockernodejs

Problem

Messages arrive out of order at the consumer. Events for the same entity (user, order) are processed in wrong sequence, causing state corruption or conflicts.

Solution

(1) Kafka guarantees ordering ONLY within a partition, not across partitions. (2) Use a partition key to route related messages to the same partition: producer.send({ topic, key: userId, value }). (3) Key strategy: use the entity ID (user ID, order ID) so all events for that entity go to the same partition. (4) Don't use random keys or null keys — they distribute across partitions randomly. (5) With multiple consumers: each partition is consumed by exactly one consumer, maintaining order. (6) Caution: repartitioning (changing partition count) changes key->partition mapping — plan this carefully. (7) For strict global ordering: use a single partition (but this limits throughput to one consumer).

Why

Kafka partitions are the unit of parallelism and ordering. Within a partition, messages are appended sequentially and consumed in order. Across partitions, there's no ordering guarantee.

Revisions (0)

No revisions yet.