principleMinor
Kafka Internal Data structure vs LSM tree
Viewed 0 times
structurelsminternalkafkadatatree
Problem
I was going through Database storage engines and found out about LSM trees. I had also read about Kafka architecture and know that Kafka internally is a commit log. I want to know if Kafka internally uses LSM data structure for append-only store or uses some other data structure for storing data?
Solution
No, Kafka does not use LSM trees or any tree based data structure for data representation. It relies upon Sequential IO so that it is not relying upon JVM to maintain cache which would then be presenting as an overhead.
Kafka generally implements a queue based structure rather than a tree based structure (memory + merge sort on disk , something like sstables as disk store format).
Details can be found in the Kafka documentation.
Kafka generally implements a queue based structure rather than a tree based structure (memory + merge sort on disk , something like sstables as disk store format).
Details can be found in the Kafka documentation.
Context
StackExchange Database Administrators Q#286205, answer score: 4
Revisions (0)
No revisions yet.