patternMinor
Should I use multiple beats input in Logstash?
Viewed 0 times
beatsinputmultipleshouldlogstashuse
Problem
After some researches around the
I'll have events coming from roughly 500 machines, with a 20/80 windows/linux distribution. I plan to use multiples beats shipper, filebeat, metricbeat and maybe packetbeat.
Is there an interest at using one input per type/os couple or would just one input and "if type=..." in the filter pipeline be enough ?
beats input plugin and specially this rewrite I wonder if I should use only one beat input or multiples to handle multiples entry types.I'll have events coming from roughly 500 machines, with a 20/80 windows/linux distribution. I plan to use multiples beats shipper, filebeat, metricbeat and maybe packetbeat.
Is there an interest at using one input per type/os couple or would just one input and "if type=..." in the filter pipeline be enough ?
Solution
So still not benchmarked much, but after a quick code inspection and a bunch of reading about netty and logstash in version 5 the input is not the bottleneck to worry about.
Logstash team did put a bunch of work in the way the filters and outputs plugins are run in parallel, the beats input plugin wait for a batch of events, and the performances problem have indeed been solved in version 3.1.0 by the rewrite I quoted in the question.
Logstash team did put a bunch of work in the way the filters and outputs plugins are run in parallel, the beats input plugin wait for a batch of events, and the performances problem have indeed been solved in version 3.1.0 by the rewrite I quoted in the question.
Context
StackExchange DevOps Q#544, answer score: 6
Revisions (0)
No revisions yet.