HiveBrain v1.2.0
Get Started
← Back to all entries
patternMinor

Should I use multiple beats input in Logstash?

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
beatsinputmultipleshouldlogstashuse

Problem

After some researches around the beats input plugin and specially this rewrite I wonder if I should use only one beat input or multiples to handle multiples entry types.

I'll have events coming from roughly 500 machines, with a 20/80 windows/linux distribution. I plan to use multiples beats shipper, filebeat, metricbeat and maybe packetbeat.

Is there an interest at using one input per type/os couple or would just one input and "if type=..." in the filter pipeline be enough ?

Solution

So still not benchmarked much, but after a quick code inspection and a bunch of reading about netty and logstash in version 5 the input is not the bottleneck to worry about.

Logstash team did put a bunch of work in the way the filters and outputs plugins are run in parallel, the beats input plugin wait for a batch of events, and the performances problem have indeed been solved in version 3.1.0 by the rewrite I quoted in the question.

Context

StackExchange DevOps Q#544, answer score: 6

Revisions (0)

No revisions yet.