snippetdockerModerate
How to have multiple log streams in docker
Viewed 0 times
logdockerstreamsmultiplehowhave
Problem
We have an application that writes three types of logs into three separate files: access logs, generic application logs and system logs. The format (and purpose) of these logs are very different. And we have separate logforwarders that send them separately to our centralised logging system.
Based on the treat logs as event streams principle, we are thinking about moving from using files to stdout. While we know some of the benefit of this approach, this would also mean that we'd get a merged stream of the differently formatted logs, which we would need to split again either before we can send them to our central system (Kibana / Splunk / etc.), or inside there.
We're wondering whether there are any tools or recommendations on how we should approach this situation.
Based on the treat logs as event streams principle, we are thinking about moving from using files to stdout. While we know some of the benefit of this approach, this would also mean that we'd get a merged stream of the differently formatted logs, which we would need to split again either before we can send them to our central system (Kibana / Splunk / etc.), or inside there.
We're wondering whether there are any tools or recommendations on how we should approach this situation.
Solution
I'm still looking for a merging/splitting approach myself, but meanwhile this approach recommended by Kubernetes documentation seems like a sound solution: Use a sidecar container for each of your separate logs.
A "sidecar" is any docker container that you use alongside another docker container to work with it in some way. In this case for each of your three logs you would have a separate container that scans or tails the logs and outputs to stdout.
This way each of your log-sidecar-containers has its own docker-log from its own stdout. Being separate like this, you can use standard docker (and kubernetes, etc) practices for separating or aggregating. Here's what the Kubernetes page has to say:
This approach allows you to separate several log streams from different parts of your application, some of which can lack support for writing to stdout or stderr. The logic behind redirecting logs is minimal, so it’s hardly a significant overhead. Additionally, because stdout and stderr are handled by the kubelet, you can use built-in tools like kubectl logs.
The "separate log streams" stem from the built-in tagging that docker applies to logs from different containers, described in the docker documentation here:
The tag log option specifies how to format a tag that identifies the
container’s log messages. By default, the system uses the first 12
characters of the container id. To override this behavior, specify a
tag option
A "sidecar" is any docker container that you use alongside another docker container to work with it in some way. In this case for each of your three logs you would have a separate container that scans or tails the logs and outputs to stdout.
This way each of your log-sidecar-containers has its own docker-log from its own stdout. Being separate like this, you can use standard docker (and kubernetes, etc) practices for separating or aggregating. Here's what the Kubernetes page has to say:
This approach allows you to separate several log streams from different parts of your application, some of which can lack support for writing to stdout or stderr. The logic behind redirecting logs is minimal, so it’s hardly a significant overhead. Additionally, because stdout and stderr are handled by the kubelet, you can use built-in tools like kubectl logs.
The "separate log streams" stem from the built-in tagging that docker applies to logs from different containers, described in the docker documentation here:
The tag log option specifies how to format a tag that identifies the
container’s log messages. By default, the system uses the first 12
characters of the container id. To override this behavior, specify a
tag option
Context
StackExchange DevOps Q#422, answer score: 16
Revisions (0)
No revisions yet.