patternjavascriptModerate
Correlating logs with trace IDs by injecting span context into log records
Viewed 0 times
pino ^8.x, @opentelemetry/api ^1.x
trace idlog correlationpino mixinspan contexttraceIdspanIdgrafana derived fieldsloki tempo
Problem
Logs and traces are stored separately in different systems (e.g., Loki and Jaeger). When an error appears in logs, finding the corresponding trace requires manually copying trace IDs. Without automatic correlation, debugging distributed requests is extremely time-consuming.
Solution
Inject the active span's trace ID and span ID into every log record. With pino, use a mixin to read the current OTel context.
In Grafana, configure a derived field on
const pino = require('pino');
const { trace } = require('@opentelemetry/api');
const logger = pino({
mixin() {
const span = trace.getActiveSpan();
if (!span) return {};
const ctx = span.spanContext();
return {
traceId: ctx.traceId,
spanId: ctx.spanId,
traceFlags: ctx.traceFlags,
};
},
});
module.exports = logger;In Grafana, configure a derived field on
traceId to create a link from logs directly to Jaeger or Tempo.Why
When trace context is embedded in logs, you can jump from a specific log line to the full distributed trace in one click. This is the backbone of a good debugging workflow across microservices.
Gotchas
- trace.getActiveSpan() returns undefined if called outside an active span context — always guard with a null check
- The mixin is called at log time, so it captures the span active at the moment of logging, not at logger creation
- If using async context and the span is not propagated correctly (e.g., across event emitters), the context may be lost
- Loki derived fields and Grafana Tempo datasource must be configured in Grafana to enable clickable trace links
Context
Building a microservices observability stack where logs and traces must be connected
Revisions (0)
No revisions yet.