Structured Logging Best Practices

Unstructured log messages (free-text strings) are impossible to query, aggregate, or alert on effectively. grep-based log analysis doesn't scale.

Use structured (JSON) logging with consistent fields:

import structlog
import logging

structlog.configure(
    processors=[
        structlog.processors.TimeStamper(fmt='iso'),
        structlog.processors.add_log_level,
        structlog.processors.JSONRenderer()
    ]
)
log = structlog.get_logger()

# BAD: unstructured
logging.info(f'User {user_id} placed order {order_id} for ${amount}')

# GOOD: structured
log.info('order_placed',
    user_id=user_id,
    order_id=order_id,
    amount=amount,
    currency='USD',
    items_count=len(items)
)
# Output: {"event": "order_placed", "user_id": 42, "order_id": "abc", ...}

Key fields to always include:

• timestamp (ISO 8601)
• level (info, warn, error)
• event or message (what happened)
• request_id / trace_id (correlation)
• service (which service)
• duration_ms (for performance events)

For errors, include:

• error_type, error_message
• stack_trace (but sanitize PII)
• Context that helps reproduce the issue

Structured logs can be indexed, queried (e.g., 'show all errors for user_id=42'), and aggregated into dashboards. Free-text logs require regex parsing that breaks when message format changes.

Building observable applications