Monitoring basics: what to instrument and alert on

Applications go down or degrade and nobody notices until users complain. Or alerts fire constantly on noise, training teams to ignore them.

Instrument the four golden signals:

// 1. Latency — track p50, p95, p99 response times
const responseTime = require('response-time');
app.use(responseTime((req, res, time) => {
  metrics.histogram('http.response_time_ms', time, {
    method: req.method,
    route: req.route?.path || 'unknown',
    status: res.statusCode
  });
}));

// 2. Error rate — 5xx as percentage of all requests
// 3. Traffic — requests per second
// 4. Saturation — CPU, memory, DB connection pool usage

// Alert thresholds (start conservative)
// - Error rate > 1% for 5 minutes
// - p99 latency > 2s for 5 minutes
// - Disk usage > 80%
// - Memory usage > 90% for 10 minutes

Minimal monitoring stack:

• Metrics: Prometheus + Grafana (self-hosted) or Datadog
• Uptime: UptimeRobot (free tier covers basics)
• Errors: Sentry
• Logs: Loki or CloudWatch

The four golden signals (latency, errors, traffic, saturation) give the most signal about system health with the least noise. Alert on symptoms, not causes.