Debug: Kubernetes pod evicted due to resource pressure

Pods are being evicted with status 'Evicted'. They keep getting removed and recreated.

Pod eviction diagnosis:

1. Check eviction reason:

kubectl get pods --field-selector=status.phase=Failed
kubectl describe pod <evicted-pod>
# Look for: 'The node was low on resource: ephemeral-storage'
# Or: 'The node was low on resource: memory'

1. Common eviction causes:

a) Disk pressure (ephemeral-storage):
# Containers writing too much to local storage
# Fix: Set resource limits
resources:
limits:
ephemeral-storage: 2Gi
# Fix: Write to volumes instead of container filesystem
# Fix: Clean up temp files, old logs

b) Memory pressure:
# Container exceeds memory limit -> OOMKilled
# Node total memory exhausted -> eviction
resources:
requests:
memory: 256Mi
limits:
memory: 512Mi

c) Node disk full:
kubectl describe node <node>
# Check: Conditions section for DiskPressure
# Fix: Clean images: docker system prune
# Fix: Increase node disk size

1. Prevention:

- Always set resource requests AND limits
- Use PriorityClass for critical pods
- Monitor node resources with metrics-server
- Set up pod disruption budgets
- Clean up completed/failed pods:
kubectl delete pods --field-selector=status.phase=Failed

1. Check eviction threshold:

kubelet --eviction-hard=memory.available<100Mi,nodefs.available<10%