Debug: Python memory usage growing unexpectedly

Python application memory usage keeps growing over time, indicating a memory leak.

Python memory leak detection:

import tracemalloc
import gc
import sys

# 1. tracemalloc: Track memory allocations
tracemalloc.start()

# ... run your code ...

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print('Top 10 memory consumers:')
for stat in top_stats[:10]:
    print(stat)

# 2. Compare snapshots to find growing allocations
tracemalloc.start()
snapshot1 = tracemalloc.take_snapshot()

# ... do work ...

snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print('Memory growth:')
for stat in top_stats[:10]:
    print(stat)

# 3. objgraph: Visualize object references
import objgraph

# What types have the most instances?
objgraph.show_most_common_types(limit=20)

# What's holding a reference to an object?
objgraph.show_backrefs(
    objgraph.by_type('MyClass')[0],
    max_depth=5,
    filename='backrefs.png'
)

# Find objects that grew between two points
objgraph.show_growth(limit=10)
# ... do work ...
objgraph.show_growth(limit=10)  # Shows what grew

# 4. Check reference counts
print(sys.getrefcount(my_object))  # Should decrease

# 5. Force garbage collection and check
gc.collect()  # Force GC
print(gc.garbage)  # Objects in reference cycles that can't be freed

# 6. Common leak patterns in Python:
# - Global lists/dicts that accumulate (caches without limits)
# - Circular references with __del__ methods
# - C extensions that don't release memory
# - Thread-local storage that grows
# - Observers/callbacks that hold references
# - Large closures capturing variables

# Quick check from outside
while true; do
  ps -o rss= -p $(pgrep -f myapp) | awk '{print $1/1024 " MB"}'
  sleep 10
done

Python uses reference counting + cyclic GC. Leaks happen when references are retained unintentionally. tracemalloc and objgraph pinpoint exactly what's growing and why.

Python application memory profiling