-
- Joined
- Mar 22, 2026
-
- Messages
- 434
-
- Reaction score
- 10
-
- Points
- 0
Memory leaks in Python can be notoriously difficult to track down, especially in long-running applications like web servers, microservices, or data processing pipelines. Because Python handles memory management automatically via reference counting and a garbage collector, developers often assume they don't need to worry about leaks. However, circular references, global caches, and improper event listener registration can easily trap objects in memory forever.
Fortunately, Python's standard library includes a powerful built-in module called
Getting Started with Tracemalloc
To start tracking where memory is being allocated, you need to enable
Once tracing is active, you can take a snapshot of the memory state at any given point and compare it against a previous snapshot to see what has changed.
Writing a Basic Leak Detector
Let's look at a practical example of how to capture a memory leak using a snapshot comparison.
Analyzing the Output
When you run the script above,
This tells us immediately that line 10 in
Grouping by Traceback
By default,
This prints the exact stack trace, allowing you to trace the leak back to the specific module, function, and line that initiated the call.
Best Practices for Production
1. Overhead:
2. Sampling: If you need production monitoring, consider using
Fortunately, Python's standard library includes a powerful built-in module called
tracemalloc that allows us to trace Python memory allocations without needing external profiling tools.Getting Started with Tracemalloc
To start tracking where memory is being allocated, you need to enable
tracemalloc as early as possible in your application's lifecycle.
Python:
import tracemalloc
# Start tracing memory allocations
tracemalloc.start()
# Your application code goes hereOnce tracing is active, you can take a snapshot of the memory state at any given point and compare it against a previous snapshot to see what has changed.
Writing a Basic Leak Detector
Let's look at a practical example of how to capture a memory leak using a snapshot comparison.
Python:
import tracemalloc
import time
class LeakyCache:
def __init__(self):
self._cache = []
def add_data(self, data):
# Simulating an unbounded cache that grows indefinitely
self._cache.append(data)
def run_simulation():
tracemalloc.start()
# Take the first snapshot
snapshot1 = tracemalloc.take_snapshot()
cache = LeakyCache()
# Simulate leaky behavior
for i in range(100000):
cache.add_data(b"x" * 1024) # Allocating 1KB chunks
# Take the second snapshot
snapshot2 = tracemalloc.take_snapshot()
# Stop tracing
tracemalloc.stop()
# Compare snapshots
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 3 differences ]")
for stat in top_stats[:3]:
print(stat)
if __name__ == "__main__":
run_simulation()Analyzing the Output
When you run the script above,
tracemalloc will output the lines of code that are responsible for the largest memory increases. The output format typically looks like this:
Code:
[ Top 3 differences ]
filename.py:10: size=97.7 MiB (+97.7 MiB), count=100000 (+100000), average=1024 B
filename.py:3: size=4.5 KiB (+4.5 KiB), count=35 (+35), average=132 BThis tells us immediately that line 10 in
filename.py allocated roughly 97.7 megabytes across 100,000 allocations. Grouping by Traceback
By default,
compare_to groups allocations by line number ('lineno'). However, if a function is called from multiple places in your codebase, grouping by line number might not give you enough context. You can group by traceback to see the full call stack leading up to the allocation:
Python:
top_stats = snapshot2.compare_to(snapshot1, 'traceback')
stat = top_stats[0]
print(f"Memory allocated: {stat.size_diff / 1024:.1f} KiB")
for line in stat.traceback.format():
print(line)This prints the exact stack trace, allowing you to trace the leak back to the specific module, function, and line that initiated the call.
Best Practices for Production
1. Overhead:
tracemalloc adds a performance and memory overhead because it has to record the traceback for every memory allocation. Avoid running it permanently in high-performance production environments unless you are actively debugging.2. Sampling: If you need production monitoring, consider using
tracemalloc.start(nframe) with a lower frame limit, or trigger snapshots via an administrative endpoint (like a SIGUSR1 signal or a diagnostic HTTP route) only when memory usage spikes.