Calculating memory fragmentation in Python

Question

I have a long running process that allocates and releases objects constantly. Although objects are being freed, the RSS mem usage goes up over time.

How can I calculate how much fragmentation is happening? One possibility is to calculate RSS / sum_of_allocations and take that as an indicator. even then, how to do I calculate the denominator (sum_of_allocations).

Answer 1

Check out the Garbage Collector interface, gc.

http://docs.python.org/2/library/gc.html

You can inspect the objects are being tracked with gc.get_objects()

"As a general rule, instances of atomic types aren’t tracked and instances of non-atomic types (containers, user-defined objects...) are."

There is also gc.garbage, which finds objects that can't be freed but are unreachable.