I found out that it was a cyclic reference issue. So when new content replaces the old content in the array, the past objects were still remaining. It was quite an interesting finding because in the memory leaks analyser it showed up as few KB data leak which you wont suspect as the non freed virtual memory was few hundred Megabytes (MB).
As a very abstract example.
ClassA
@property (strong) ClassB *obj
----------
ClassB
@property (strong) ClassA *obj
- (id)initWithA:(ClassA *)objA;
----------
So when you remove A neither object will be deallocated properly. In my case leak traced by the leak analyser was few KB for both of the objects even though the CoreGraphics calculations were hanging onto about 200MB data in virtual memory.
Fix was to mark the A reference in ClassB as weak.
ClassB
@property (weak) ClassA *obj
- (id)initWithA:(ClassA *)objA;
Verdict
- Never under estimate a memory leak, no matter how big or small & arc or mrc