I experienced the same problem with my application with UILabel(CALayer) increasing in memory usage in Xcode's profile tool. At the end of the day, UILabel(CALayer) increasing in memory ended up being a symptom of a memory leak caused by another issue (specifically a strong reference to a delegate).
I would check the following to make sure another is issue isn't causing the UILabel(CALayer) to be retained:
- Invalidate any NSTimers
- Remove any observers to NSNotificationCenter
- Make sure you are using weak references to self in blocks
- Make sure that any delegate properties use weak references
Source: http://www.reigndesign.com/blog/debugging-retain-cycles-in-objective-c-four-likely-culprits/