Memory leak when resizing images using CoreGraphics on background thread

Question

I'm building an app that uploads resized versions of many images to the server, and I seem to have a memory leak that I can't track down. Here's my method that creates the multi-part request (using Restkit 0.2):

[self.objectManager
     multipartFormRequestWithObject:nil
     method:RKRequestMethodPUT
     path:pathString
     parameters:@{@"photo": photoParamater}
     constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
       @autoreleasepool {
         if (uploadImage) {
           NSData *imageData =
           [photo scaledImageDataWithSmallerDimension:IMAGE_UPLOAD_SMALLER_DIMENSION
                                   compressionQuality:IMAGE_UPLOAD_JPEG_QUALITY];
           imageDataBytes += imageData.length;
           [formData appendPartWithFileData:imageData
                                       name:photo.file_key.absoluteString
                                   fileName:photo.filename
                                   mimeType:@"image/jpg"];
         }
       }
     }];

This is part of a method that is called from the main() function of a NSOperation on a background queue.

Here's are the photo methods that get me the actual photo data. The meat of it is in //3

//1
- (NSData *)scaledImageDataWithSmallerDimension:(CGFloat)length compressionQuality:(float)quality
{
    CGSize newSize = [self scaledSizeWithSmallerDimension:length];
    return [self imageDataResizedToFitSize:newSize compressionQuality:quality];
}

//2
- (NSData *)imageDataResizedToFitSize:(CGSize)size compressionQuality:(float)quality
{
    return [self thumbnailDataForAsset:self.asset maxPixelSize:MAX(size.height, size.width) compressionQuality:quality];
}

//3
- (NSData *)thumbnailDataForAsset:(ALAsset *)asset
                     maxPixelSize:(NSUInteger)size
               compressionQuality:(float)quality {
  @autoreleasepool {
    NSParameterAssert(asset != nil);
    NSParameterAssert(size > 0);

    ALAssetRepresentation *rep = [asset defaultRepresentation];

    CGDataProviderDirectCallbacks callbacks = {
      .version = 0,
      .getBytePointer = NULL,
      .releaseBytePointer = NULL,
      .getBytesAtPosition = getAssetBytesCallback,
      .releaseInfo = releaseAssetCallback,
    };

    CGDataProviderRef provider = CGDataProviderCreateDirect((void *)CFBridgingRetain(rep),
                                                            [rep size],
                                                            &callbacks);
    CGImageSourceRef source = CGImageSourceCreateWithDataProvider(provider, NULL);

    NSDictionary *imageOptions =
    @{
      (NSString *)kCGImageSourceCreateThumbnailFromImageAlways : @YES,
      (NSString *)kCGImageSourceThumbnailMaxPixelSize : [NSNumber numberWithUnsignedInteger:size],
      (NSString *)kCGImageSourceCreateThumbnailWithTransform : @YES,
      };
    CGImageRef imageRef = CGImageSourceCreateThumbnailAtIndex(source,
                                                              0,
                                                              (__bridge CFDictionaryRef) imageOptions);
    CFRelease(source);
    CFRelease(provider);

    if (!imageRef) {
      return nil;
    }


    NSMutableData *outputData = [[NSMutableData alloc] init];
    CGImageDestinationRef destRef = CGImageDestinationCreateWithData((__bridge CFMutableDataRef) outputData,
                                                                     kUTTypeJPEG,
                                                                     1,
                                                                     NULL);
    NSDictionary *imageAddOptions =
    @{
      (NSString *)kCGImageDestinationLossyCompressionQuality : [NSNumber numberWithFloat:quality],
      };
    CGImageDestinationAddImage(destRef,
                               imageRef,
                               (__bridge CFDictionaryRef) imageAddOptions);
    CGImageDestinationFinalize(destRef);
    CFRelease(imageRef);
    CFRelease(destRef);

    return outputData;
  }
}

// Helper methods for thumbnailDataForAsset:maxPixelSize:
static size_t getAssetBytesCallback(void *info, void *buffer, off_t position, size_t count) {
    ALAssetRepresentation *rep = (__bridge id)info;

    NSError *error = nil;
    size_t countRead = [rep getBytes:(uint8_t *)buffer fromOffset:position length:count error:&error];

    if (countRead == 0 && error) {
        // We have no way of passing this info back to the caller, so we log it, at least.
        NSLog(@"thumbnailForAsset:maxPixelSize: got an error reading an asset: %@", error);
    }

    return countRead;
}

static void releaseAssetCallback(void *info) {
    // The info here is an ALAssetRepresentation which we CFRetain in thumbnailDataForAsset:maxPixelSize:.
    // This release balances that retain.
    CFRelease(info);
}

Note that I started using these after I experienced a similar memory leak using UIImageJPEGRepresentation() on a method that turned the CGImageRef into a UIImageinstead.

I'm pretty sure that there's a memory leak because the app eventually terminates due to a memory error. When I run instruments on it, there are tons of living malloc 464/592/398 KB allocations, which instruments gives this stack for:

   0 libsystem_malloc.dylib malloc_zone_realloc
   1 libsystem_malloc.dylib realloc
   2 Foundation _NSMutableDataGrowBytes
   3 Foundation -[NSConcreteMutableData appendBytes:length:]
   4 ImageIO CGImageWriteSessionPutBytes
   5 ImageIO emptyOutputBuffer
   6 ImageIO encode_mcu_huff
   7 ImageIO compress_data
   8 ImageIO process_data_simple_main
   9 ImageIO _cg_jpeg_write_scanlines
  10 ImageIO writeOne
  11 ImageIO _CGImagePluginWriteJPEG
  12 ImageIO CGImageDestinationFinalize
  13 MyApp -[MyPhoto thumbnailDataForAsset:maxPixelSize:compressionQuality:] 
  14 MyApp -[MyPhoto imageDataResizedToFitSize:compressionQuality:] 
  15 MyApp -[MyPhoto scaledImageDataWithSmallerDimension:compressionQuality:] 

Is this a bug in CGImageDestinationFinalize or am I responsible for freeing something I'm not?

Thanks in advance.

Answer 1

I looked over your code pretty carefully and the memory management appears sound. You have the key bits enclosed in @autoreleasepool blocks, so any autorleased objects should be cleaned up. Your handling of CF objects also looks correct. (Admittedly memory bugs can be hard to spot...)

If you get a similar sort of leak when using the system method UIImageJPEGRepresentation, it starts to smell more and more like a system bug. Have you tested it against different OS versions (on device). It might be that this leak is new to iOS 7, and does not show up in older iOS versions. If so, it's important to make Apple aware of the bug quickly so they actually fix it in 7, rather than ignoring it until iOS 8 is released.

And have you tested it on device as well as on the sim? I have, more than once in the past, spent a lot of time trying to track down apparent memory leaks when running my code on the sim, only to conclude that the bug was actually in the simulator implementation of system libraries, and the exact same code did not leak when run on a device.

I would suggest submitting 2 bugs to Apple's bug reporter system: The leak in your UIImageJPEGRepresentation approach, as well as the leak in the CGImageSourceCreateThumbnailAtIndex approach. Provide minimal test applications in both cases, or Apple's wonderful (read clueless) bug screeners will kick it back to you immediately.

Once the front-line bug screeners run out of stupid reasons to kick it back to you, they should submit it to the real engineers, who can determine if it really is a framework bug or not. (Can you tell I have a low opinion of the first-level bug screeners at Apple?)

Answer 2

Why do you need to use so much complex things when the framework already provides one. The ImageIO frameworks has some functions which lets you create the thumbnails from the source image.

CGImageSourceRef src = CGImageSourceCreateWithURL(url);
NSDictionary *options = (CFDictionaryRef)[NSDictionary 
dictionaryWithObject:[NSNumber numberWithInt:1024
forKey:(id)kCGImageSourceThumbnailMaxPixelSize];

CGImageRef thumbnail = CGImageSourceCreateThumbnailAtIndex(src,0,options);
UIImage *image = [UIImage imageWithCGImage:thumbnail];
CGImageRelease(thumbnail);
CGImageSourceRelease(src);

Answer 3

I encountered a similar problem and after trying various methods and solutions, I tried also using autoreleasepool and it did't help... and after a while I came across the following insights:

When dealing with images using UIImage and CGImage system methods, it consumes a significant amount of memory. For instance, a simple image with dimensions around 4000px x 3000px can take up approximately 45MB of memory. If your app needs to handle multiple images or if you're working with a share extension, it's possible for your app to exhaust all available memory and crash. Share extensions have a limited memory allocation of only 120MB, other extension even have even less memory, so working with larger images (above 7000px in height/width) can easily lead to app crashes. This issue has even affected some of the most popular apps on iOS.

So, there is no straightforward solution other than avoiding the handling of large image files or using third-party tools.

One such open-source library that addresses this problem, (and which I used to solve this problem) is SDWebImage, available at https://github.com/SDWebImage/SDWebImage. After importing this library into your project, you can use it to handle images without worrying about memory leaks or crashes.

Here's an example of how you can use SDWebImage in your project to resize or compress image:

func resizeImage(atPath imagePath: URL) -> UIImage? {
    guard let image = UIImage(contentsOfFile: imagePath.path) else { return nil }
    
    guard let thumbnailData = SDImageIOCoder.shared.encodedData(with: image, format: .undefined, options: [.encodeMaxPixelSize: CGSize(width: 1920, height: 1920), .encodeCompressionQuality: 0.7]) else { return nil }
    
    return SDImageIOCoder.shared.decodedImage(with: thumbnailData)
}