Memory issues when encrypting/decrypting a large file with RNCryptor on iOS

Question 1

I finally tried the solution given here, which uses semaphores instead of depending on the callback to wait for the stream. This works perfectly :) The memory usage hovers around 1.1 MB according to the Allocations Instrument. It may not look as neat because of the semaphore syntax, but at least it does what I need it to do.

Other suggestions are still welcome of course :)

- (void)encryptWithSemaphore:(NSURL *)url {
  dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);

  __block int total = 0;
  int blockSize = 32 * 1024;

  NSString *encryptedFile = [[url lastPathComponent] stringByDeletingPathExtension];
  NSURL *docsURL = [[NSFileManager defaultManager] URLForDirectory:NSDocumentDirectory inDomain:NSUserDomainMask appropriateForURL:nil create:NO error:nil];
  self.tempURL = [[docsURL URLByAppendingPathComponent:encryptedFile isDirectory:NO] URLByAppendingPathExtension:@"crypt"];

  NSInputStream *inputStream = [NSInputStream inputStreamWithURL:url];
  __block NSOutputStream *outputStream = [NSOutputStream outputStreamWithURL:self.tempURL append:NO];
  __block NSError *encryptionError = nil;

  [inputStream open];
  [outputStream open];

  RNEncryptor *encryptor = [[RNEncryptor alloc] initWithSettings:kRNCryptorAES256Settings
                                                        password:self.password
                                                         handler:^(RNCryptor *cryptor, NSData *data) {
                                                           @autoreleasepool {
                                                             [outputStream write:data.bytes maxLength:data.length];
                                                             dispatch_semaphore_signal(semaphore);

                                                             data = nil;
                                                             if (cryptor.isFinished) {
                                                               [outputStream close];
                                                               encryptionError = cryptor.error;
                                                               // call my delegate that I'm finished with decrypting
                                                             }
                                                           }
                                                         }];
  while (inputStream.hasBytesAvailable) {
    @autoreleasepool {
      uint8_t buf[blockSize];
      NSUInteger bytesRead = [inputStream read:buf maxLength:blockSize];
      if (bytesRead > 0) {
        NSData *data = [NSData dataWithBytes:buf length:bytesRead];

        total = total + bytesRead;
        [encryptor addData:data];

        dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
      }
    }
  }

  [inputStream close];
  [encryptor finish];  
}

Question 2

Just run:

self.cryptorQueue = dispatch_queue_create([queueName cStringUsingEncoding:NSUTF8StringEncoding], NULL);

dispatch_async(self.cryptorQueue, ^{
        readStreamBlock();
    });

trouble: stack is growing and autorelease pull will not execute release for buffer.

solution: add async in the same queue.. this will let current block to finish execution.

Here is the code:

__block NSMutableData *data = [NSMutableData dataWithLength:blockSize];
__block RNDecryptor *decryptor = nil;


dispatch_block_t readStreamBlock = ^{

    [data setLength:blockSize];

    NSInteger bytesRead = [inputStream read:[data mutableBytes] maxLength:blockSize];
    if (bytesRead < 0) {
        // Throw an error
    }
    else if (bytesRead == 0) {
        [decryptor finish];
    }
    else {

        [data setLength:bytesRead];
        [decryptor addData:data];
    }
};

decryptor = [[RNDecryptor alloc] initWithPassword:@"blah" handler:^(RNCryptor *cryptor, NSData *data) {

        [decryptedStream write:data.bytes maxLength:data.length];
        _percentStatus = (CGFloat)[[decryptedStream propertyForKey:NSStreamFileCurrentOffsetKey] intValue] / (CGFloat)_inputFileSize;
        if (cryptor.isFinished)
        {
            [decryptedStream close];
            [self decryptFinish];
        }
        else
        {
            dispatch_async(cryptor.responseQueue, ^{
                readStreamBlock();
            });
            [self decryptStatusChange];
        }

}];


// Read the first block to kick things off

decryptor.responseQueue = self.cryptorQueue;
[self decryptStart];
dispatch_async(decryptor.cryptorQueue, ^{
    readStreamBlock();
});

Question 3

I may be wrong, but I think your do...while loop prevents the autorelease pool from draining frequently enough.

Why are you using this loop to wait for the decryptor to finish? You should use the completion block to notify your controller that the drcryptor has finished, instead.

(BTW, welcome to SO, your question is really well asked and that's highly appreciated).

Question 4

I might again be wrong, but in your readStreamBlock, data should be a parameter to the block, rather than a reference to a __block NSMutableData declared outside of it. As you can see, the RNEncryptor handler provides its own data variable, which is different from the one you've declared yourself.

Ideally, put all your readStreamBlock directly inside the handler, without even declaring it a block.