ARC seems to overrelease objects referenced in blocks that are created and dispatched in a loop

Question

I am trying to put some complex computations on a background thread using dispatch_async but the objects I am using in the blocks seem to be overreleased. I am using ARC so I assumed that I do not have to care much about retain and release, but either I missed something important or ARC overreleases objects in my case.

The problem only appears if

• I call dispatch_async creating a block in a for loop
• I reference an object in the block created outside of the block
• the loop does at least two iterations (thus at least two blocks are created and added to the queue)
• a RELEASE build configuration is used (so it's probably related to some optimization)

It does not seem to matter

• whether it is a serial or concurrent queue
• what kind of object is used

This question is not about blocks being release in the RELEASE configuration (as in iOS 5 blocks crash only with Release Build), but the objects referenced in the block being overreleased.

I created a small example using an NSURL object:

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
    NSURL *theURL = [NSURL URLWithString:@"/Users/"];
    dispatch_queue_t myQueue = dispatch_queue_create("several.blocks.queue", DISPATCH_QUEUE_SERIAL);

    dispatch_async(myQueue, ^(){
        NSURL *newURL = [theURL URLByAppendingPathComponent:@"test"];
        NSLog(@"Successfully created new url: %@ in initial block", newURL);
    });

    for (int i = 0; i < 2; i++)
    {
        dispatch_async(myQueue, ^(){
            NSURL *newURL = [theURL URLByAppendingPathComponent:@"test"];
            NSLog(@"Successfully created new url: %@ in loop block %d", newURL, i);
        });
    }
}

The first block which is not in the for-loop will work without issues. As will the second if the loop has only one iteration. In the given example however it does two iterations and will crash if run with a RELEASE configuration. Enabling NSZombie in the scheme outputs this:

2013-01-07 23:33:33.331 BlocksAndARC[17185:1803] Successfully created new url: /Users/test in initial block
2013-01-07 23:33:33.333 BlocksAndARC[17185:1803] Successfully created new url: /Users/test in loop block 0
2013-01-07 23:33:33.333 BlocksAndARC[17185:1803] *** -[CFURL URLByAppendingPathComponent:]: message sent to deallocated instance 0x101c32790

with the debugger stopping at the URLByAppendingPathComponent call in the block in the for-loop.

When using a concurrent queue the failing call will actually be a release call with _Block_release in the call stack:

2013-01-07 23:36:13.291 BlocksAndARC[17230:5f03] *** -[CFURL release]: message sent to deallocated instance 0x10190dd30
(lldb) bt
* thread #6: tid = 0x3503, 0x00007fff885914ce CoreFoundation`___forwarding___ + 158, stop reason = EXC_BREAKPOINT (code=EXC_I386_BPT, subcode=0x0)
    frame #0: 0x00007fff885914ce CoreFoundation`___forwarding___ + 158
    frame #1: 0x00007fff885913b8 CoreFoundation`_CF_forwarding_prep_0 + 232
    frame #2: 0x00007fff808166a3 libsystem_blocks.dylib`_Block_release + 202
    frame #3: 0x00007fff89f330b6 libdispatch.dylib`_dispatch_client_callout + 8
    frame #4: 0x00007fff89f38317 libdispatch.dylib`_dispatch_async_f_redirect_invoke + 117
    frame #5: 0x00007fff89f330b6 libdispatch.dylib`_dispatch_client_callout + 8
    frame #6: 0x00007fff89f341fa libdispatch.dylib`_dispatch_worker_thread2 + 304
    frame #7: 0x00007fff852f0cab libsystem_c.dylib`_pthread_wqthread + 404
    frame #8: 0x00007fff852db171 libsystem_c.dylib`start_wqthread + 13

but this is probably just due to slightly different timing.

I think both errors indicate that the NSURL object referenced by theURL is overreleased. But why is that? Did I miss something or is that a bug in the combination of ARC and blocks?

What I would expect to happen is that either before the dispatch_async call or in the implementation of dispatch_async (anyway: inside the for-loop, once for each dispatch_async-call) every variable referenced inside the block is retained and it is released at the end of (but in) the block.

What actually seems to happen is that the variables are retained once for an occurrence of dispatch_async in the code but release is called at the end of the block so whenever it is executed, which leads to more release calls than retain calls in a loop.

But maybe I'm overlooking something. Is there a better explanation? Did I misuse blocks or ARC in some way or is this a bug?

EDIT: I tried @Joshua Weinberg's suggestion of copying the referenced variable to a local one inside the for-loop. It work's in the given sample code, but does not work, when a function call is involved:

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
    NSObject *theObject = [[NSObject alloc] init];

    [self blocksInForLoopWithObject:theObject];
}

-(void)blocksInForLoopWithObject:(NSObject *)theObject
{
    dispatch_queue_t myQueue = dispatch_queue_create("several.blocks.queue", DISPATCH_QUEUE_SERIAL);
    for (int i = 0; i < 2; i++)
    {
        NSObject *theSameObject = theObject;
        dispatch_async(myQueue, ^(){
            NSString *description = [theSameObject description];
            NSLog(@"Successfully referenced object %@ in loop block %d", description, i);
        });
    }
}

So why does it work in on case, but not in the other? I don't see the difference.

Answer 1

I was just able to reproduce this when I tried it out. Your diagnosis seems spot on and as far as I can tell is an issue with some optimization on how the blocks get copied/retain their scope going awry. Seems radar worthy.

As far as what you can do to work around this.

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
    NSURL *theURL = [NSURL URLWithString:@"/Users/"];
    dispatch_queue_t myQueue = dispatch_queue_create("several.blocks.queue", DISPATCH_QUEUE_SERIAL);

    dispatch_async(myQueue, ^(){
        NSURL *newURL = [theURL URLByAppendingPathComponent:@"test"];
        NSLog(@"Successfully created new url: %@ in initial block", newURL);
    });

    for (int i = 0; i < 2; i++)
    {
        NSURL *localURL = theURL;
        dispatch_async(myQueue, ^(){
            NSURL *newURL = [localURL URLByAppendingPathComponent:@"test"];
            NSLog(@"Successfully created new url: %@ in loop block %d", newURL, i);
        });
    }
}

Copying that to the stack forces the block to recapture it each time and enforces your intended memory semantics.

Answer 2

To help people trying to troubleshoot this problem, I was able to reproduce the problem with this simplified version on my XCode 4.5, Release configuration:

- (id)test {
  return [[NSObject alloc] init];
}

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {

  id foo = [self test];
  for (int i = 0; i < 2; i++)
  {
    [^(){
      NSLog(@"%@", foo);
    } copy];
  }
  NSLog(@"%@", foo);

  return YES;
}

From profiling it, it seems that ARC is incorrectly inserting a release at the end of the inside of the loop.