Core Data NSPersistentStoreCoordinator's +metadataForPersistentStoreOfType:URL:error: sometimes returns nil

Question

I have a method that progressively migrates a core data sqlite store though multiple NSManagedObjectModel versions until the store is at a the current version. The method is inspired by the code in Marcus Zarra's Core Data book.

The app is in production but my method is failing in about 0.5% of cases. When it fails it returns NO and an error is logged using Crashlytics:

NSSQLiteErrorDomain = 14; NSUnderlyingException = "I/O error for database at <store url>. SQLite error code:14, 'unable to open database file'"

The sqlite store use's write-ahead logging (WAL) and I am able to sometimes get the same error if I call +metadataForPersistentStoreOfType:URL:error: after first manually deleting the sqlite-WAL file.

Progressive Migration Code:

- (BOOL)progressivelyMigrateURL:(NSURL*)sourceStoreURL 
                        toModel:(NSManagedObjectModel*)finalModel 
                          error:(NSError**)error
{
    NSURL *storeDirectoryURL = [sourceStoreURL URLByDeletingLastPathComponent];
    NSString *storeExtension = [sourceStoreURL pathExtension];
    
    NSDictionary *sourceMetadata = [NSPersistentStoreCoordinator 
        metadataForPersistentStoreOfType:NSSQLiteStoreType 
                                     URL:sourceStoreURL 
                                   error:error];
    if (!sourceMetadata) return NO;
    
    while (![finalModel isConfiguration:nil 
            compatibleWithStoreMetadata:sourceMetadata]) {
        
        NSManagedObjectModel *sourceModel = 
            [self managedObjectModelForMetadata:sourceMetadata];
        if (!sourceModel) return NO;
        
        NSString *modelName = nil;
        NSManagedObjectModel *targetModel = 
            [self suitableTargetModelForMigrationFromSourceModel:sourceModel 
                                                       modelName:&modelName];
        if (!targetModel) return NO;
        
        NSMigrationManager *manager = 
            [[NSMigrationManager alloc] initWithSourceModel:sourceModel 
                                           destinationModel:targetModel];
        NSMappingModel *mappingModel = 
            [NSMappingModel mappingModelFromBundles:nil 
                                     forSourceModel:sourceModel 
                                   destinationModel:targetModel];
        NSURL *destinationStoreURL = 
            [[storeDirectoryURL URLByAppendingPathComponent:modelName] 
                URLByAppendingPathExtension:storeExtension];
        
        BOOL migrated = [manager migrateStoreFromURL:sourceStoreURL 
                                                type:NSSQLiteStoreType 
                                             options:nil 
                                    withMappingModel:mappingModel 
                                    toDestinationURL:destinationStoreURL 
                                     destinationType:NSSQLiteStoreType 
                                  destinationOptions:nil 
                                               error:error];
        if (!migrated) return NO;
        
        NSString *sourceModelName = 
            [self versionStringForManagedObjectModel:sourceModel];
        NSURL *backUpURL = [self backupURLWithDirectoryURL:storeDirectoryURL 
                                             pathExtension:storeExtension 
                                                 modelName:sourceModelName];
        BOOL replaced = [self replaceStoreAtURL:sourceStoreURL 
                                 withStoreAtURL:destinationStoreURL 
                                      backupURL:backUpURL 
                                          error:error];
        if (replaced == NO) return NO;
        
        sourceMetadata = [NSPersistentStoreCoordinator 
            metadataForPersistentStoreOfType:NSSQLiteStoreType 
                                         URL:sourceStoreURL 
                                       error:error];
        if (!sourceMetadata) return NO;
    }
    
    return YES;
}

- (NSManagedObjectModel *)managedObjectModelForMetadata:(NSDictionary *)metadata
{
    for (NSURL *URL in [self modelURLs]) {
        NSManagedObjectModel *model = 
            [[NSManagedObjectModel alloc] initWithContentsOfURL:URL];
        if ([model isConfiguration:nil compatibleWithStoreMetadata:metadata]) {
            return model;
        }
    }
    return nil;
}

- (NSManagedObjectModel *)suitableTargetModelForMigrationFromSourceModel:(NSManagedObjectModel *)sourceModel 
                                                               modelName:(NSString **)modelName
{
    for (NSURL *modelURL in [self modelURLs]) {
        NSManagedObjectModel *targetModel = 
            [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL];
        NSMappingModel *mappingModel = 
            [NSMappingModel mappingModelFromBundles:nil 
                                     forSourceModel:sourceModel 
                                   destinationModel:targetModel];
        
        if (mappingModel) {
            *modelName = [[modelURL lastPathComponent] stringByDeletingPathExtension];
            return targetModel;
        }
    }
    
    return nil;
}

- (NSArray *)modelURLs
{
    NSMutableArray *modelURLs = 
        [[[NSBundle mainBundle] URLsForResourcesWithExtension:@"mom" 
                                                 subdirectory:nil] mutableCopy];
    
    NSArray *momdURLs = 
        [[[NSBundle mainBundle] URLsForResourcesWithExtension:@"momd" 
                                                 subdirectory:nil] mutableCopy];
    for (NSURL *momdURL in momdURLs) {
        NSString *directory = [momdURL lastPathComponent];
        NSArray *array = 
            [[NSBundle mainBundle] URLsForResourcesWithExtension:@"mom" 
                                                    subdirectory:directory];
        [modelURLs addObjectsFromArray:array];
    }
    
    return [modelURLs copy];
}

- (NSURL *)backupURLWithDirectoryURL:(NSURL *)URL 
                       pathExtension:(NSString *)extension 
                           modelName:(NSString *)name
{
    NSString *GUID = [[NSProcessInfo processInfo] globallyUniqueString];
    NSString *pathComponant = [NSString stringWithFormat:@"%@-%@", GUID, name];
    
    return [[URL URLByAppendingPathComponent:pathComponant] 
        URLByAppendingPathExtension:extension];
}

- (BOOL)replaceStoreAtURL:(NSURL *)originalStoreURL 
           withStoreAtURL:(NSURL *)newStoreURL 
                backupURL:(NSURL *)backupURL 
                    error:(NSError **)error
{
    BOOL storeMoved = [self moveStoreAtURL:originalStoreURL 
                                     toURL:backupURL 
                                     error:error];
    if (!storeMoved) return NO;
    
    storeMoved = [self moveStoreAtURL:newStoreURL 
                                toURL:originalStoreURL 
                                error:error];
    if (!storeMoved) return NO;
    
    return YES;
}

- (BOOL)moveStoreAtURL:(NSURL *)sourceURL 
                 toURL:(NSURL *)targetURL 
                 error:(NSError **)error
{
    NSMutableArray *sourceURLs = [@[sourceURL] mutableCopy];
    NSMutableArray *targetURLs = [@[targetURL] mutableCopy];
    
    NSString *walExtension = @"sqlite-wal";
    if ([self storeAtURL:sourceURL hasAccessoryFileWithExtension:walExtension]) {
        [sourceURLs addObject:[self URLByReplacingExtensionOfURL:sourceURL 
                                                   withExtension:walExtension]];
        [targetURLs addObject:[self URLByReplacingExtensionOfURL:targetURL 
                                                   withExtension:walExtension]];
    }
    
    NSString *shmExtension = @"sqlite-shm";
    if ([self storeAtURL:sourceURL hasAccessoryFileWithExtension:shmExtension]) {
        [sourceURLs addObject:[self URLByReplacingExtensionOfURL:sourceURL 
                                                   withExtension:shmExtension]];
        [targetURLs addObject:[self URLByReplacingExtensionOfURL:targetURL 
                                                   withExtension:shmExtension]];
    }
    
    NSFileManager *fileManager = [NSFileManager defaultManager];
    for (int i = 0; i < [sourceURLs count]; i++) {
        BOOL fileMoved = [fileManager moveItemAtURL:sourceURLs[i] 
                                              toURL:targetURLs[i] 
                                              error:error];
        if (!fileMoved) return NO;
    }
    
    return YES;
}

- (BOOL)storeAtURL:(NSURL *)URL hasAccessoryFileWithExtension:(NSString *)extension
{
    NSURL *accessoryURL = [self URLByReplacingExtensionOfURL:URL 
                                               withExtension:extension];
    return [[NSFileManager defaultManager] fileExistsAtPath:[accessoryURL path]];
}

- (NSURL *)URLByReplacingExtensionOfURL:(NSURL *)URL withExtension:(NSString *)extension
{
    return [[URL URLByDeletingPathExtension] URLByAppendingPathExtension:extension];
}

- (NSString *)versionStringForManagedObjectModel:(NSManagedObjectModel *)model
{
    NSString *string = @"";
    for (NSString *identifier in model.versionIdentifiers) {
        string = [string stringByAppendingString:identifier];
    }
    return string;
}

Sorry for the super long code.

Answer 1

The likely cause is your moveStoreAtURL:toURL:error: method. The error you're getting is mentioned in Apple's docs as being the result of failing to copy all of a persistent store's files. It looks like you're trying to hit all of them, but either (a) there's a bug in the copy code that I can't find right now or (b) the store is "live" in your app, being used by a persistent store coordinator, and so you're not getting a consistent state from the copy.

You might be able to fix it with some debugging, and if you ensured that the store was not in use. It would be better and probably more reliable to change the journal mode so that you don't have wal and shm files (which that link also describes). Even better, if your store files aren't too huge, use migratePersistentStore:toURL:options:withType:error to have Core Data make the copy. That should be pretty much guaranteed to work, though in some cases it can use too much memory.

Answer 2

I'm using NSMigrationManager, so I can't use NSPersistentStoreCoordinator's - migratePersistentStore..., so my solution was to force a checkpoint operation:

- (void)performCheckpointStoreWithSourceModel:(NSManagedObjectModel *)sourceModel sourceStoreURL:(NSURL *)sourceStoreURL {
    NSPersistentStoreCoordinator *tempPSC = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:sourceModel];
    [tempPSC addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:sourceStoreURL options:@{NSSQLitePragmasOption: @{@"journal_mode": @"DELETE"}} error:nil];
    [tempPSC removePersistentStore:[tempPSC persistentStoreForURL:sourceStoreURL] error:nil];
}

...before performing migration with NSMigrationManager:

if (![manager migrateStoreFromURL:sourceStoreURL
                             type:type
                          options:nil
                 withMappingModel:mappingModel
                 toDestinationURL:destinationStoreURL
                  destinationType:type
               destinationOptions:nil
                            error:error]) {
    return NO;
}