How to correctly propagate delete from main thread's NSManagedObjectContext to child context on a background thread?

I’m trying to figure out how to solve the following situation

  1. There’s a main thread NSManagedObjectContext with NSMainQueueConcurrencyType. It spawns several background threads giving them the NSManagedObjectID of some object they will work on.
  2. Background threads perform some work (e.g. send the object data to the server, receive response and update the object accordingly). Threads use child contexts with NSConfinenmentConcurrencyType
  3. Meanwhile the user deletes the object from main thread’s context (via UI).
  4. Background contexts should be notified about this and handle the situation to prevent ‘cannot fulfil a fault’ exception on background context save.

I thought the main context (some custom object that manages it) could keep the record of the object ids that were deleted during background thread lifetime (or, more precisely, between creating background context and final save of the background context). Then the background context would have to perform deleteObject: on these objects just before it saves. And everything will go smoothly.

  • iOS5 NSManagedObjectContext Concurrency types and how are they used?
  • What is NSManagedObjectContext's performBlock: used for?
  • Core Data not saving transformable NSMutableDictionary
  • Undoing Core Data insertions that are performed off the main thread
  • Error creating a separate NSManagedObjectContext
  • Core Data Multithreading: Code Examples
  • In order to be sure that main context does not manage to delete the object when background thread finished deleting the objects and is about to call save: on its context, and to guarantee that main context’s delete does not happen after the child context is created but before child thread registers itself to be “notified” about deleted objects, I employed several mutex locks and came up with the following proof-of-concept code:

    @property (nonatomic, strong) id deleteLock;
    @property (nonatomic, strong) NSMutableDictionary *deletedObjectIdsPerThreadLifetime;
    
    - (void)coreDataDeleteSyncExample {
    
        static int lastThreadNo = 0;
    
        self.deleteLock = [[NSObject alloc] init];
        self.deletedObjectIdsPerThreadLifetime = [[NSMutableDictionary alloc] init];
    
        // main context is created using NSMainQueueConcurrencyType
        NSManagedObjectContext *mainContext = [self mainContext];
    
        NSManagedObjectID *myObjectId = nil;
    
        // creating the Object
        Order *order = (Order*)[NSEntityDescription
                                insertNewObjectForEntityForName:@"Order"
                                inManagedObjectContext:mainContext];
    
        Payment *payment = (Payment*)[NSEntityDescription
                                      insertNewObjectForEntityForName:@"Payment"
                                      inManagedObjectContext:mainContext];
    
    
        if (order) {
            [payment setOrder:order];
            [payment setAmount:[NSDecimalNumber decimalNumberWithString:@"103"]];
    
            NSError *error = nil;
            if (![mainContext save:&error]) {
                NSLog(@"main context save failed");
            }
    
            myObjectId = [order objectID]; // so I have non-temporary objectId here that I can pass around
        }
    
        int threadNo;
    
        for (threadNo = lastThreadNo ; threadNo < 50+lastThreadNo; threadNo++) {
            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
    
                NSNumber *threadNumber = [NSNumber numberWithInt:threadNo];
                NSManagedObjectContext *bckContext = nil;
                NSError *error = nil;
    
                @synchronized(self.deleteLock) {
                    bckContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
                    bckContext.parentContext = mainContext;
    
                    [self.deletedObjectIdsPerThreadLifetime setObject:[NSMutableSet set] forKey:threadNumber];
                    NSLog(@"Bck #%d created delete list/dict", threadNo);
                }
    
                Order *order = (Order*)[bckContext existingObjectWithID:myObjectId error:&error];
    
                for (int i = 0; i < 30; i++) {
                    order.status = [NSString stringWithFormat:@"some status set by background thread, %d/%d", threadNo, i];
                    NSLog(@"(dont clutter log):%d/%@", threadNo, order.status);
                }
    
                // background context now is going to save the order, but before that it deletes
                // from it all the objects that have been deleted from the main context in the meantime
    
                // we make it @synchronized call to make sure mainContext has no chances to delete
                // additional objects after we delete the ones from the set
                // and before we save background
                NSLog(@"Bck #%d saving context...", threadNo);
                @synchronized(self.deleteLock) {
                    NSSet *objsToDelete = [self.deletedObjectIdsPerThreadLifetime objectForKey:threadNumber];
                    for (NSManagedObjectID *objectId in objsToDelete) {
                        NSManagedObject *obj = [bckContext objectWithID:objectId];
                        NSLog(@"Bck #%d deleted obj %@ because it was on the list", threadNo,objectId);
                        [bckContext deleteObject:obj];
                    }
    
                    if (objsToDelete == nil) {
                        NSLog(@"Bck #%d is NOT included in delete dictionary list.", threadNo);
                    } else {
                        NSLog(@"Bck #%d has empty list of objs to delete.", threadNo);
                    }
    
    
    
                    NSLog(@"Bck #%d JUST before save...", threadNo);
                    // saving bck outside the lock is wrong
                    error = nil;
                    if (![bckContext save:&error]) {
                        NSLog(@"Bck context #%d failed to save: %@", threadNo, error);
                    } else {
                        NSLog(@"Bck #%d saved its context!", threadNo);
                    }
                }
    
                // saving main context outside the lock
                [mainContext performBlockAndWait:^{
                    NSError *error = nil;
                    NSLog(@"Main thread will save context (requested by Bck #%d)", threadNo);
                    if (![mainContext save:&error]) {
                        NSLog(@"main context save failed");
                    } else {
                        NSLog(@"main context saved (requested by bck #%d)", threadNo);
                    }
                }];
            });
        }
    
        lastThreadNo = threadNo;
    
        // now let's delete that object in the meantime on the main thread, and save the main context after a while
        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(150 * NSEC_PER_MSEC)), dispatch_get_main_queue(), ^{
    
            Order *o = (Order*)[mainContext objectWithID:myObjectId];
            NSLog(@"Main - will delete...");
    
            //@synchronized(self.deleteLock) {
            objc_sync_enter(self.deleteLock);
                for (NSNumber *threadNumber in self.deletedObjectIdsPerThreadLifetime) {
                    NSMutableSet *deletedIds = [self.deletedObjectIdsPerThreadLifetime objectForKey:threadNumber];
    
                    [deletedIds addObject:myObjectId];
                }
                NSLog(@"Main -deleting- %@", myObjectId);
                [mainContext deleteObject:o];
                NSLog(@"Main -deleted- %@", myObjectId);
    
            //}
    
            dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(150 * NSEC_PER_MSEC)), dispatch_get_main_queue(), ^{
    
                DLog(@"AND NOW WE SAVE MAIN!");
                NSError *error = nil;
                if (![mainContext save:&error]) {
                    NSLog(@"main context save failed");
                } else {
                    NSLog(@"main context saved (requested by main context)");
                }
    
                objc_sync_exit(self.deleteLock);
            });
    
        });
    
    
    }
    

    It turns out that the code has several problems:
    1. It deadlocks. When background threads starts the save “transactions” it acquires the lock, then if mainThread manages to encounter the @synchronized block it waits. Background then proceeds to its save: call. There it seems that CoreData wants to save the child to main context, so it tries to use that context. Since it can be used only on main thread, and main thread is blocked by the lock acquired by background thread, we have a deadlock.
    2. It still crashes with ‘cannot fulfil a fault’. It happens only sometimes when main context’s delete and save happens just before background context is created and fetches the object. Normally in this situation that object is nil. But sometimes it is not (why???) and we got a crash on background context save, like in this situation:

    2014-11-27 14:00:13.179 ConcurrentCoreData[70490:1403] Bck #0 created delete list/dict
    2014-11-27 14:00:13.186 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/0
    2014-11-27 14:00:13.187 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/1
    2014-11-27 14:00:13.189 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/2
    2014-11-27 14:00:13.189 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/3
    2014-11-27 14:00:13.190 ConcurrentCoreData[70490:2c07] Bck #1 created delete list/dict
    2014-11-27 14:00:13.190 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/4
    2014-11-27 14:00:13.192 ConcurrentCoreData[70490:3907] Bck #2 created delete list/dict
    2014-11-27 14:00:13.191 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/5
    (...)
    2014-11-27 14:00:13.309 ConcurrentCoreData[70490:4b03] (dont clutter log):7/some status set by background thread, 7/10
    2014-11-27 14:00:13.309 ConcurrentCoreData[70490:2c07] (dont clutter log):1/some status set by background thread, 1/23
    2014-11-27 14:00:13.311 ConcurrentCoreData[70490:1403] (dont clutter log):0/some status set by background thread, 0/29
    2014-11-27 14:00:13.329 ConcurrentCoreData[70490:90b] Main - will delete...
    2014-11-27 14:00:13.333 ConcurrentCoreData[70490:4e03] Bck #8 created delete list/dict
    2014-11-27 14:00:13.316 ConcurrentCoreData[70490:4b03] (dont clutter log):7/some status set by background thread, 7/11
    2014-11-27 14:00:13.365 ConcurrentCoreData[70490:5003] Bck #9 created delete list/dict
    2014-11-27 14:00:13.367 ConcurrentCoreData[70490:5103] Bck #10 created delete list/dict
    2014-11-27 14:00:13.367 ConcurrentCoreData[70490:5203] Bck #11 created delete list/dict
    2014-11-27 14:00:13.366 ConcurrentCoreData[70490:4b03] (dont clutter log):7/some status set by background thread, 7/12
    2014-11-27 14:00:13.316 ConcurrentCoreData[70490:2c07] (dont clutter log):1/some status set by background thread, 1/24
    2014-11-27 14:00:13.311 ConcurrentCoreData[70490:3807] (dont clutter log):3/some status set by background thread, 3/20
    2014-11-27 14:00:13.312 ConcurrentCoreData[70490:3b03] (dont clutter log):4/some status set by background thread, 4/19
    2014-11-27 14:00:13.316 ConcurrentCoreData[70490:3c03] (dont clutter log):5/some status set by background thread, 5/18
    2014-11-27 14:00:13.314 ConcurrentCoreData[70490:3907] (dont clutter log):2/some status set by background thread, 2/22
    2014-11-27 14:00:13.312 ConcurrentCoreData[70490:4603] (dont clutter log):6/some status set by background thread, 6/17
    2014-11-27 14:00:13.365 ConcurrentCoreData[70490:1403] Bck #0 saving context...
    2014-11-27 14:00:13.369 ConcurrentCoreData[70490:90b] Main -deleting- 0x8b24cd0 <x-coredata://06DFA035-E3DF-497C-89B4-20E845A09712/Order/p549>
    (...)
    2014-11-27 14:00:13.372 ConcurrentCoreData[70490:90b] Main -deleted- 0x8b24cd0 <x-coredata://06DFA035-E3DF-497C-89B4-20E845A09712/Order/p549>
    (...)
    2014-11-27 14:00:13.420 ConcurrentCoreData[70490:2c07] Bck #1 saving context...
    (...)
    2014-11-27 14:00:13.453 ConcurrentCoreData[70490:3907] Bck #2 saving context...
    (...)
    2014-11-27 14:00:13.475 ConcurrentCoreData[70490:3807] Bck #3 saving context...
    (...)
    2014-11-27 14:00:13.488 ConcurrentCoreData[70490:3b03] Bck #4 saving context...
    (...)
    2014-11-27 14:00:13.496 ConcurrentCoreData[70490:3c03] Bck #5 saving context...
    (...)
    2014-11-27 14:00:13.558 ConcurrentCoreData[70490:90b] __43-[ViewController coreDataDeleteSyncExample]_block_invoke_2178 [Line 260] AND NOW WE SAVE MAIN!
    2014-11-27 14:00:13.559 ConcurrentCoreData[70490:4603] (dont clutter log):6/some status set by background thread, 6/28
    (...)
    2014-11-27 14:00:13.564 ConcurrentCoreData[70490:4e03] (dont clutter log):8/some status set by background thread, 8/13
    2014-11-27 14:00:13.565 ConcurrentCoreData[70490:90b] main context saved (requested by main context)
    2014-11-27 14:00:13.565 ConcurrentCoreData[70490:4603] (dont clutter log):6/some status set by background thread, 6/29
    2014-11-27 14:00:13.566 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/13
    (...)
    2014-11-27 14:00:13.663 ConcurrentCoreData[70490:2c07] Bck #1 saved its context!
    2014-11-27 14:00:13.664 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/24
    2014-11-27 14:00:13.667 ConcurrentCoreData[70490:90b] Main thread will save context (requested by Bck #1)
    2014-11-27 14:00:13.667 ConcurrentCoreData[70490:90b] main context saved (requested by bck #1)
    2014-11-27 14:00:13.667 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/25
    2014-11-27 14:00:13.668 ConcurrentCoreData[70490:3907] Bck #2 deleted obj 0x8b24cd0 <x-coredata://06DFA035-E3DF-497C-89B4-20E845A09712/Order/p549> because it was on the list
    2014-11-27 14:00:13.668 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/26
    2014-11-27 14:00:13.668 ConcurrentCoreData[70490:3907] Bck #2 has empty list of objs to delete.
    2014-11-27 14:00:13.668 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/27
    2014-11-27 14:00:13.669 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/28
    2014-11-27 14:00:13.669 ConcurrentCoreData[70490:5003] (dont clutter log):9/some status set by background thread, 9/29
    2014-11-27 14:00:13.670 ConcurrentCoreData[70490:5003] Bck #9 saving context...
    2014-11-27 14:00:13.668 ConcurrentCoreData[70490:3907] Bck #2 JUST before save...
    2014-11-27 14:00:13.666 ConcurrentCoreData[70490:4e03] (dont clutter log):8/some status set by background thread, 8/18
    2014-11-27 14:00:13.671 ConcurrentCoreData[70490:3907] Bck #2 saved its context!
    2014-11-27 14:00:13.671 ConcurrentCoreData[70490:5a03] Bck #12 created delete list/dict
    2014-11-27 14:00:13.672 ConcurrentCoreData[70490:5b03] Bck #13 created delete list/dict
    2014-11-27 14:00:13.672 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/0
    2014-11-27 14:00:13.673 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/1
    (!! and here queue #13 HAS the object! But it was deleted from main context and the main context was saved before we spawned that child context!)
    2014-11-27 14:00:13.673 ConcurrentCoreData[70490:5b03] (dont clutter log):13/some status set by background thread, 13/0 
    2014-11-27 14:00:13.674 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/2
    2014-11-27 14:00:13.674 ConcurrentCoreData[70490:5b03] (dont clutter log):13/some status set by background thread, 13/1
    2014-11-27 14:00:13.674 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/3
    2014-11-27 14:00:13.674 ConcurrentCoreData[70490:5b03] (dont clutter log):13/some status set by background thread, 13/2
    2014-11-27 14:00:13.673 ConcurrentCoreData[70490:5c03] Bck #14 created delete list/dict
    2014-11-27 14:00:13.675 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/4
    2014-11-27 14:00:13.675 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/5
    2014-11-27 14:00:13.676 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/6
    2014-11-27 14:00:13.676 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/7
    2014-11-27 14:00:13.676 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/8
    2014-11-27 14:00:13.675 ConcurrentCoreData[70490:5b03] (dont clutter log):13/some status set by background thread, 13/3
    2014-11-27 14:00:13.676 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/9
    2014-11-27 14:00:13.677 ConcurrentCoreData[70490:5a03] (dont clutter log):12/some status set by background thread, 12/10
    (!!! QUEUE #14 AS WELL???)
    2014-11-27 14:00:13.675 ConcurrentCoreData[70490:5c03] (dont clutter log):14/some status set by background thread, 14/0
    (…)
    2014-11-27 14:00:14.503 ConcurrentCoreData[70490:7a03] (dont clutter log):44/(null)
    2014-11-27 14:00:14.504 ConcurrentCoreData[70490:7f03] (dont clutter log):49/(null)
    2014-11-27 14:00:14.505 ConcurrentCoreData[70490:90b] Main thread will save context (requested by Bck #9)
    2014-11-27 14:00:14.505 ConcurrentCoreData[70490:7e03] (dont clutter log):48/(null)
    2014-11-27 14:00:14.505 ConcurrentCoreData[70490:7b03] (dont clutter log):45/(null)
    2014-11-27 14:00:14.505 ConcurrentCoreData[70490:7d03] (dont clutter log):47/(null)
    2014-11-27 14:00:14.505 ConcurrentCoreData[70490:5b03] Bck #13 has empty list of objs to delete.
    2014-11-27 14:00:14.506 ConcurrentCoreData[70490:7903] (dont clutter log):43/(null)
    2014-11-27 14:00:14.506 ConcurrentCoreData[70490:7c03] (dont clutter log):46/(null)
    2014-11-27 14:00:14.509 ConcurrentCoreData[70490:90b] main context saved (requested by bck #9)
    2014-11-27 14:00:14.508 ConcurrentCoreData[70490:7a03] Bck #44 saving context...
    2014-11-27 14:00:14.510 ConcurrentCoreData[70490:7e03] (dont clutter log):48/(null)
    2014-11-27 14:00:14.510 ConcurrentCoreData[70490:7b03] (dont clutter log):45/(null)
    (QUEUE #13 tries to save and it crashes!)
    2014-11-27 14:00:14.510 ConcurrentCoreData[70490:5b03] Bck #13 JUST before save...
    2014-11-27 14:00:14.510 ConcurrentCoreData[70490:7d03] (dont clutter log):47/(null)
    2014-11-27 14:00:14.508 ConcurrentCoreData[70490:7f03] (dont clutter log):49/(null)
    2014-11-27 14:00:14.511 ConcurrentCoreData[70490:7903] Bck #43 saving context...
    2014-11-27 14:00:14.511 ConcurrentCoreData[70490:7c03] (dont clutter log):46/(null)
    2014-11-27 14:00:14.514 ConcurrentCoreData[70490:7e03] (dont clutter log):48/(null)
    2014-11-27 14:00:14.516 ConcurrentCoreData[70490:90b] *** Terminating app due to uncaught exception 'NSObjectInaccessibleException', reason: 'CoreData could not fulfill a fault for '0x8b24cd0 <x-coredata://06DFA035-E3DF-497C-89B4-20E845A09712/Order/p549>''
    *** First throw call stack:
    (
        0   CoreFoundation                      0x018001e4 __exceptionPreprocess + 180
        1   libobjc.A.dylib                     0x0157f8e5 objc_exception_throw + 44
        2   CoreData                            0x01a8cbeb _PFFaultHandlerLookupRow + 2715
        3   CoreData                            0x01abee88 -[NSFaultHandler fulfillFault:withContext:] + 40
        4   CoreData                            0x01b33169 -[NSManagedObject(_NSInternalMethods) _updateFromRefreshSnapshot:includingTransients:] + 265
        5   CoreData                            0x01ac7902 -[NSManagedObjectContext(_NestedContextSupport) _copyChildObject:toParentObject:fromChildContext:] + 994
        6   CoreData                            0x01ac71e8 -[NSManagedObjectContext(_NestedContextSupport) _parentProcessSaveRequest:inContext:error:] + 1480
        7   CoreData                            0x01b3fa14 __82-[NSManagedObjectContext(_NestedContextSupport) executeRequest:withContext:error:]_block_invoke + 676
        8   CoreData                            0x01ac1b81 internalBlockToNSManagedObjectContextPerform + 17
        9   libdispatch.dylib                   0x01f784d0 _dispatch_client_callout + 14
        10  libdispatch.dylib                   0x01f67439 _dispatch_barrier_sync_f_slow_invoke + 80
        11  libdispatch.dylib                   0x01f784d0 _dispatch_client_callout + 14
        12  libdispatch.dylib                   0x01f66726 _dispatch_main_queue_callback_4CF + 340
        13  CoreFoundation                      0x0186543e __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 14
        14  CoreFoundation                      0x017a65cb __CFRunLoopRun + 1963
        15  CoreFoundation                      0x017a59d3 CFRunLoopRunSpecific + 467
        16  CoreFoundation                      0x017a57eb CFRunLoopRunInMode + 123
        17  GraphicsServices                    0x03b0a5ee GSEventRunModal + 192
        18  GraphicsServices                    0x03b0a42b GSEventRun + 104
        19  UIKit                               0x0023ff9b UIApplicationMain + 1225
        20  ConcurrentCoreData                  0x00009d7d main + 141
        21  libdyld.dylib                       0x021ab725 start + 0
    )
    libc++abi.dylib: terminating with uncaught exception of type _NSCoreDataException
    (lldb) 
    

    I understand the cause of the first problem (deadlock). I have no idea how to solve it, I guess custom locking like this is not possible when using main context’s child contexts.

    But the second one is really weird. Why the object is not nil? After all Core Data did delete and save the object before child context was created. Why usually I get nil there but sometimes I get the object? Is it some caching issue? Can I not trust Core Data on returning nil in child context for the object that was deleted in main context (and saved!) before the child context was even created?! Is my solution fundamentally flawed?

    What is the correct way to handle this situation with background contexts having to deal with main context’s deletes. I’ve got a feeling that this whole main/child context feature is really nice and easy to use, unless you start to delete objects in main contexts. Then this whole thing becames useless and we still have to resort to saving to the store and merging contexts.