What is the most efficient way to delete a large number (10.000+) objects in Core Data?

The way I’m trying to delete multiple sets of 10.000+ NSManagedObjects is just too memory intensive (around 20MB live bytes), and my app is being jettisoned. Here is the implementation of the delete method:

+ (void)deleteRelatedEntitiesInManagedObjectContext:(NSManagedObjectContext *)context 
{
    NSFetchRequest *fetch = [[NSFetchRequest alloc] init];
    [context setUndoManager:nil];

    [fetch setEntity:[NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context]];
    [fetch setIncludesPropertyValues:NO];

    NSError *error = nil;
    NSArray *entities = [context executeFetchRequest:fetch error:&error];

    NSInteger deletedCount = 0;
    for (NSManagedObject *item in entities) {
        [context deleteObject:item];
        deletedCount++;

        if (deletedCount == 500) {
            [context save:&error];
            deletedCount = 0;
        }
    }

    if (deletedCount != 0) {
        [context save:&error];
    }
}

I’ve tried: -setFetchBatchSize, but there’s even more memory used.

  • Swift: Adjusting the line height of UITextView
  • Can downloaded images/files be added to my app bundle?
  • UIWebView not finishing loading?
  • Programmatically change tabBar item?
  • prepareForSegue destination controller property not being set
  • Swift NSString function syntax usage
  • What would be a more memory-efficient way to do this?

    6 Solutions Collect From Internet About “What is the most efficient way to delete a large number (10.000+) objects in Core Data?”

    EDIT: Just watched 2015 WWDC “What’s New in Core Data” (it’s always the first video I watch, but I’ve been very busy this year) and they announced a new API: NSBatchDeleteRequest that should be much more efficient than any previous solution.


    Efficient has multiple meanings, and most often means some sort of trade-off. Here, I assume you just want to contain memory while deleting.

    Core Data has lots of performance options, beyond the scope of any single SO question.

    How memory is managed depends on the settings for your managedObjectContext and fetchRequest. Look at the docs to see all the options. In particular, though, you should keep these things in mind.

    Also, keep in mind the performance aspect. This type of operation should be performed on a separate thread.

    Also, note that the rest of your object graph will also come into play (because of how CoreData handles deletion of related objects.

    Regarding memory consumption, there are two properties on MOC in particular to pay attention to. While there is a lot here, it is by no means close to comprehensive. If you want to actually see what is happening, NSLog your MOC just before and after each save operation. In particular, log registeredObjects and deletedObjects.

    1. The MOC has a list of registered objects. By default, it does not retain registered objects. However, if retainsRegisteredObjects is YES, it will retain all registered objects.

    2. For deletes in particular, setPropagatesDeletesAtEndOfEvent tells the MOC how to handle related objects. If you want them handled with the save, you need to set that value to NO. Otherwise, it will wait until the current event is done

    3. If you have really large object sets, consider using fetchLimit. While faults do not take a lot of memory, they still take some, and many thousands at a time are not insignificant. It means more fetching, but you will limit the amount of memory

    4. Also consider, any time you have large internal loops, you should be using your own autorelease pool.

    5. If this MOC has a parent, saving only moves those changes to the parent. In this case, if you have a parent MOC, you are just making that one grow.

    For restricting memory, consider this (not necessarily best for your case — there are lots of Core Data options — only you know what is best for your situation, based on all the options you are using elsewhere.

    I wrote a category on NSManagedObjectContext that I use for saving when I want to make sure the save goes to the backing store, very similar to this. If you do not use a MOC hierarchy, you don’t need it, but… there is really no reason NOT to use a hierarchy (unless you are bound to old iOS).

    - (BOOL)cascadeSave:(NSError**)error {
        __block BOOL saveResult = YES;
        if ([self hasChanges]) {            
            saveResult = [self save:error];
        }
        if (saveResult && self.parentContext) {
            [self.parentContext performBlockAndWait:^{
                saveResult = [self.parentContext cascadeSave:error];
            }];
        }
        return saveResult;
    }
    

    I modified your code a little bit…

    + (void)deleteRelatedEntitiesInManagedObjectContext:(NSManagedObjectContext *)context 
    {
        NSFetchRequest *fetch = [[NSFetchRequest alloc] init];
        [context setUndoManager:nil];
    
        [fetch setEntity:[NSEntityDescription entityForName:NSStringFromClass(self) inManagedObjectContext:context]];
        [fetch setIncludesPropertyValues:NO];
        [fetch setFetchLimit:500];
    
        NSError *error = nil;
        NSArray *entities = [context executeFetchRequest:fetch error:&error];
        while ([entities count] > 0) {
            @autoreleasepool {
                for (NSManagedObject *item in entities) {
                    [context deleteObject:item];
                }
                if (![context cascadeSave:&error]) {
                    // Handle error appropriately
                }
            }
            entities = [context executeFetchRequest:fetch error:&error];
        }
    }
    

    In a moment of inspiration, I removed [fetch setIncludesPropertyValues:NO]; and it was good. From the docs:

    During a normal fetch (includesPropertyValues is YES), Core Data
    fetches the object ID and property data for the matching records,
    fills the row cache with the information, and returns managed object
    as faults (see returnsObjectsAsFaults). These faults are managed
    objects, but all of their property data still resides in the row cache
    until the fault is fired. When the fault is fired, Core Data retrieves
    the data from the row cache—there is no need to go back to the
    database.

    I managed to reduce allocated live bytes to ~13MB, which is better.

    NSBatchDeleteRequest Worked for me; reduced the time for deletion of managed objects by a factor of 5, with no memory spike.

    This would be an interesting test, try using Magical Record. There is a truncate method in there that is supposed to be very efficient (I have used it on data sets as large as 3000 records without issue. Be interesting to see how it handles 10,000.

    I wouldn’t just use it for that feature alone, if you haven’t tried it, you should. It makes dealing with Core Data so much easier and with much less code.

    Hope this helps.

    I have by no means tested it, but if memory is your main concern, you could try encapsulating those batches of 500 deletes in extra autorelease pool. It’s possible that context:save creates quite a few autoreleased objects that don’t get released until you’re done with the run loop cycle. With 10 000+ records it could add up quite nicely.

    If you do not want to use another API, try another feature of NSFetchRequest, fetchLimit, maybe in conjunction with fetchOffset. I have seen this in one of the iTunes U iPad courses in an example involving heavy number crunching with Core Data.

    NSInteger limit = 500;
    NSInteger count = [context countForFetchRequest:fetch error:nil];
    [fetch setFetchLimit:limit];
    for (int i=0; i < count/limit+1; i++) {
       // do the fetch and delete and save
    }
    

    You can adjust the fetchLimit to satisfy your memory requirements.