Core Data Fetch requests slow on large data set

Question

This is my first Core Data project, and I need advice on speeding up my fetch requests.

My Core Data model contains 2 entities, Wells and Fluids. Wells has 50,000 records, and Fluids has 2 million records. They look like the following.

Wells
nams  
relation  
wellsToFluids

Fluids
text1, text2, etc.  
relation  
fluidsToWells  

Fetch requests on Wells are very fast. Fetch requests on Wells combined with data from Fluids accessed via compound predicates via the wellsToFluids relation are slow. And, I am seeing unexpected fetch times on different predicates.

I am building a compound predicate, based on user selections. But basically the situation is the following

Searches on Wells:

predicateWithFormat: @"(wellNumber == 1)"

.001 sec

Searches on Fluids:

predicateWithFormat: @"(ANY  wellsToFluids.text2 CONTAINS[c] stringToFind)"

1.3 sec. (on Mac simulator – REALLY slow on iPhone or iPad)

Fetch on Wells and Fluids:

predicateWithFormat: @"(wellNumber == 1)  AND (ANY  wellsToFluids.text1 CONTAINS[c] stringToFind)"

3.2 sec.

Fetch on Wells and multiple Fluids attributes:

predicateWithFormat: @"(wellNumber == 1)  AND (ANY  wellsToFluids.text1 CONTAINS[c] stringToFind) AND (ANY  wellsToFluids.text2 CONTAINS[c] stringToFind)"

6 sec.

Changing Fluids portion of predicate to subquery yeld weird results.

Fetch on only Fluids with subquery:

predicateWithFormat:@"(SUBQUERY(wellsToFluids, $x, ANY $x.text1 CONTAINS[c] stringToFind).@count !=0)"

12 sec.

Fetch on Wells and Fluids with subquery:

predicateWithFormat: @"(wellNumber == 1)  AND  (SUBQUERY(wellsToFluids, $x, ANY $x.text1 CONTAINS[c] stringToFind).@count !=0)"

3.2 sec.

Time does not change for additional attributes added in fetch from Fluids, stays at a fairly constant 3.2 sec.

Also tried fetching from Fluids, then using relation back to Wells – but that did not improve speed.

Can someone can give me some pointers on improving the fetch setup or it is just a feature of 2 million records?

Answer 1

Here some hints to achieve what you want. Anyway, you always need to measure your changes to see if they are valid or not. It's a feedback process. Change and measure. Change and measure and so on.

• Prefetched relationships
• Use batches
• Prefer beginwith or endswith instead of contains
• Avoid [cd] when necessary
• Normalize strings
• ...

These are just hints.

I really suggest to take a look High Performance Core Data blog. There are a video, slides and code on how to achieve this. Matthew Morey did a very good job.

Also Apple Developer site (you must be a member to watch it), in WWDC 2013 Videos, has a session called Core Data Performance Optimization and Debugging (session 211) which talks about performance.