linq statement very slow

Question

I have a linq to sql data context set up. I try to expect an amount of logs identified by a mappingID, so im using that to write a webclient, that shows the status of these downloads. Right now i have a situation where a linq statement is taking forever, although the amount of rows seems relatively low.

The statement taking forever is:

 var dlCatToUnion = (from cl in currentLogs
                            where ndcat.All(x=>x!=cl.CategoryCountryCategoryTypeMapping.CategoryID)
                            group cl by cl.CategoryCountryCategoryTypeMapping.Category.CategoryID into t1
                            select new CategoryStruct
                            {
                                CategoryName = t1.Max(x => x.CategoryCountryCategoryTypeMapping.Category.Name),
                                Status = t1.Any(x=>x.Response!=(int)ErrorCodes.staticCodes.success)
                                                ? (int)ErrorCodes.staticCodes.genericFailure : (int)ErrorCodes.staticCodes.success,
                                AverageResponseTime = 0,
                                categoryId = t1.Key
                            }
                            );

Specifically if you look at the second line where it says where ndcat.All(x=>x!=cl.CategoryCountryCategoryTypeMapping.CategoryID) if i take this part out, its instant.

To take a look at what that line is doing:

 var ndcat = (from ndid in notDownloadedIds
                     where ndid.Category.StorefrontID==StorefrontID
                     group ndid by ndid.CategoryID into finalCat
                     select finalCat.Key);

And then notDownloadedIds

 notDownloadedIds = cDataContext.CategoryCountryCategoryTypeMappings.Where(mapping =>!
currentLogs.Select(dll => dll.CategoryCountryCategoryTypeMappingID).Any(id => id == mapping.CategoryCountryCategoryTypeMappingID));

To give some estimates of row counts, currentLogs is around 25k rows, CategoryCountryCategoryTYpeMappingID is about 53k rows. ndcat ends up being 47 rows(also it enumerates just about instantly).

Also to note ive changed the suspect line to a ! ...Any(...) statement and its just as slow.

Is there anywhere where im being ineffecient?

Answer 1

Have you tried changing:

where ndcat.All(x => x != cl.CategoryCountryCategoryTypeMapping.CategoryID)

to:

where !ndcat.Any(x => x == cl.CategoryCountryCategoryTypeMapping.CategoryID)

??

Answer 2

I ended up changing a bit, but long story short, i did the ndcat check after grouping instead of before, which made quite a bit faster.