How to "split up" or optimize a Linq to Sql query.

Question

Background

We have a AdvancedSearchModel which is just a struct with several strings. SerialNumbers is a string of serial numbers (10 characters in length) separated by commas. This is split into an array of strings. We are searching over a table Devices which has a column, SerialNumber. We need to partial match these search phrases as well, which explains the Any and Contains usage. There are about 10 strings in the AdvancedSearchModel and we are able to apply each to the query without issue.

public static IQueryable<Device> ApplySearchQuery(IQueryable<Device> query, AdvancedSearchModel searchModel)
{
    if (!String.IsNullOrWhiteSpace(searchModel.SerialNumbers))
    {
        string[] serialNumbers = searchModel.SerialNumbers.Split(',');
        query = query.Where(d => serialNumbers.Any(s => d.SerialNumber.Contains(s))); 

     }
     // ... more queries added that are part of the search model
}

The Issue

Let me preface this with when this was implemented, we expected about 10 serial numbers to be search at once max. However, We've found users putting up to 100 Serial numbers. We get an error that the query is too deeply nested from this. So even if we only search by SerialNumbers, it will throw an exception when serialNumbers grows above 43 elements.

I'm not really sure how to split up this query, as individually they are simple (albeit we expect the result to be slow).

Perhaps we could have an ArrayList of string[] of size ~25, and grow this array list, ie if a user searches for 100 serial numbers, then the ArrayList would have 4 string arrays, but I'm not sure that really helps...

Any ideas?

Answer 1

I imagine you can get EF to generate a flat OR statement instead of a nested subquery if you generate the OR expression manually:

string[] serialNumbers = searchModel.SerialNumbers.Split(',');
// GetMethod defined at http://www.codeducky.org/10-utilities-c-developers-should-know-part-two/
var stringContainsMethod = Helpers.GetMethod((string s) => s.Contains(default(string)));
var serialNumberProperty = Helpers.GetProperty((Device d) => d.SerialNumber);
var parameter = Expression.Parameter(typeof(Device));

// generate a condition d.SerialNumber.Contains(sn) for each sn
var conditions = serialNumbers.Select(sn => Expression.Call(  
    Expression.MakeMemberAccess(parameter, serialNumberProperty), // d.SerialNumber
    stringContainsMethod, // the method to invoke
    Expression.Constant(sn) // equivalent of a string literal for the serial number
));

// generate a lambda we can pass to Where: 
// d => d.SerialNumber.Contains(sn1) || d.SerialNumber.Contains(sn1) || ..
var predicate = Expression.Lambda<Func<Device, bool>>(
    // merge each condition with ORs
    conditions.Aggregate<Expression>((c1, c2) => Expression.OrElse(c1, c2)),
    parameter
);

// apply the predicate
query = query.Where(predicate);

Batching will also, work of course. However, it shouldn't be necessary for so small an argument set.