Optimize Linq query with PredicateBuilder with N-N join

Question

I'm using Linq to query MS CRM 2011 Web Services. I've got a query that results in very poor SQL, it fetches too much intermediary data and its performance is horrible!! I'm new to it, so it may very well be the way I'm using it...

I've got two entities linked via an N-N relationship: Product and SalesLink. I want to recover a bunch of Product from their SerialNumber along with all SalesLink associated to them.

This is the query I have using PredicateBuilder:

// Build inner OR predicate on Serial Number list
var innerPredicate = PredicateBuilder.False<Xrm.c_product>();
foreach (string sn in serialNumbers) {
   string temp = sn; // This temp assignement is important!
   innerPredicate = innerPredicate.Or(p => p.c_SerialNumber == temp);
}

// Combine predicate with outer AND predicate
var predicate = PredicateBuilder.True<Xrm.c_product>();
predicate = predicate.And(innerPredicate);
predicate = predicate.And(p => p.statecode == (int)CrmStateValueType.Active);

// Inner Join Query
var prodAndLinks = from p in orgContext.CreateQuery<Xrm.c_product>().AsExpandable()
                                                                    .Where(predicate)
                                                                    .AsEnumerable()
                   join link in orgContext.CreateQuery<Xrm.c_saleslink>()
                        on p.Id equals link.c_ProductSalesLinkId.Id
                   where link.statecode == (int)CrmStateValueType.Active
                   select new {
                         productId = p.Id
                       , productSerialNumber = p.c_SerialNumber
                       , accountId = link.c_Account.Id
                       , accountName = link.c_Account.Name
                   };
...

Using SQL profiler, I saw that it causes an intermediate SQL query that has no WHERE clause, looking like this:

select 
top 5001 "c_saleslink0".statecode as "statecode"
  ...
, "c_saleslink0".ModifiedOnBehalfByName as "modifiedonbehalfbyname"
, "c_saleslink0".ModifiedOnBehalfByYomiName as "modifiedonbehalfbyyominame" 
from
 c_saleslink as "c_saleslink0" order by
 "c_saleslink0".c_saleslinkId asc

This returns a huge amount of (useless) data. I think the join is done on the client side instead of on the DB side...

How should I improve this query? I runs in around 3 minutes and that's totally unacceptable.

Thanks.

---

"Solution"

Based on Daryl's answer to use QueryExpression instead of Linq to CRM, I got this which gets the exact same result.

var qe = new QueryExpression("c_product");
qe.ColumnSet = new ColumnSet("c_serialnumber");
var filter = qe.Criteria.AddFilter(LogicalOperator.Or);
filter.AddCondition("c_serialnumber", ConditionOperator.In, serialNumbers.ToArray());
var link = qe.AddLink("c_saleslink", "c_productid", "c_productsaleslinkid");
link.LinkCriteria.AddCondition("statecode", ConditionOperator.Equal, (int)CrmStateValueType.Active);
link.Columns.AddColumns("c_account");
var entities = serviceProxy.RetrieveMultiple(qe).Entities.ToList();;

var prodAndLinks = entities.Select(x => x.ToEntity<Xrm.c_product>()).Select(x => 
                   new {
                      productId = x.c_productId
                    , productSerialNumber = x.c_SerialNumber
                    , accountId = ((Microsoft.Xrm.Sdk.EntityReference)((Microsoft.Xrm.Sdk.AliasedValue)x["c_saleslink1.c_account"]).Value).Id
                    , accountName = ((Microsoft.Xrm.Sdk.EntityReference)((Microsoft.Xrm.Sdk.AliasedValue)x["c_saleslink1.c_account"]).Value).Name
                   }).ToList();

I really would have liked to find a solution using Linq, but it seems to Linq to CRM is just not there yet...

Answer 1

95% of the time when you're having performance issues with a complicated query in CRM, the easiest way to improve the performance is to run a straight SQL query against the database (assuming this is not CRM online of course). This may be one of the 5% of the time.

In your case, the major performance issue you're experiencing is due to the predicate builder forcing a CRM Server (not the SQL database) side join of data. If you used a Query Expression (which is what your link statement get's translated) you can specify a Condition Expression with an IN operator that would allow you to pass in your serialNumbers collection. You could also use FetchXml as well. Both of these methods would allow CRM to perform a SQL side join.

Edit:

This should get you 80% of the way with Query Expressions:

IOrganizationService service = GetService();
var qe = new QueryExpression("c_product");
var filter = qe.Criteria.AddFilter(LogicalOperator.Or);
filter.AddCondition("c_serialnumber", ConditionOperator.In, serialNumbers.ToArray());
var link = qe.AddLink("c_saleslink", "c_productid", "c_productsaleslinkid");
link.LinkCriteria.AddCondition("statecode", ConditionOperator.Equal, (int)CrmStateValueType.Active);
link.Columns.AddColumns("c_Account");
var entities = service.RetrieveMultiple(qe).Entities.ToList();

Answer 2

You will probably find you can get better control by not using Linq to Crm. You could try:

1. FetchXml, this is an xml syntax, similar in approach to tsql MSDN.

2. QueryExpression, MSDN.

3. You could issue a RetrieveRequest, blog.