How can I get only unique field values from a SharePoint CSOM query?

Question

I'm using the SharePoint client object model to retrieve data from a SharePoint 2013 List. The following code works, and gets me all values from the keyCol List field, correctly sorted:

string keyCol = "SPFieldName";
CamlQuery keyColumnValuesQuery = CamlQuery.CreateAllItemsQuery();
ListItemCollection keyColumnValues = srcDocLib.GetItems(keyColumnValuesQuery);
spContext.Load(keyColumnValues, items => items.Include(item => item[keyCol]), items => items.OrderBy(item => item[keyCol]));
spContext.ExecuteQuery();

However, I would only like to retrieve just one Item for every unique value in the field, a Distinct. When I add the Distinct() Linq/Lambda to the Load method retrievals argument (no compile time issues):

spContext.Load(keyColumnValues, items => items.Include(item => item[keyCol]), items => items.OrderBy(item => item[keyCol]), items => items.Distinct());

I get an InvalidQueryExpressionException: the query expression 'items.Distinct()' is not supported.

Does this mean the Linq provider on the SharePoint side doesn't support that expression and I'm just out of luck using this technique? Am I doing something incorrectly? Thanks.

Answer 1

Linq provider for SharePoint does not support Distinct operator.

According to MSDN:

Some LINQ queries cannot be completely translated into CAML. However, however such queries can, in principle, run correctly because they can be executed in two stages. First, the LINQ to SharePoint provider translates as much of the query into CAML as it can and executes that query

Please refer Unsupported LINQ Queries and Two-stage Queries for a more details.

Two stage approach

To correct this error, you should cut your queries in two stages to force the first query execution before the second one. To do that, you should for example transform the first IEnumerable<T> in a list thanks to ToList() method.

The following example demonstrates how to return unique values :

   //1 Execute query against SP Linq provider   
    string keyCol = "SPFieldName";
    CamlQuery keyColumnValuesQuery = CamlQuery.CreateAllItemsQuery();
    ListItemCollection keyColumnValues = srcDocLib.GetItems(keyColumnValuesQuery);
    spContext.Load(keyColumnValues, items => items.Include(item => item[keyCol]), items => items.OrderBy(item => item[keyCol]));
    spContext.ExecuteQuery();
    //2. Transform LiItemCollection to a List and perform Distinct operation 
    var uniqueKeyColumnValues = keyColumnValues.ToList().Select(i => i[keyCol]).Distinct();