TableServiceContext and dynamic query

Question

I m trying to do something that look very simple but I hit massive difficulties when I want to make that more dynamic.

    Expression<Func<TableServiceEntity, bool>> predicate = (e) => e.PartitionKey == "model" && (e.RowKey == "home" || e.RowKey == "shared");

context.CreateQuery<TableServiceEntity>(tableName).Where(predicate);

I would like to pass an array of rowKey instead of having to hard code the predicate.

When I try to build an expression tree I receive a not supported exception I think it doesn't support invoking as part of the expression tree.

Does someone know how to build and expression tree exactly as the predicate to avoid the not supported exception?

Thank you by advance

Answer 1

So, you can build the query dynamically by using something like this (taken from PhluffyFotos sample):

        Expression<Func<PhotoTagRow, bool>> search = null;
        foreach (var tag in tags)
        {
            var id = tag.Trim().ToLowerInvariant();

            if (String.IsNullOrEmpty(id))
            {
                continue;
            }

            Expression<Func<PhotoTagRow, bool>> addendum = t => t.PartitionKey == id;

            if (search == null)
            {
                search = addendum;
            }
            else
            {
                search = Expression.Lambda<Func<PhotoTagRow, bool>>(Expression.OrElse(search.Body, addendum.Body), search.Parameters);
            }
        }

Now, once you have 'search' you can just pass that as the predicate in your Where clause.

However, I want to convince you not to do this. I am answering your question, but telling you that it is a bad idea to do a multiple '|' OR clause in Table storage. The reason is that today at least, these queries cannot be optimized and they cause a full table scan. The performance will be horrendous with any non-trivial amount of data. Furthermore, if you build your predicates dynamically like this you run the risk of blowing the URL limit (keep that in mind).

This code in PhluffyFotos shows how, but it is actually a bad practice (I know, I wrote it). It really should be optimized to run each OR clause separately in parallel. That is how you really should do it. AND clauses are ok, but OR clauses should be parallelized (use PLINQ or TPL) and you should aggregate the results. It will be much faster.

HTH.

Answer 2

I believe what HTH said about this kind of query doing a full table scan is incorrect from the documentation I have read. Azure will perform a PARTITION scan rather than a TABLE scan which is a big difference in performance.

Answer 3

Here is my solution please read also the answer from HTH who pointed out that this is not a best practice.

var parameter = Expression.Parameter(typeof(TableServiceEntity), "e");

var getPartitionKey = typeof(TableServiceEntity).GetProperty("PartitionKey").GetGetMethod();
var getRowKey = typeof(TableServiceEntity).GetProperty("RowKey").GetGetMethod();

var getPartition = Expression.Property(parameter, getPartitionKey);
var getRow = Expression.Property(parameter, getRowKey);

var constPartition = Expression.Constant("model", typeof(string));
var constRow1 = Expression.Constant("home", typeof(string));
var constRow2 = Expression.Constant("shared", typeof(string));

var equalPartition = Expression.Equal(getPartition, constPartition);
var equalRow1 = Expression.Equal(getRow, constRow1);
var equalRow2 = Expression.Equal(getRow, constRow2);

var and = Expression.AndAlso(equalPartition, Expression.OrElse(equalRow1, equalRow2));

return Expression.Lambda<Func<TableServiceEntity, bool>>(and, parameter);