LINQ : Dynamic select

Question

Consider we have this class :

    public  class Data
{
    public string Field1 { get; set; }
    public string Field2 { get; set; }
    public string Field3 { get; set; }
    public string Field4 { get; set; }
    public string Field5 { get; set; }

}

How do I dynamically select for specify columns ? something like this :

  var list = new List<Data>();

  var result= list.Select("Field1,Field2"); // How ?

Is this the only solution => Dynamic LINQ ?
Selected fields are not known at compile time. They would be specified at runtime

Answer 1

You can do this by dynamically creating the lambda you pass to Select:

Func<Data,Data> CreateNewStatement( string fields )
{
    // input parameter "o"
    var xParameter = Expression.Parameter( typeof( Data ), "o" );

    // new statement "new Data()"
    var xNew = Expression.New( typeof( Data ) );

    // create initializers
    var bindings = fields.Split( ',' ).Select( o => o.Trim() )
        .Select( o => {

            // property "Field1"
            var mi = typeof( Data ).GetProperty( o );

            // original value "o.Field1"
            var xOriginal = Expression.Property( xParameter, mi );

            // set value "Field1 = o.Field1"
            return Expression.Bind( mi, xOriginal );
        }
    );

    // initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
    var xInit = Expression.MemberInit( xNew, bindings );

    // expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
    var lambda = Expression.Lambda<Func<Data,Data>>( xInit, xParameter );

    // compile to Func<Data, Data>
    return lambda.Compile();
}

Then you can use it like this:

var result = list.Select( CreateNewStatement( "Field1, Field2" ) );

Answer 2

In addition for Nicholas Butler and the hint in comment of Matt(that use T for type of input class), I put an improve to Nicholas answer that generate the property of entity dynamically and the function does not need to send field as parameter.

For Use add class as below:

public static class Helpers
{
    public static Func<T, T> DynamicSelectGenerator<T>(string Fields = "")
    {
        string[] EntityFields;
        if (Fields == "")
            // get Properties of the T
            EntityFields = typeof(T).GetProperties().Select(propertyInfo => propertyInfo.Name).ToArray();
        else
            EntityFields = Fields.Split(',');

        // input parameter "o"
        var xParameter = Expression.Parameter(typeof(T), "o");

        // new statement "new Data()"
        var xNew = Expression.New(typeof(T));

        // create initializers
        var bindings = EntityFields.Select(o => o.Trim())
            .Select(o =>
            {

                // property "Field1"
                var mi = typeof(T).GetProperty(o);

                // original value "o.Field1"
                var xOriginal = Expression.Property(xParameter, mi);

                // set value "Field1 = o.Field1"
                return Expression.Bind(mi, xOriginal);
            }
        );

        // initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
        var xInit = Expression.MemberInit(xNew, bindings);

        // expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
        var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);

        // compile to Func<Data, Data>
        return lambda.Compile();
    }
}

The DynamicSelectGenerator method get entity with type T, this method have optional input parameter Fields that if you want to select special field from entity send as a string such as "Field1, Field2" and if you don't send anything to method, it returns all of the fields of entity, you could use this method as below:

 using (AppDbContext db = new AppDbContext())
            {
                //select "Field1, Field2" from entity
                var result = db.SampleEntity.Select(Helpers.DynamicSelectGenerator<SampleEntity>("Field1, Field2")).ToList();

                //select all field from entity
                var result1 = db.SampleEntity.Select(Helpers.DynamicSelectGenerator<SampleEntity>()).ToList();
            }

(Assume that you have a DbContext with name AppDbContext and the context have an entity with name SampleEntity)

Answer 3

I simplified the amazing method DynamicSelectGenerator() created by Ali and made this extension method that overrides the LINQ Select() to take a column separated parameters to simplify the usage and for more readability:

public static IEnumerable<T> Select<T>(this IEnumerable<T> source, string parameters)
{
    return source.Select(DynamicSelectGenerator<T>(parameters));
}

So instead of:

var query = list.Select(Helpers.DynamicSelectGenerator<Data>("Field1,Field2")).ToList();

Will be:

var query = list.Select("Field1,Field2").ToList();

Answer 4

The OP mentioned Dynamic Linq library, so I'd like to lay out an explanation on its usage.

1. Dynamic Linq Built-In Select

Dynamic Linq has a built-in Select method, which can be used as follows:

var numbers = new List<int> { 1, 2, 3 };
var wrapped = numbers.Select(num => new { Value = num }).ToList();

// the "it" keyword functions as the lambda parameter,
// so essentialy it's like calling: numbers.Select(num => num)
var selectedNumbers = numbers.Select("it"); 

// the following is the equivalent of calling: wrapped.Select(num => num.Value)
var selectedValues = wrapped.Select("Value");

// the following is the equivalent of calling: numbers.Select(num => new { Value = num })
var selectedObjects = numbers.Select("new(it as Value)"); 

foreach (int num in selectedNumbers) Console.WriteLine(num);
foreach (int val in selectedValues) Console.WriteLine(val);
foreach (dynamic obj in selectedObjects) Console.WriteLine(obj.Value);

The Downside

There's somewhat a downside using the built-in Select:

Since it's an IQueryable - not IQueryable<T> - extension method, with IQueryable as its return type, common materialization methods - like ToList or FirstOrDefault - can't be used. This is why the above example uses foreach - it's simply the only convenient way of materializing the results.

So to make things more convenient, let's support these methods.

2. Supporting Select<T> in Dynamic Linq (to enable using ToList and alike)

To support Select<T>, it needs to be added into the Dynamic Linq file. The simple steps for doing that are explained in this answer and in my comment on it.

After doing so, it can be used in the following way:

var numbers = new List<int> { 1, 2, 3 };
var wrapped = numbers.Select(num => new { Value = num }).ToList();

// the following is the equivalent of calling: numbers.Select(num => num).ToList()
var selectedNumbers = numbers.Select<int>("it").ToList(); 

// the following is the equivalent of calling: wrapped.Select(num => num.Value).ToList()
var selectedValues = wrapped.Select<int>("Value").ToList();

// the following is the equivalent of calling: numbers.Select(num => new { Value = num }).ToList()
var selectedObjects = numbers.Select<object>("new(it as Value)").ToList(); 

The Downside

Arguably, this implementation introduces yet another kind of downside: By having to explicitly parameterize the Select<T> call (e.g., having to call Select<int>), we're losing the dynamic nature of the library.

Nevertheless, since we can now call any materialization Linq method, this usage may still be quite useful.

Answer 5

I writing the method in following line for you can work with nested fields taking advantage of Nicholas Butler and Ali.

You can use this method for dynamically creating to lambda for pass to select and also works for nested fields. You can also work with IQueryable cases.

    /// <param name="Fields">
    /// Format1: "Field1"
    /// Format2: "Nested1.Field1"
    /// Format3: "Field1:Field1Alias"
    /// </param>
    public static Expression<Func<T, TSelect>> DynamicSelectGenerator<T, TSelect>(params string[] Fields)
    {
        string[] EntityFields = Fields;
        if (Fields == null || Fields.Length == 0)
            // get Properties of the T
            EntityFields = typeof(T).GetProperties().Select(propertyInfo => propertyInfo.Name).ToArray();

        // input parameter "x"
        var xParameter = Expression.Parameter(typeof(T), "x");

        // new statement "new Data()"
        var xNew = Expression.New(typeof(TSelect));

        // create initializers
        var bindings = EntityFields
            .Select(x =>
            {
                string[] xFieldAlias = x.Split(":");
                string field = xFieldAlias[0];

                string[] fieldSplit = field.Split(".");
                if (fieldSplit.Length > 1)
                {
                    // original value "x.Nested.Field1"
                    Expression exp = xParameter;
                    foreach (string item in fieldSplit)
                        exp = Expression.PropertyOrField(exp, item);

                    // property "Field1"
                    PropertyInfo member2 = null;
                    if (xFieldAlias.Length > 1)
                        member2 = typeof(TSelect).GetProperty(xFieldAlias[1]);
                    else
                        member2 = typeof(T).GetProperty(fieldSplit[fieldSplit.Length - 1]);

                    // set value "Field1 = x.Nested.Field1"
                    var res = Expression.Bind(member2, exp);
                    return res;
                }
                // property "Field1"
                var mi = typeof(T).GetProperty(field);
                PropertyInfo member;
                if (xFieldAlias.Length > 1)
                    member = typeof(TSelect).GetProperty(xFieldAlias[1]);
                else member = typeof(TSelect).GetProperty(field);

                // original value "x.Field1"
                var xOriginal = Expression.Property(xParameter, mi);

                // set value "Field1 = x.Field1"
                return Expression.Bind(member, xOriginal);
            }
        );

        // initialization "new Data { Field1 = x.Field1, Field2 = x.Field2 }"
        var xInit = Expression.MemberInit(xNew, bindings);

        // expression "x => new Data { Field1 = x.Field1, Field2 = x.Field2 }"
        var lambda = Expression.Lambda<Func<T, TSelect>>(xInit, xParameter);

        return lambda;
    }

Usage:

var s = DynamicSelectGenerator<SalesTeam, SalesTeamSelect>(
            "Name:SalesTeamName",
            "Employee.FullName:SalesTeamExpert"
            );

var res = _context.SalesTeam.Select(s);

public class SalesTeam
{
    public string Name {get; set; }

    public Guid EmployeeId { get; set; }
    public Employee Employee { get; set; }
}
public class SalesTeamSelect
{
    public string SalesTeamName {get; set; }
    public string SalesTeamExpert {get; set; }
}

Answer 6

I have generate my own class for same purpose of usage.

github gist : https://gist.github.com/mstrYoda/663789375b0df23e2662a53bebaf2c7c

It generates dynamic select lambda for given string and also support for two level nested properties.

Example of usage is :

class Shipment {
   // other fields...
   public Address Sender;
   public Address Recipient;
}

class Address {
    public string AddressText;
    public string CityName;
    public string CityId;
}

// in the service method
var shipmentDtos = _context.Shipments.Where(s => request.ShipmentIdList.Contains(s.Id))
                .Select(new SelectLambdaBuilder<Shipment>().CreateNewStatement(request.Fields)) // request.Fields = "Sender.CityName,Sender.CityId"
                .ToList();

It compiles the lambda as below:

s => new Shipment {
    Sender = new Address {
        CityId = s.Sender.CityId,
        CityName = s.Sender.CityName
    }
}

You can also find my quesion and answer here :c# - Dynamically generate linq select with nested properties

public class SelectLambdaBuilder<T>
{
// as a performence consideration I cached already computed type-properties
private static Dictionary<Type, PropertyInfo[]> _typePropertyInfoMappings = new Dictionary<Type, PropertyInfo[]>();
private readonly Type _typeOfBaseClass = typeof(T);

private Dictionary<string, List<string>> GetFieldMapping(string fields)
{
    var selectedFieldsMap = new Dictionary<string, List<string>>();

    foreach (var s in fields.Split(','))
    {
        var nestedFields = s.Split('.').Select(f => f.Trim()).ToArray();
        var nestedValue = nestedFields.Length > 1 ? nestedFields[1] : null;

        if (selectedFieldsMap.Keys.Any(key => key == nestedFields[0]))
        {
            selectedFieldsMap[nestedFields[0]].Add(nestedValue);
        }
        else
        {
            selectedFieldsMap.Add(nestedFields[0], new List<string> { nestedValue });
        }
    }

    return selectedFieldsMap;
}

public Func<T, T> CreateNewStatement(string fields)
{
    ParameterExpression xParameter = Expression.Parameter(_typeOfBaseClass, "s");
    NewExpression xNew = Expression.New(_typeOfBaseClass);

    var selectFields = GetFieldMapping(fields);

    var shpNestedPropertyBindings = new List<MemberAssignment>();
    foreach (var keyValuePair in selectFields)
    {
        PropertyInfo[] propertyInfos;
        if (!_typePropertyInfoMappings.TryGetValue(_typeOfBaseClass, out propertyInfos))
        {
            var properties = _typeOfBaseClass.GetProperties();
            propertyInfos = properties;
            _typePropertyInfoMappings.Add(_typeOfBaseClass, properties);
        }

        var propertyType = propertyInfos
            .FirstOrDefault(p => p.Name.ToLowerInvariant().Equals(keyValuePair.Key.ToLowerInvariant()))
            .PropertyType;

        if (propertyType.IsClass)
        {
            PropertyInfo objClassPropInfo = _typeOfBaseClass.GetProperty(keyValuePair.Key);
            MemberExpression objNestedMemberExpression = Expression.Property(xParameter, objClassPropInfo);

            NewExpression innerObjNew = Expression.New(propertyType);

            var nestedBindings = keyValuePair.Value.Select(v =>
            {
                PropertyInfo nestedObjPropInfo = propertyType.GetProperty(v);

                MemberExpression nestedOrigin2 = Expression.Property(objNestedMemberExpression, nestedObjPropInfo);
                var binding2 = Expression.Bind(nestedObjPropInfo, nestedOrigin2);

                return binding2;
            });

            MemberInitExpression nestedInit = Expression.MemberInit(innerObjNew, nestedBindings);
            shpNestedPropertyBindings.Add(Expression.Bind(objClassPropInfo, nestedInit));
        }
        else
        {
            Expression mbr = xParameter;
            mbr = Expression.PropertyOrField(mbr, keyValuePair.Key);

            PropertyInfo mi = _typeOfBaseClass.GetProperty( ((MemberExpression)mbr).Member.Name );

            var xOriginal = Expression.Property(xParameter, mi);

            shpNestedPropertyBindings.Add(Expression.Bind(mi, xOriginal));
        }
    }

    var xInit = Expression.MemberInit(xNew, shpNestedPropertyBindings);
    var lambda = Expression.Lambda<Func<T,T>>( xInit, xParameter );

    return lambda.Compile();
}

Answer 7

You must use reflection to get and set property value with it's name.

  var result = new List<Data>();
  var data = new Data();
  var type = data.GetType();
  var fieldName = "Something";

  for (var i = 0; i < list.Count; i++)
  {
      foreach (var property in data.GetType().GetProperties())
      {
         if (property.Name == fieldName)
         {
            type.GetProperties().FirstOrDefault(n => n.Name == property.Name).SetValue(data, GetPropValue(list[i], property.Name), null);
            result.Add(data);
         }
      }
  }

And here is GetPropValue() method

public static object GetPropValue(object src, string propName)
{
   return src.GetType().GetProperty(propName).GetValue(src, null);
}

Answer 8

Another approach I've used is a nested ternary operator:

string col = "Column3";
var query = table.Select(i => col == "Column1" ? i.Column1 :
                              col == "Column2" ? i.Column2 :
                              col == "Column3" ? i.Column3 :
                              col == "Column4" ? i.Column4 :
                              null);

The ternary operator requires that each field be the same type, so you'll need to call .ToString() on any non-string columns.

Answer 9

Using Reflection and Expression bulid can do what you say. Example:

var list = new List<Data>();
//bulid a expression tree to create a paramter
ParameterExpression param = Expression.Parameter(typeof(Data), "d");
//bulid expression tree:data.Field1
Expression selector = Expression.Property(param,typeof(Data).GetProperty("Field1"));
Expression pred = Expression.Lambda(selector, param);
//bulid expression tree:Select(d=>d.Field1)
Expression expr = Expression.Call(typeof(Queryable), "Select",
    new Type[] { typeof(Data), typeof(string) },
    Expression.Constant(list.AsQueryable()), pred);
//create dynamic query
IQueryable<string> query = list.AsQueryable().Provider.CreateQuery<string>(expr);
var result=query.ToList();

Answer 10

Using ExpandoObject you can build a dynamic objects or return the full object from the example below.

public object CreateShappedObject(object obj, List<string> lstFields)
{
    if (!lstFields.Any())
    {
        return obj;
    }
    else
    {
        ExpandoObject objectToReturn = new ExpandoObject();
        foreach (var field in lstFields)
        {
            var fieldValue = obj.GetType()
                .GetProperty(field, BindingFlags.IgnoreCase | BindingFlags.Public | BindingFlags.Instance)
                .GetValue(obj, null);

            ((IDictionary<string, object>)objectToReturn).Add(field, fieldValue);
        }

        return objectToReturn;
    }
}

The following is an example of how to use this from your controller.

http://localhost:12345/api/yourapi?fields=field1,field2

public IHttpActionResult Get(string fields = null)
{
    try
    {
        List<string> lstFields = new List<string>();
        if (fields != null)
        {
            lstFields = fields.ToLower().Split(',').ToList();
        }
   
        // Custom query
        var result = db.data.Select(i => CreateShappedObject(new Data()
        , lstFields)).ToList();

        return Ok(result);

    }
    catch(Exception)
    {
        return InternalServerError();
    }
}

Answer 11

var result = from g in list.AsEnumerable()
                select new {F1 = g.Field1,F2  = g.Field2};