Which methods to close a compiled query

Question

As we know you can't add an extra clause like .Where() or .First() to a compiled query, because that changes the query and forces a recompile. What I would like to know is which methods can be used to "close" a compiled query.

I know most people use either .AsEnumerable() or .ToList(), but which other methods work as well? Can I use .AsQueryable(), or is this a no-op?

And which is better performance-wise? I know .AsEnumerable() is faster than .ToList(), but if I want an IQueryable, is .AsEnumerable().AsQueryable() better than .ToList()?

Answer 1

In most cases, AsEnumerable().AsQueryable() is probably what you want, because:

• By doing an explicit AsEnumerable(), you're not running the risk of the underlying implementation making AsQueryable() into a no-op, thereby ruining your attempt to close the query. I'm not saying that today's EF makes AsQueryable() a no-op (as far as I can tell, it doesn't), only that the behavior-- either no-op or transparently call AsEnumerable()-- isn't documented so relying on it isn't safe.
• AsEnumerable(), unlike ToList(), doesn't load your entire resultset in memory in order to query it. This matters a lot with large resultsets. It's theoretically possible that for smaller resultsets, there could be some advantage to using ToList() (e.g. an optimized ToList() implementation pulls data in big chunks from the underlying provider, while enumeration involves more context switching) but that seems unlikely and hard to depend on across providers and versions, while the large-resultset advantage of AsEnumerable() will live forever.

The one case where I do like calling ToList() is when I explicitly do want to force a query to execute right now. For example, if I want to catch errors in a query earlier in a method so that I can simplify error handling later, or I want to validate all the underlying data before continuing with the rest of the query. Or if the query is more easily tested when chopped in two. And I'll never do this, unless I know that my recordset is going to be small, since calling ToList() on a multi-million-row query will kill your RAM.

To answer your other question, Converting Data Types in the LINQ documentation on MSDN details which LINQ methods force query execution. According to that page, ToArray(), ToDictionary(), ToList(), and ToLookup() all force query execution.

AsEnumerable(), by contrast, doesn't force immediate query execution, but does "close" the query (using your term here, not sure if there's an official term for this). Per http://msdn.microsoft.com/en-us/library/bb335435.aspx:

the AsEnumerable method can be used to hide the custom methods and instead make the standard query operators available.

In other words, running AsEnumerable will force all calls like Take() and Where() to use the generic LINQ implementations and not anythign custom which would cause a re-compile.

Answer 2

which methods can be used to "close" a compiled query.

Methods that return a sequence use deferred execution, unless the method is something like ToXYZ. Where, Select, Take, Skip, GroupBy and OrderBy etc falls under this. Methods that return a single object forces execution of query, like First, Single, ToList and ToArray, ToDictionary, ToLookup, Any, All etc. See this excellent thread for more: Linq - What is the quickest way to find out deferred execution or not?

I know most people use either .AsEnumerable() or .ToList(), but which other methods work as well? Can I use .AsQueryable(), or is this a no-op?

They all are different. Justin has a grand explanation. You might also want to see: What's the difference(s) between .ToList(), .AsEnumerable(), AsQueryable()? which has a good answer.

---

In general, you can understand the semantics of a method by seeing the name of the method itself. A method named AsSomething implies it does nothing but returns the input as something. That may or may not involve returning a new object, but a reference is somehow maintained. For instance, aList<T>.AsEnumerable() merely does a cast to IEnumerable<T>(of course it has bigger meaning in linq context). You can cast it back to List<T> and mutate it reflecting the change everywhere. To test it:

var list = new List<int> { 1, 2 };
var enum = list.AsEnumerable();
var newlist = enum as List<string>;
newlist.Add(3);
//print enum.Count() -> 3

While methods that look like ToSomething, you get a totally new object often transformed to something else.

var list = new List<int> { 1, 2 };
var newlist = list.ToList();
newlist.Add(3);
//print list.Count -> 2

Let's consider something outside the context of linq. object.ToString() results in new string representation (strings are anyway immutable so thats a bit pointless). An interesting semantics is that of List<T>.AsReadonly which returns a new ReadOnlyCollection<T> instance, but mutating the list outside of it changes the internal list of ReadOnlyCollection<T> too, hence the naming AsReadonly.

var list = new List<int> { 1, 2 };
var readonlylist = list.AsReadonly();
list.Add(3);
//print readonlylist.Count -> 3