How to select data with linq after filter parameters are determined

Question

I have a report in which the user can select different filters to apply to a dataset.

At design time, I don't know what filters the user will apply, so I grab the whole dataset.

For instance, if the user wants to see contacts that live in Chicago, I first grab all contacts...

IList<contact> contacts = db.contacts.ToList();

then I check the form collection for filters and apply them...

contacts = contacts.Where(t => t.state == form["state"]).ToList();

The issue is that getting all contacts to filter down from is resource intensive. Is there a way to wait to retrieve the contacts from the db until I have all the parameters for the query?

I realize I'm doing this wrong, but I need to know the correct way to approach this.

Answer 1

Just don't do the first call to ToList(). This pulls down all the data into memory. While you haven't done this, the data is in the form of an IEnumerable. This uses 'lazy evaluation' which means that the items to be enumerated aren't actually produced until they are requested. In the case of Linq To SQL, the database query won't be run until you do the second call to ToList().

An IEnumerable is not really a container, rather it's an object, containing a piece of code that knows how to get the next element in a sequence. This piece of code could be various things - it could be reading from an actual container, generating each value on-the-fly, or getting the values from somewhere else, like a database. A list however, is a container. When you call ToList, the next item is repeatedly extracted from the IEnumerable and placed in a List. At this point you have exactly a collection of objects in memory. You often call ToList when you want to stop having a nebulous 'thing' which gets values from somewhere and have an actual container full of actual elements. For example, an IEnumerable might not give you the same objects each time you use it. Once you have them in a List you know that they can't change. In your case you want to stay with that nebulous thing until you have decided exactly what you are going to have to ask the database for.

The way this works is fundamental to LINQ, and also is the main reason that Linq To SQL is so nice. I highly recommend that you read something about how Linq works, and also play around with List and IEnumerable a little bit. C# legend Jon Skeet's blog has loads of discussion of this, including an example reimplementation of much of Linq so you can see how it might work. If you read all of it you will be a Linq expert (and it will also be very late), but this is an introduction to how laziness works.

Answer 2

Don't call ToList().

You're getting an IEnumerable back from EF. Use the lazy evaluation to your advantage - it's especially useful with large datasets, as every call to .ToList() fills the memory with the list.