Comparing i4o vs. PLINQ for larger collections

Question

I have a question for anyone who has experience on i4o or PLINQ. I have a big object collection (about 400K ) needed to query. The logic is very simple and straightforward. For example, there has a collection of Person objects, I need to find the persons matched with same firstName, lastName, datebirth, or the first initial of FirstName/lastname, etc. It is just a time consuming process using LINQ to Object.

I am wondering if i4o (http://www.codeplex.com/i4o)

or PLINQ can help on improving the query performance. Which one is better? And if there has any approach out there.

Thanks!

Answer 1

With 400k objects, I wonder whether a database (either in-process or out-of-process) wouldn't be a more appropriate answer. This then abstracts the index creation process. In particular, any database will support multiple different indexes over different column(s), making the queries cited all very supportable without having to code specifically for each (just let the query optimizer worry about it).

Working with it in-memory may be valid, but you might (with vanilla .NET) have to do a lot more manual index management. By the sounds of it, i4o would certainly be worth investigating, but I don't have any existing comparison data.

Answer 2

i4o : is meant to speed up quering using linq by using indexes like old relational database days.

PLinq: is meant to use extra cpu cores to process the query in parallel.

If performance is your target, depending on your hardware, I say go with i4o it will make a hell of improvement.

Answer 3

I haven't used i4o but I have used PLINQ.

Without know specifics of the query you're trying to improve it's hard to say which (if any) will help.

PLINQ allows for multiprocessing of queries, where it's applicable. There are time however when parallel processing won't help.

i4o looks like it helps with indexing, which will speed up some calls, but not others.

Bottom line is, it depends on the query being run.