Is there a way to runtime-conditionally Parallelise a query that produces anonymous types?

StackOverflow https://stackoverflow.com/questions/22924839

  •  29-06-2023
  •  | 
  •  

Question

Edit: mostly solved below. However, one other thing I'd be curious about (to improve google-fu in the future is - how would your phrase this question? I'm aware my title is a bit weird but could not think of a more concise way to summarise it. Suggestions in comments are welcome.

Suppose my workflow involves a potentially expensive selection. At runtime I'd like to be able to choose whether or not to make this parallel.

To explain better. Consider this query (broken code - do-what-i-want-not-what-i-say):

void Main()
{
    bool runTimeParallelOption = false; // assume this option comes from e.g. user/config

    var j = Enumerable.Range(1,5);

    if (runTimeParallelOption)
        j = j.AsParallel();

    var results = j.Select(x => 
    {
        Thread.Sleep(1000);
        return new { Number = x, Square = x * x };
    });

    var sw = Stopwatch.StartNew();
    foreach (var result in results)
        Console.WriteLine(result.Square);

        Console.WriteLine("Elapsed: : " + sw.Elapsed);
}

The hope here is that if runTimeParallelOption is set, it would make the query parallel. However obviously (in hindsight!) this doesn't work because the Type of j means I am still executing the IEnumerable extension method.

The fix for this is trivial if you don't mind refactoring to make { Number = x, Square = x * x } a first-class Type, but this isn't very appealling to me. Is there a way to have my cake and eat it?

Was it helpful?

Solution

I think the simplest option here it to always use AsParallel(). Then, if you don't want to parallelize it, add WithDegreeOfParallelism(1).

OTHER TIPS

Inspired somewhat by this, the best I've come up with is this:

void Main()
{
    bool runTimeParallelOption = true; // assume this option comes from e.g. user/config

    var j = Enumerable.Range(1,5);

    var results = Enumerable.Empty<int>().Select(x => new { Number = (int)0, Square = (int)0 }).Take(0);

    var mutate = InferredProjection(1, results.FirstOrDefault(),  x => { Thread.Sleep(1000); return new { Number = x, Square = x * x }; });

    var sw = Stopwatch.StartNew();
    if (runTimeParallelOption)
        results = j.AsParallel().Select(mutate);
    else
        results = j.Select(mutate);
    sw.Stop();

    foreach (var result in results)
        Console.WriteLine(result.Square);

    Console.WriteLine("Elapsed: : " + sw.Elapsed);
}


static Func<T, U> InferredProjection<T, U>(T infer1, U infer, Func<T, U> project)
{
    return project;
}
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top