Should I not use TPL when I need scalability?

Question

Threadpool threads are important reuseable threads which (in asp.net for example ) - helps serving requests.

As opposed to the primitive new Thread().start(....) which is NOT using threadpool thread , and does not support cancellation tokens , continuity , result value - there is no doubt that TPL is the preferred strategy.

But the problem is that Task also uses threadpool threads.

In TPL, a TaskScheduler is responsible for actually queuing the tasks up for execution. The Default scheduler will use the thread-pool .

And this makes me wonder :

Say I have a site with many concurrent users , and I need to do 3 compute bound tasks ( non-IO) for each user.

I'm afraid that the threadpool will have no threads left ( because I will do the 3 compute operation (for each user) via TASK which uses Threadpool Thread. - which will cause creating NEW threadpool threads.

Which leads me to the conclusion that when I need scalability - it's better to use the old fashion new Thread().start(...) ?

What am I missing? Are we back to square one ?

Answer 1

You rarely need to use Task.Run or Task.Factory.StartNew in ASP.NET. These APIs are for CPU-bound work, and it usually doesn't make sense to offload CPU-bound work to another thread while processing an HTTP request (unlike with a client-side UI app). That would only hurt scalability. Just do the work on the current thread.

If you need to spawn a long-running task with life-time across multiple HTTP requests, use a separate process a suggested by @Damien_The_Unbeliever in the comments, e.g., a WCF service on the same or a separate host.

That said, you still can have a non-pool thread wrapped as Task with all goodies, at least with the current TPL implementation. It's created when you use TaskCreationOptions.LongRunning with Task.Factory.StartNew. Inside such thread, Thread.IsThreadPoolThread will be false. This may prevent ThreadPool starvation, but certainly will increase the working set of the process. Moreover, such threads are not reused by TPL like ThreadPool threads, so it may be quite costly to create non-pool threads repetitively.

As an alternative to TaskCreationOptions.LongRunning, it's easy to wrap new Thread().Start() as a Task using TaskCompletionSource, with the same cancellation, exception and result propagation logic.

Yet another idea is to use a custom task scheduler which always queues the task on a new thread:

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine(new { Thread.CurrentThread.ManagedThreadId });

        var task = Task.Factory.StartNew(
            () => Console.WriteLine(new { 
                Thread.CurrentThread.IsThreadPoolThread,
                Thread.CurrentThread.ManagedThreadId}),
            CancellationToken.None,
            // hide the scheduler from inner tasks
            TaskCreationOptions.HideScheduler,
            NewThreadTaskScheduler.Scheduler);

        task.Wait();
    }
}

class NewThreadTaskScheduler : TaskScheduler
{
    public static readonly NewThreadTaskScheduler Scheduler = 
        new NewThreadTaskScheduler();

    NewThreadTaskScheduler()
    {
    }

    protected override void QueueTask(Task task)
    {
        var thread = new Thread(() =>
        {
            base.TryExecuteTask(task);
        });
        thread.IsBackground = true;
        thread.Start();
    }

    protected override bool TryExecuteTaskInline(
        Task task, 
        bool taskWasPreviouslyQueued)
    {
        return false;
    }

    protected override IEnumerable<Task> GetScheduledTasks()
    {
        return null;
    }

    public override int MaximumConcurrencyLevel { 
        get { return Int32.MaxValue; } }
}