Threads vs TPL vs Async Delegates in ASP.NET

Question

I have an application that is working well in production, but I wonder if I could have implemented the concurrency better....

ASP.NET .NET 4, C#

Basically, it generates n number of sql statements on the fly (approx 50 at the moment) and then runs them concurrently and writes the data to .csv files.

EDIT: First I create a thread to do all the work on so the page request can return. Then on that thread...

For each of the SQL statements I create a new Task using the TPL and execute it using a datareader and write the data to disk. When the last file is created I write some summary data to a summary file and zip it all up and give it to the user.

Should I have used Threads or Asynchronous Delegates instead?

I haven't posted code as I am really just wondering if my overall approach (i.e. TPL) is the best option in this situation.

Please don't lecture me about creating dynamic sql, it is totally necessary due to the technicalities of the database I am reading from and not relevant to the question. (Its the back end of a proprietary system. Got 7 thousand+ tables).

Answer 1

Should I have used Threads or Asynchronous Delegates instead?

Apparently, your background thread operation spans across the boundaries of a single HTTP request. In this case, it doesn't really matter what API you use to run such operation: Task.Run, Delegate.BeginInvoke, ThreadPool.QueueUserWorkItem, new Thread or anything else.

You shouldn't be running a lengthy background thread operation, which lifetime spans multiple HTTP requests, inside ASP.NET address space. While it's relatively easy to implement, this approach may have issues with IIS maintainability, scalability and security. Create a WCF service for that and call it from your ASP.NET page:

How to: Host a WCF Service in a Managed Windows Service.

Answer 2

If we start a new thread in ASP.Net from the thread which is serving the http request, and new thread has an unhandled exception, the worker process will crash immediately. Even if we use WCF service and call that from ASP.Net the ASP.Net thread is going to wait for the result. So better use any queuing mechanism so that the requests is in queue and queue can process in a different time based on the processing capacity. Of course when we say queuing we need to think about queue failure, requeue etc...But its worth if the application is big and needs to scale.