i have database with larger number of records.(database values are updated via a webservice)

Each record/row is of type (id,xmlfilename,operation,parameters,operationId,status) ie. we have perform operation as specified by'operation' on a xmlfile specified by "xmlfilename" with parameters for operation specified by "parameters".. status part is "intially" free as is updated as when reqd.

I have to do these (each row) operation using one thread per 'id' number of long as there are 'id' rows in db fetch and perform "operation"

how can i do this efficiently with maximum parallelism and/or concurrency.

Is there a better way?

What is best suited Threadpool or custom threads or asyn programming?

Edit Added:Here is pseudo code i tried

string operationId=null;

while(operationId = DBWorker.getNextFreeOperationId ())
//how do i check this?another query??untill there are operations with status "Free" 
//,keep selecting operationids 
//note db is updating asynchronously.    
  //retrieve all rows with operationid=operationId eg:800 . 1 thread/qid???? and
  // status="free" ...
//there are multiple operations with same operationIds 

  DataRowCollection dbRows=DBWorker.retrieveQueuedEntries(operationId); 
  MyWorkItem workItem = new DBWorker.MyWorkItem();
  workItem.DataRows = dbRows; 
  workItem.Event = new AutoResetEvent(false); 
 //MyWorkItem.DoWork will do the necessary "Operation" 
 ThreadPool.QueueUserWorkItem(new WaitCallback(workItem.DoWork),workItem); 
 MyWorkItem.DoWork(obj x){ 
//for brevity 
for each DataRow row in this.DataRows
    performOperation(row);//use row["operation"] ..
bool performOperation(DataRow row)
  string operation = (string)row["operation"];//retrieve other similarly
    case Operations.Add: //call Add operation ..

above code is using.net2.0.. doesnt achieve multithreading .. scenario may be the database is updating from one end asynchronously,while above code will be running as windows service at the same time.


No correct solution


If you're not using .net 4.0 - Use ThreadPool I would create a function that would receive the row (event better if you convert the rows to strongly type objects in DAL) it has to process, and do what it has to do with it - something like

using System;
using System.Threading;

namespace SmallConsoleAppForTests
class Program

    private static AutoResetEvent[] events;

    static void Main(string[] args)

        int dataLength = 3;
        // creating array of AutoResetEvent for signalling that the processing is done
        AutoResetEvent[] events = new AutoResetEvent[dataLength];

        // Initializing the AutoResetEvent array to "not-set" values;
        for (int i = 0; i < dataLength; i++)
            events[i] = new AutoResetEvent(false);

        //Processing the data
        for (int i = 0; i < dataLength; i++)
            var data = new MyWorkItem { Event = events[i], Data = new MyDataClass() };

            ThreadPool.QueueUserWorkItem(x =>
                var workItem = (MyWorkItem)x;
                    // process the data

                catch (Exception e)
                    //exception handling
            }, data);

        //Wait untill all the threads finish


public class MyWorkItem
    public AutoResetEvent Event { get; set; }
    public MyDataClass Data { get; set; }

// You can also use DataRow instead
public class MyDataClass

If you are using .net 4.0 - look into Tasks and Parallel Extensions (PLINQ)

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow