Writing a sync application using Windows Service to process files in Parallel

Question

I have a folder on my windows server, where people will be uploading CSV files to, C:\Uploads.

I want to write a simple windows service application that will scan this uploads folder (every 5 seconds) and collect the files in and process them in parallel (Thread /per File?). However, the main scanning process should not overlap, i.e. locking is required.

So, I was experimenting with it like this: I am aware this is not windows service code, it's a console app to test ideas...

Updated Code, based on dcastro's reply

class Program
{
    static Timer _InternalTimer;
    static Object _SyncLock = new Object();

    static void Main(string[] args)
    {
        _InternalTimer = new Timer(InitProcess, null, 0, 5000); // Sync cycle is every 5 sec

        Console.ReadKey();
    }

    private static void InitProcess(Object state)
    {
        ConsoleLog("Starting Process");
        StartProcess();
    }

    static void StartProcess()
    {
        bool lockTaken = false;
        try
        {
            Monitor.TryEnter(_SyncLock, ref lockTaken);
            if (lockTaken)
            {
                ConsoleLog("Lock Acquired. Doing some dummy work...");

                List<string> fileList = new List<string>()
                {
                    "fileA.csv",
                    "fileB.csv"
                };

                Parallel.ForEach(fileList, (string fileName) =>
                {
                    ConsoleLog("Processing File: " + fileName);
                    Thread.Sleep(10000); // 10 sec to process each file
                });

                GC.Collect();
            }
            else
                ConsoleLog("Sync Is Busy, Skipping Cycle");
        }
        finally
        {
            if (lockTaken)
                Monitor.Exit(_SyncLock);
        }
    }

    static void ConsoleLog(String Message)
    {
        Console.WriteLine("[{0}]: {1}",
            DateTime.UtcNow.ToString("HH:mm:ss tt"),
            Message);
    }
}

When it runs, it looks like this:

Does this look right? Any help/tips on improving this will be much appreciated.

Answer 1

It seems fine to me, apart from the fact that you don't need to start a task with Task.Factory.StartNew. The System.Threading.Timer already executes your callback on the ThreadPool, so there's no need to launch yet another task that will also be run on the thread pool.

Also, if your timer ticks every 5 seconds, and you expect it to take about 10 secs to process the files, then your threads will begin to queue up waiting for the lock to be released. That happened on the example you posted.

If this is the case, I would either increase the timer's period to more than 10 secs, or use Monitor.TryEnter instead of a regular lock. TryEnter will try to acquire the lock, and return immediately regardless of whether or not the lock was taken. If the lock is currently taken by another thread, you just skip this tick entirely.

---