Process files in dynamically created directories

Question

I have searched extensively without success to the following problem.

Processing a large text file in C# line by line and creating text messages and sending to a JMS queue is taking more time than allowed. I am breaking the large file in to smaller files and running multiple instances of the text message conversion program.

The large text file varies in size so the number of smaller files created will not always be the same. I am dynamically creating a directory to hold each of the smaller text files, so that each instance of the text message conversion program will have a separate directory to work from.

I am able to do the processing mentioned above, but how can I get an instance of my text message conversion program to run and process each small file held in the dynamically created directories, one instance for each directory. I plan to call the text message conversion program from the program that creates the smaller files and places them in the dynamically created directories.

Thanks

Greg

Answer 1

Loop over the root directory and for each directory in that folder, start a process, so something like this:

foreach (var dir in Directory.GetDirectories("z:\temp"))
{
    ProcessStartInfo psi = new ProcessStartInfo();
    psi.Arguments = String.Format("-dir=\"{0}\"", dir);
    psi.FileName = @"C:\Path\To\Your\Converter.exe";

    Process.Start(psi);
}

Answer 2

I would suggest using threads instead of separate instances. Threads are light weight than process and since the work done is independent of each other you can use parallel available in .net 4

using System.Threading;

using System.Threading.Tasks;

string strParentDir = @"C:\ProgramData\Text2Msg\";
string[] strDirectories = Directory.GetDirectories(strParentDir)); 
Parallel.ForEach (string strDir in strDirectories)  
{  
             Txt2MsgConverter(strDir);
} 

public void Txt2MsgConverter(string strDirName)
{
   //your text to message code

}