Feedback for Excel Automation and execution of long running macro

Question

I have the most god-awful task of automating an excel workbook in C#. Part of the process is to call a macro in the workbook which takes around 2 minutes to execute. This all happens in a Windows service and happily runs from start to finish. At present it writes events to a database table just before calling the macro and when the macro has finished executing. It does a lot of calculations and exporting data to a text file within the macro code.

As it takes so long the users have asked if they can be notified at various parts of the process.

My initial thought with reservation was to periodically poll the Application.StatusBar which gets update whilst running the macro using System.Timers.Timer. I thought there might be some kind of thread issues with this - which I think is happening as the call from the timer to get the StatusBar does not return/complete for quite a large period of time (tens of seconds).

I have my workbook wrapped up in the following class which makes sure Excel closes correctly and runs the macro:

internal class myWorkbook : IDisposable
{
    private Microsoft.Office.Interop.Excel.Application app = null;
    private Microsoft.Office.Interop.Excel.Workbook myWorkbook = null;
    private string _myWorkbookUri;

    public myWorkbook(string myWorkbookUri, string)
    {
        _myWorkbookUri = myWorkbookUri;
    }

    public string Export(DateTime date)
    {
        app = new Microsoft.Office.Interop.Excel.Application();
        app.Visible = false;
        app.DisplayAlerts = false;
        app.Interactive = false;
        app.AskToUpdateLinks = false;
        myWorkbook = app.Workbooks.Open(_myWorkbookUri, 0, true, 5, "", "", false, Microsoft.Office.Interop.Excel.XlPlatform.xlWindows, "", true, false, 0, true, true, false);

        return (string)app.Run("GenerateTextFile", date);       
    }

    /// <summary>
    /// Disposes the object instance and all unmanaged resources.
    /// </summary>
    void IDisposable.Dispose()
    {
        if (myWorkbook != null)
        {
            myWorkbook.Close(false);
            System.Runtime.InteropServices.Marshal.ReleaseComObject(myWorkbook);
        }

        if (app != null)
        {
            app.Quit();
            System.Runtime.InteropServices.Marshal.ReleaseComObject(app);
        }
    }

    public string Status
    {
        get
        {
            if (myWorkbook == null)
                return string.Empty;
            else
            {   
                return myWorkbook.Application.StatusBar.ToString();
            }
        }
    }
}

Then I have tried to do the monitoring using the following nested/internal class in my report processing class:

private class MyMonitor : System.Timers.Timer
{
    private MyWorkbook _wb;
    private ReportGeneratorProcess _parent;
    private string _lastStatus = string.Empty;
    private bool handlingTimer = false;

    public MyMonitor(MyWorkbook wb, ReportGeneratorProcess parent)
    {
        _wb = wb;
        _parent = parent;
        this.AutoReset = true;
        this.Elapsed += new System.Timers.ElapsedEventHandler(this.timer_Elapsed);                
    }

    private void timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
    {
        if (_wb != null && !handlingTimer)
        {
            try
            {
                handlingTimer = true;
                string status = _wb.Status;
                if (status != _lastStatus && status.ToLower() != "false")
                    _parent.AddEvent(MSG_TITLE_RUN_My, status);

                _lastStatus = status;
            }
            finally
            {
                handlingTimer = false;
            }
        }
    }
}

This is all triggered by executing the following code in the ReportGeneratorProcess class (I've omitted the unnecessary code)

string outputFilename = null;
MyMonitor monitor = null;

try
{
    using (MyWorkbook wb = new MyWorkbook(_MyWorkbookUri))
    {
        monitor = new MyMonitor(wb, this);
        monitor.Start();
        outputFilename = wb.Export(month);
        monitor.Stop();
    }

    AddEvent("My Complete", "Generated file " + outputFilename);
    return outputFilename;
}
catch (Exception ex)
{
    AddEvent("", "failed");
    throw ex;
}
finally
{
    if (monitor != null)
    {
        monitor.Stop();
    }
}

AddEvent simply adds a new event to the database using the main class.

I remain defeated at the moment trying to think of an alternative solution/nice way around this. Anyone got any hints?

Unfortunately the process HAS to operate as is. There is no scope to move anything out of excel.

Answer 1

Basically I modified the workbook slightly to log to a text file. The text file was then monitored by my process using a FileSystemWatcher.

The workbook creates the log file locally on the server. I've had issues in the past with FileSystemWatcher where it monitors a unc path and the network appears to go down causing it to stop working so I was confident that this would be ok.

I think FileSystemWatcher raises the Changed event on a separate thread (not verified) as my service runs the Export() function on a timer thread which would be blocked until the excel macro has finished executing.

As it turns out, the service is happily running and monitoring the file.

internal class myWorkbook : IDisposable
{
    private Microsoft.Office.Interop.Excel.Application app = null;
    private Microsoft.Office.Interop.Excel.Workbook myWorkbook = null;
    private string _myWorkbookUri;
    private FileSystemWatcher watcher;
                private bool _visible;
                private string _logFile;

    public myWorkbook(string myWorkbookUri, string)
    {
        _myWorkbookUri = myWorkbookUri;
    }

    public string Export(DateTime date)
    {
        app = new Microsoft.Office.Interop.Excel.Application();
        app.Visible = false;
        app.DisplayAlerts = false;
        app.Interactive = false;
        app.AskToUpdateLinks = false;
        myWorkbook = app.Workbooks.Open(_myWorkbookUri, 0, true, 5, "", "", false, Microsoft.Office.Interop.Excel.XlPlatform.xlWindows, "", true, false, 0, true, true, false);
        CreateWatcher();

        return (string)app.Run("GenerateTextFile", date);       
    }


    private void CreateWatcher()
    {
        watcher = new FileSystemWatcher();
        watcher.Path = Path.GetDirectoryName(_logFile);
        watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.FileName | NotifyFilters.DirectoryName;
        watcher.Filter = "*.txt"; // could use _logFile;
        watcher.Changed += watcher_Changed;
        watcher.EnableRaisingEvents = true;
    }

    void watcher_Changed(object sender, FileSystemEventArgs e)
    {
        if (e.FullPath == _logFile)
        {
            using (FileStream fs = new FileStream(_logFile,
                                  FileMode.Open,
                                  FileAccess.Read,
                                  FileShare.ReadWrite))
            {
                using (StreamReader sr = new StreamReader(fs))
                {
                    while (sr.Peek() >= 0)
                    {
                        var line = sr.ReadLine();
                        var tabs = line.Split(new char[] { '\t' });
                        DateTime eventTime = DateTime.Parse(tabs[0]);

                        if (DateTime.Compare(eventTime, lastEventTime) > 0)
                        {
                            string eventMessage = tabs[1];
                            OnProgressChanged(new ProgressChangedEventArgs(eventTime, eventMessage));

                            lastEventTime = eventTime;
                        }
                    }
                }
            }
        }
    }

}

Answer 2

I've done this before. Create a DB table to where you update your long running job status. Your application sends request to start this job, and should create an Id for this job. Your application starts to monitor this job on a timer. Your win service picks up this job and starts to update your record and your application will now show this update to the user. Your win service passes handle to the Excel automation, which now responsible for updating long running job record. Your client app will reflect changes to the user. Once processing completed, your client should act correspondingly. A message, a pop-up, etc. will notify user of completed or failed job.

Also, many developers create win service that constantly runs on a timer. One way to do it more efficient is to create a listener. So the win service will listen to some TCP port and your app will signal to that port. Win service will wake up and start processing jobs.