Best way to update cmdlet progress from a separate thread

Question

I have a powershell cmdlet written in C# (deriving from PSCmdlet) which will start a long-running task which should update its progress using WriteProgress() while it is running. Since powershell will not allow a separate thread to use WriteObject or WriteProgress I had to create a Queue<object> in the main thread and I add items to the queue from the task that I want to be written to the Pipeline/Progress. A while loop will dequeue objects as they come in and write to the pipline / progress bar.

This is working, but I wanted to see if there were any better practices for multi-threading with a powershell cmdlet that is written in C#/VB. For example with WPF I can always step onto the UI thread with UIComponent.Dispatcher.Invoke() if I need to update a progress bar or UI Component. Is there anything equivalent that I can use to 'step onto' the powershell thread to update the UI or write to the pipeline?

Answer 1

Here is an example of the queue system encapsulated in a class so it is easier to use and mimics Cmdllet.WriteObject's behavior. This way you can call WriteObject from within the separate thread and the object will be marshalled onto the powershell thread and written to the pipeline.

[Cmdlet("Test", "Adapter")]
public class TestCmdlet : PSCmdlet
{
    protected override void ProcessRecord()
    {
        PowerShellAdapter adapter = new PowerShellAdapter(this, 100);
        Task.Factory.StartNew(() => {
            for (int x = 0; x < 100; x++) {
                adapter.WriteObject(x);
                Thread.Sleep(100);
            }
            adapter.Finished = true;
        });
        adapter.Listen();
    }
}   

---

public class PowerShellAdapter
{
    private Cmdlet Cmdlet { get; set; }
    private Queue<object> Queue { get; set; }
    private object LockToken { get; set; }
    public bool Finished { get; set; }
    public int Total { get; set; }
    public int Count { get; set; }

    public PowerShellAdapter(Cmdlet cmdlet, int total)
    {
        this.Cmdlet = cmdlet;
        this.LockToken = new object();
        this.Queue = new Queue<object>();
        this.Finished = false;
        this.Total = total;
    }

    public void Listen()
    {
        ProgressRecord progress = new ProgressRecord(1, "Counting to 100", " ");
        while (!Finished || Queue.Count > 0)
        {
            while (Queue.Count > 0)
            {
                progress.PercentComplete = ++Count*100 / Total;
                progress.StatusDescription = Count + "/" + Total;
                Cmdlet.WriteObject(Queue.Dequeue());
                Cmdlet.WriteProgress(progress);
            }

            Thread.Sleep(100);
        }
    }

    public void WriteObject(object obj)
    {
        lock (LockToken)
            Queue.Enqueue(obj);
    }
}

Answer 2

The answer provided by Despertar will work, but it can be improved on slightly.

Polling in a loop with Thread.Sleep should be replaced with use of an AutoResetEvent. This will cause the main thread to only "wake up" when there is actually data available, and can allow the cmdlet to complete faster than 100ms. The Thread.Sleep will always cause the cmdlet to take at least 100ms, even if it could run must faster. This might not be a problem if you have a simple cmdlet, but if you insert it into a complex pipeline this 100ms can easily multiply and cause things to run very slowly. Additionally, a lock should be taken when accessing the Queue on the main thread inside the Listen method.

The moral of the story: if you do cross-thread synchronization Thread.Sleep is not the right tool.

using System.Threading;
public class PowerShellAdapter
{
    private Cmdlet Cmdlet { get; set; }
    private Queue<object> Queue { get; set; }
    AutoResetEvent sync;
    private object LockToken { get; set; }
    // volatile, since it will be written/read from different threads.
    volatile bool finished;
    public bool Finished
    {
        get { return finished; }
        set
        {
            this.finished = value;
            // allow the main thread to exit the outer loop.
            sync.Set();
        }
    }
    public int Total { get; set; }
    public int Count { get; set; }

    public PowerShellAdapter(Cmdlet cmdlet, int total)
    {
        this.Cmdlet = cmdlet;
        this.LockToken = new object();
        this.Queue = new Queue<object>();
        this.finished = false;
        this.Total = total;
        this.sync = new AutoResetEvent(false);
    }

    public void Listen()
    {
        ProgressRecord progress = new ProgressRecord(1, "Counting to 100", " ");
        while (!Finished)
        {
            while (true) { // loop until we drain the queue
                object item;
                lock (LockToken) {
                    if (Queue.Count == 0)
                        break; // exit while
                    item = Queue.Dequeue();
                }

                progress.PercentComplete = ++Count * 100 / Total;
                progress.StatusDescription = Count + "/" + Total;
                Cmdlet.WriteObject(item);
                Cmdlet.WriteProgress(progress);
            }
            sync.WaitOne();// wait for more data to become available
        }
    }

    public void WriteObject(object obj)
    {
        lock (LockToken)
        {
            Queue.Enqueue(obj);
        }
        sync.Set(); // alert that data is available
    }
}

Note, I haven't actually tested this code, but it illustrates the idea.

Answer 3

You could take a look at the Start-Job cmdlet together with Get-Job, Wait-Job and Receive-Job.

Start-Job will effectively start a new thread and output a JobId which you can query with Receive-Job to get the output. You could then loop through all currently running jobs and update your progress bar.

Take a look at http://blogs.technet.com/b/heyscriptingguy/archive/2012/08/10/use-background-jobs-to-run-a-powershell-server-uptime-report.aspx