SqlDependency using BackgroundWorker

Question

I have a table in a SQL Server database that represents a log file of some actions inserted from a running windows service. Everything is working well.

But, I have a Windows application that gets the latest rows that have been inserted in the log table and views it in a DataGridView. While developing this application I depended on Using SqlDependency in a Windows Application from MSDN. It is working well, but when the log table receives a large number of log details, the Windows app hangs up and the main thread pool becomes too busy.

I want to run the same code referenced in the previous link in a separated thread pool by using Thread class or BackgroundWorker control. This means a thread for using UI controls and another one for listening to the database changes and get it into the DataGridView.

You can see the UI screenshot from this link "UI"

No. (1): This GroupBox represents the UI tools which users can use it while monitoring.

No. (2): The Start button is responsible for beginning to listen and receive updates from the database and refill the DataGridView.

No. (3): This grid represents the new logs that have been inserted in the database.

No. (4): This number (38 changes) represents the count of listening of sql dependency to the database changes.

My code: public partial class frmMain : Form { SqlConnection conn;

    const string tableName = "OutgoingLog";
    const string statusMessage = "{0} changes have occurred.";
    int changeCount = 0;

    private static DataSet dataToWatch = null;
    private static SqlConnection connection = null;
    private static SqlCommand command = null;

    public frmMain()
    {
        InitializeComponent();
    }

    private bool CanRequestNotifications()
    {
        // In order to use the callback feature of the
        // SqlDependency, the application must have
        // the SqlClientPermission permission.
        try
        {
            SqlClientPermission perm = new SqlClientPermission(PermissionState.Unrestricted);

            perm.Demand();

            return true;
        }
        catch
        {
            return false;
        }
    }

    private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
    {
        // This event will occur on a thread pool thread.
        // Updating the UI from a worker thread is not permitted.
        // The following code checks to see if it is safe to
        // update the UI.
        ISynchronizeInvoke i = (ISynchronizeInvoke)this;

        // If InvokeRequired returns True, the code
        // is executing on a worker thread.
        if (i.InvokeRequired)
        {
            // Create a delegate to perform the thread switch.
            OnChangeEventHandler tempDelegate = new OnChangeEventHandler(dependency_OnChange);

            object[] args = { sender, e };

            // Marshal the data from the worker thread
            // to the UI thread.
            i.BeginInvoke(tempDelegate, args);

            return;
        }

        // Remove the handler, since it is only good
        // for a single notification.
        SqlDependency dependency = (SqlDependency)sender;

        dependency.OnChange -= dependency_OnChange;

        // At this point, the code is executing on the
        // UI thread, so it is safe to update the UI.
        ++changeCount;
        lblChanges.Text = String.Format(statusMessage, changeCount);
        this.Refresh();

        // Reload the dataset that is bound to the grid.
        GetData();
    }

    private void GetData()
    {
        // Empty the dataset so that there is only
        // one batch of data displayed.
        dataToWatch.Clear();

        // Make sure the command object does not already have
        // a notification object associated with it.
        command.Notification = null;

        // Create and bind the SqlDependency object
        // to the command object.
        SqlDependency dependency = new SqlDependency(command);

        dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);

        using (SqlDataAdapter adapter = new SqlDataAdapter(command))
        {
            adapter.Fill(dataToWatch, tableName);

            dgv.DataSource = dataToWatch;
            dgv.DataMember = tableName;
            dgv.FirstDisplayedScrollingRowIndex = dgv.Rows.Count - 1;
        }
    }

    private void btnStart_Click(object sender, EventArgs e)
    {
        changeCount = 0;
        lblChanges.Text = String.Format(statusMessage, changeCount);

        // Remove any existing dependency connection, then create a new one.
        SqlDependency.Stop("<my connection string>");
        SqlDependency.Start("<my connection string>");

        if (connection == null)
        {
            connection = new SqlConnection("<my connection string>");
        }

        if (command == null)
        {
            command = new SqlCommand("select * from OutgoingLog", connection);
        }

        if (dataToWatch == null)
        {
            dataToWatch = new DataSet();
        }

        GetData();
    }

    private void frmMain_Load(object sender, EventArgs e)
    {
        btnStart.Enabled = CanRequestNotifications();
    }

    private void frmMain_FormClosing(object sender, FormClosingEventArgs e)
    {
        SqlDependency.Stop("<my connection string>");
    }
}

What I want exactly: when user click the Start button, the application run the code in a separated thread pool.

Answer 1

First if I understand well the change notification is already executed on another thread so using one more threading layer should be useless.

Indeed what causes the application to hang is the update of the UI, on the UI thread.

Could you show the code responsible for this update please?

If there is a lot of notifications visual update will be longer and you can't do much:

• update the grid by chunks to smooth the update: instead of inserting 1000 new records you run 10 updates of 100 records, but you take the risk of being overwhelmed by data if you don't process them fast enough

• using a collection that handles notification natively like a BindingList could help

Moreover what you can do to enhance the user experience, avoiding the unpleasant "it hangs" effect, is displaying a progress bar or a simple spinner.

UPDATE:

So if the first part of the GetData function is the bottleneck then indeed you could use another thread, e.g. from the thread-pool:

private void GetData()
{
    // Start the retrieval of data on another thread to let the UI thread free
    ThreadPool.QueueUserWorkItem(o =>
    {
        // Empty the dataset so that there is only
        // one batch of data displayed.
        dataToWatch.Clear();

        // Make sure the command object does not already have
        // a notification object associated with it.
        command.Notification = null;

        // Create and bind the SqlDependency object
        // to the command object.
        SqlDependency dependency = new SqlDependency(command);

        dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);

        using (SqlDataAdapter adapter = new SqlDataAdapter(command))
        {
            adapter.Fill(dataToWatch, tableName);

            // Update the UI
            dgv.Invoke(() =>
                {
                    dgv.DataSource = dataToWatch;
                    dgv.DataMember = tableName;
                    dgv.FirstDisplayedScrollingRowIndex = dgv.Rows.Count - 1;
                });
        }
    });
}

So the only part that will run on the UI thread is the update of the datagrid.

Not tested but hope this helps...

LAST? UPDATE:

With some synchronization to avoid concurrent execution:

AutoResetEvent running = new AutoResetEvent(true);

private void GetData()
{
    // Start the retrieval of data on another thread to let the UI thread free
    ThreadPool.QueueUserWorkItem(o =>
    {
        running.WaitOne();

        // Empty the dataset so that there is only
        // one batch of data displayed.
        dataToWatch.Clear();

        // Make sure the command object does not already have
        // a notification object associated with it.
        command.Notification = null;

        // Create and bind the SqlDependency object
        // to the command object.
        SqlDependency dependency = new SqlDependency(command);

        dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);

        using (SqlDataAdapter adapter = new SqlDataAdapter(command))
        {
            adapter.Fill(dataToWatch, tableName);

            running.Set();

            // Update the UI
            dgv.Invoke(() =>
                {
                dgv.DataSource = dataToWatch;
                dgv.DataMember = tableName;
                dgv.FirstDisplayedScrollingRowIndex = dgv.Rows.Count - 1;
                });
        }
    });
}