Move record insert to SQL Server CE database into another thread? C# Compact Framework 3.5

Question

I had some trouble trying to place piece of code into another thread to increase performance.

I have following code below (with thread additions with comments), where I parse large XML file (final goal 100,000 rows) and then write it to a SQL Server CE 3.5 database file (.sdf) using record and insert (SqlCeResultSet/SqlCeUpdatableRecord).

Two lines of code in if statement inside the while loop,

xElem = (XElement)XNode.ReadFrom(xmlTextReader);

and

rs.Insert(record);

take about the same amount of time to execute. I was thinking to run rs.Insert(record); while I am parsing the next line of xml file. However, I still was unable to do it using either Thread or ThreadPool.

I have to make sure that the record that I pass to thread is not changed until I finish executing rs.Insert(record); in existing thread. Thus, I tried to place thread.Join() before writing new record (record.SetValue(i, values[i]);), but I still get conflict when I try to run the program - program crashes with bunch of errors due to trying to write identical row several times (especially for index).

Can anyone help me with some advise? How can I move rs.Insert(record); into another thread to increase performance?

XmlTextReader xmlTextReader = new XmlTextReader(modFunctions.InFName);
XElement xElem = new XElement("item");

using (SqlCeConnection cn = new SqlCeConnection(connectionString))
{
    if (cn.State == ConnectionState.Closed)
        cn.Open();

    using (SqlCeCommand cmd = new SqlCeCommand())
    {
        cmd.Connection = cn;
        cmd.CommandText = "item";
        cmd.CommandType = CommandType.TableDirect;

        using (SqlCeResultSet rs = cmd.ExecuteResultSet(ResultSetOptions.Updatable))
        {
            SqlCeUpdatableRecord record = rs.CreateRecord();

            // Thread code addition
            Thread t = new Thread(new ThreadStart(() => rs.Insert(record));

            while (xmlTextReader.Read())
            {
                if (xmlTextReader.NodeType == XmlNodeType.Element &&
                    xmlTextReader.LocalName == "item" &&
                    xmlTextReader.IsStartElement() == true)
                {
                    xElem = (XElement)XNode.ReadFrom(xmlTextReader);

                    values[0] = (string)xElem.Element("Index"); // 0
                    values[1] = (string)xElem.Element("Name"); // 1
                    ~~~
                    values[13] = (string)xElem.Element("Notes"); // 13

                    // Thread code addition -- Wait until previous thread finishes
                    if (ThreadStartedS == 1)
                    {
                        t.Join()
                    }

                    // SetValues to record
                    for (int i = 0; i < values.Length; i++)
                    {
                        record.SetValue(i, values[i]); // 0 to 13
                    }

                    // Thread code addition -- Start thread to execute rs.Insert(record)
                    ThreadStartedS = 1;
                    t.Start();

                    // Original code without threads
                    // Insert Record
                    //rs.Insert(record);
                }
            }
        }
    }
}

Answer 1

If all of your processing is going to be done on the device (reading from the XML file on the device then parsing the data on the device), then you will see no performance increase from threading your work.

These Windows Mobile devices only have a single processor, so for them to multithread means one process works for a while, then another process works for a while. You will never have simultaneous processes running at the same time.

On the other hand, if the data from your XML file were located on a remote server, you could call the data in chunks. As a chunk arrives, you could process that data in another thread while waiting on the next chunk of data to arrive in the main thread.

If all of this work is being done on one device, you will not have good luck with multithreading.

You can still display a progress bar (from 0 to NumberOfRecords) with a cancel button so the person waiting for the data collection to complete does not go insane with anticipation.