Copying table using Parallel causes deadlock on MySQL

Question

I'm making a program that combines all tables' data from db1 to one table in db2. But it gives deadlock error.

Here is my code.

Parallel.ForEach(tableList, table =>
{
    using (MySqlConnection conn = new MySqlConnection(DBconnection))
    {
        using (MySqlCommand cmd = new MySqlCommand(query, conn))
        {
            try
            {
                conn.Open();
                cmd.CommandText = "INSERT INTO db2.targetTable (id, value1, value2) " +
                "SELECT (SELECT id FROM db2.User u WHERE u.id = user LIMIT 1), " +
                "value1, value2 FROM db1." + table;
                cmd.CommandTimeout = 600000;
                cmd.ExecuteNonQuery();
            }
            catch (Exception ex)
                Console.WriteLine("ERROR : " + ex.Message);
        }
    }
}

I think the subquery causes the problem.

Maybe if I add this code, it can solve the problem:

  Object sync = new Object();
  lock (sync)
  {
     cmd.ExecuteNonQuery();
  }

But I think it is not efficient. Is this a right solution? or is there better solution, please give me an example code.

Answer 1

It's not clear what's causing the deadlock. (Note that SQL operations involving MyISAM tables attempt to obtain exclusive locks on the entire table, so I'm assuming you are using InnoDB, or some other storage engine other than MyISAM.)

I'd avoid the correlated subquery in the SELECT list, and use a JOIN pattern instead. I believe that with a subquery in the SELECT list, MySQL will execute that subquery for every row returned in the outer query.

INSERT INTO db2.targetTable (id, value1, value2) 
SELECT u.id
     , t.value1
     , t.value2 
  FROM db1.table t
  LEFT
  JOIN db2.User u
    ON u.id = t.user

The subquery in your statement includes a LIMIT 1 clause. If that is needed because the id column in the db2.User table is not unique (which would be odd, but possible) then the query I gave above would need to include a GROUP BY <db1.table1 PRIMARY KEY>, to avoid creating duplicate rows from a source row.

If the db1.table has a PRIMARY KEY of id, then:

INSERT INTO db2.targetTable (id, value1, value2) 
SELECT u.id
     , t.value1
     , t.value2 
  FROM db1.table t
  LEFT
  JOIN db2.User u
    ON u.id = t.user
 GROUP BY t.id