Using OpenMP and Eigen causes infinite loop/deadlock

Question

I'm solving a much larger problem and have run into a bug when I try to use OpenMP to parallelize some loops. I've reproduced the problem with some simpler code below that mimics my own code.

The problem is that when I run the program, it will randomly go into some sort of infinite loop/deadlock (CPU is 100%, but doesn't do anything). From what I can tell from my testing, one of the threads attempts to compute the matrix-matrix product but never finishes for some reason.

I know that if you enable OpenMP, Eigen will parallelize matrix-matrix products using OpenMP . I'm also adding another parallel loop outside of this. However, this bug still occurs if I disable Eigen's parallelization by defining EIGEN_DONT_PARALLELIZE.

I'm using gcc version 4.6.0 20101127 on MacOS 10.6.8 with Eigen 3.0.4.

I can't figure out what could be going wrong...

#include <iostream>
#include <Eigen/Core>

using namespace std;
using namespace Eigen;

MatrixXd Test(MatrixXd const& F, MatrixXd const& G)
{
  MatrixXd H(F.rows(), G.cols());
  H.noalias() = F*G;

  return H;
}

int main()
{
  MatrixXd F = MatrixXd::Random(2,2);
  MatrixXd G = MatrixXd::Random(2,2);

  #pragma omp parallel for
  for (unsigned int i = 0; i < 10000; ++i)
    MatrixXd H = Test(F,G);

  cout << "Done!" << endl;
}

Answer 1

After some debugging, I think the problem is located in Eigen. In the file src/Core/products/GeneralBlockPanelKernel.h there is a function called manage_caching_sizes that declares two static variables:

static std::ptrdiff_t m_l1CacheSize = 0;
static std::ptrdiff_t m_l2CacheSize = 0;

Changing this to:

static std::ptrdiff_t m_l1CacheSize = 0;
static std::ptrdiff_t m_l2CacheSize = 0;
#pragma omp threadprivate(m_l1CacheSize, m_l2CacheSize)

fixed my problem.

Answer 2

I had the same problem, even with the most recent version of Eigen (3.0.5). I tried the fix proposed above and it is not possible with version 3.0.5, because of the new initializers. So I made the following change :

static std::ptrdiff_t m_l1CacheSize;
static std::ptrdiff_t m_l2CacheSize;
#pragma omp threadprivate(m_l1CacheSize, m_l2CacheSize)

if (m_l1CacheSize==0)
{
    m_l1CacheSize = manage_caching_sizes_second_if_negative(queryL1CacheSize(),8 * 1024);
    m_l2CacheSize = manage_caching_sizes_second_if_negative(queryTopLevelCacheSize(),1*1024*1024);
}

fixed my problem.

Answer 3

I was having the same problem using Microsoft Visual Studio 2010 SP1 PPL / parallel_for. The solution is described in

http://eigen.tuxfamily.org/dox/TopicMultiThreading.html

Using Eigen in a multi-threaded application

In the case your own application is multithreaded, and multiple threads make calls to Eigen, then you have to initialize Eigen by calling the following routine before creating the threads:

#include <Eigen/Core>

int main(int argc, char** argv)
{
  Eigen::initParallel();

  ...
}

In the case your application is parallelized with OpenMP, you might want to disable Eigen's own parallization as detailed in the previous section.