c++: MPI communicator as global variable

Question

I need the MPI world communicator to be accessible in functions/class member functions. But by design/convention, MPI environments and communicators are always defined and initialized at the beginning of int main().

The only, simple solution I can think of using a global pointer to the communicator.

Does anybody know of a better way? Is it dangerous to use the global pointer solution?

This problem applies equally well to bare-bones MPI, and to Boost::MPI (which I use below)

Example of my proposed solution (untested):

//globals.h
extern   boost::mpi::communicator  * my_MPI_world_ptr;

and

//main.cpp
...
int main(int argc, char* argv[])
{        
    boost::mpi::environment my_boost_mpi_env(argc, argv);
    boost::mpi::communicator my_MPI_world; 
    my_MPI_world_ptr = &my_MPI_world;       

    my_MPI_rank = my_MPI_world_ptr->rank();
    size_MPI_WORLD = my_MPI_world_ptr->size();    

    my_class an_Object;
    an_Object.member_function_that_uses__MPI_world();
   ...
}

Answer 1

Do you mean the actual MPI MPI_COMM_WORLD communicator (or the Boost wrapper of it)? That is already global. If you are using a different communicator to separate communication from a library that you are writing, it would be better to avoid using a global variable for it at all. In that case, you might want to just pass it (or a pointer to it) around and store it in the classes that need it.

Answer 2

I don't like global pointers in general: who's responsible for deleting them? How do you make sure the pointer is not accessed before the object is created, or after the object is destroyed?

I'd be tempted to wrap the pointer and its access in a class. (Warning: The following hasn't seen a compiler so might have all sorts of issues, and I'm not familiar with MPI)

class CMPIwrapper
{
public:
    CMPIwrapper(boost::mpi::communicator& myMPIworld):myMPIworld_(myMPIworld){}
    rank_type GetRank()const
    {
        return( my_MPI_world_ptr->rank() );
    }
private:
    boost::mpi::communicator& myMPIworld_;
};    

int main(int argc, char* argv[])
{        
    boost::mpi::environment my_boost_mpi_env(argc, argv);
    boost::mpi::communicator my_MPI_world; 
    CMPIwrapper my_MPI_wrapper(my_MPI_world);       

    my_MPI_rank = CMPIwrapper.GetRank();
}

Your own objects that used to use the pointer could work in the same way: their constructor could be passed a reference to your boost::mpi::communicator, or if the set of operations on your boost::mpi::communicator are well defined, they could be passed a reference to the wrapper.

Answer 3

1. For boost mpi, the default constructed (i.e., empty initializer) communicator corresponds to MPI_COMM_WORLD, so you could simply define another

   mpi::communicator world;
   

   inside your function use it as if it is the one you defined outside.

2. MPI_INIT is called when constructing mpi::environment. So as long as this is placed in the beginning of the of your main program, you are free to define a global mpi::communicator somewhere else. It is not necessary to use a pointer. (Actually you can even place MPI_INIT somewhere else as well, see below).

3. For bare-bones MPI, I have tested that calling MPI_INIT elsewhere besides the main is also allowed. For example, you can define the following wrapper for a global worker in a header file,

       class MpiWorker_t
       {
       public:
         int  NumberOfWorkers, WorkerId, NameLen;
         char HostName[MPI_MAX_PROCESSOR_NAME];
         MpiWorker_t()
         {
               MPI_Init(NULL, NULL);
               MPI_Comm_size(MPI_COMM_WORLD,&NumberOfWorkers);
               MPI_Comm_rank(MPI_COMM_WORLD,&WorkerId);
               MPI_Get_processor_name(HostName, &NameLen);
         }
         ~MpiWorker_t()
         {
               MPI_Finalize();
         }
       }
   
      extern MpiWorker_t GlobalWorker;
   

   and in the source file, define a global instance at any place outside main():

     MpiWorker_t GlobalWorker;
   

   The construction and destruction of the global variable should be able to take care of MPI initialization and finalization, before and after any MPI function call.