Open MPI ranks are not in order

Question

When i run an "Open MPI" program, it generally assigns ranks in random order I want to know is there a way to always assign ranks in order?

So instead of this

Hello, World. I am 2 of 3
Hello, World. I am 0 of 3
Hello, World. I am 1 of 3 

can i get this

Hello, World. I am 0 of 3
Hello, World. I am 1 of 3
Hello, World. I am 2 of 3 

EDIT

here is the code

    PROGRAM hello
INCLUDE 'mpif.h'
INTEGER*4 :: numprocs, rank, ierr
CALL MPI_INIT(ierr)
CALL MPI_COMM_SIZE(MPI_COMM_WORLD, numprocs, ierr)
CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr)
write(*,*) 'Hello World. I am', rank, 'of', numprocs
CALL MPI_FINALIZE(ierr)
    END PROGRAM hello

I have tested it on i5 processor (4 threads) when i run

    mpirun -np 4 myprog

it works as i want it to, ranks printed in order 0-3, otherwise (like with 3 as shown above) it just wont do it (tested it like 100 time)

Answer 1

The order in which ranks are assigned is never random with most MPI implementations and usually there are mechanisms to precisely control it. What is random is the order in which the output from the different ranks arrives at the MPI launcher (mpirun, mpiexec, etc.) through the IO redirection mechanism. Due to the buffering that's usually involved, one could never be sure that if e.g. rank 0 outputs some text before rank 1, then the output from rank 0 would necessarily arrive before the output from rank 1. The only portable way to guarantee ordering of the text output is to have all ranks channel their messages through a single rank that does IO.

With some implementations it might be possible to do something like linear token passing or a sequence of barriers, e.g.:

int rank, size;

MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

for (int i = 0; i < size; i++)
{
   if (i == rank)
      printf("Hello world! I am %d of %d\n", rank, size);
   MPI_Barrier(MPI_COMM_WORLD);
}

The rationale behind such code is that MPI_Barrier could possibly progress pending communication operations before it completes, including those that carry the redirected standard output. Still there is no guarantee that the output from printf() is immediately shown in the console output of mpirun/mpiexec.

Answer 2

Many mpi implementations will let you prefix the output with the rank of that process. then you can sort it after the fact. In MPICH, this is --prepend-rank or -l. In OpenMPI it is --tag-output