Using Boost MPI for sending files?

Question

Can I use the Boost MPI to send files to other computers? My worry is that it is optimized for very small messages and not larger (I need to send 700 MB or more in one go). The reason is that I will probably use mpi for other parts of the program anyway so I thought it would be a pretty clean solution to not mix and match.

Has it been done? Would you recommend it?

Answer 1

Keep in mind that Boost MPI requires that you already have an MPI implementation on the systems you want to send files with (OpenMPI, MPICH2, MVAPICH2, etc). You'll need to get one of these to go along with Boost MPI.

Also keep in mind that MPI is designed for high performance clusters, and most MPI implementations assume that you have reliable hardware. That is, if you've written an application in MPI and one of your processes goes down, more than likely they're all going to die.

If you're planning to use this in a local environment on your own network, and you intend to run mostly batch jobs and not persistent services, then MPI might be a good choice. If not, then look into some other messaging solution designed for a more distributed, less reliable environment.

Answer 2

MPI was designed for high-performance parallel applications, so I would absolutely recommend it. (Keep in mind that all Boost libraries that have been accepted are thoroughly vaildated and tested

The 700mb files should not be any problem according to the reference documentation

Another (hopefully interesting) feature: "Boost.MPI processes can be spread across many different machines, possibly with different operating systems and underlying architectures."