compiling an OpenMPI app with the Intel compiler suite
https://stackoverflow.com/questions/11919822
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
I have the latest OpenMPI on linux which I compiled with the Intel compiler suite. I am trying to compile an OpenMPI application. My particular application uses RInside and RcppEigen. If I comment out the openMPI parts of my code, the compile string is:
icpc -I/usr/share/R/include -I/usr/lib/R/site-library/Rcpp/include -I/usr/local/lib/R/site-library/RInside/include -O3 -pipe -g -Wall -I/usr/local/lib/R/site-library/RcppEigen/include sjb_simple_smle_with_Rinside.cpp -L/usr/lib/R/lib -lR -lblas -llapack -L/usr/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/lib/R/site-library/Rcpp/lib -L/usr/local/lib/R/site-library/RInside/lib -lRInside -Wl,-rpath,/usr/local/lib/R/site-library/RInside/lib -o sjb_simple_smle_with_Rinside
Thus, I tried to compile with mpic++ using:
mpic++ -I/usr/share/R/include -I/usr/lib/R/site-library/Rcpp/include -I/usr/local/lib/R/site-library/RInside/include -O3 -pipe -g -Wall -I/usr/local/lib/R/site-library/RcppEigen/include sjb_simple_smle_with_Rinside.cpp -L/usr/lib/R/lib -lR -lblas -llapack -L/usr/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/lib/R/site-library/Rcpp/lib -L/usr/local/lib/R/site-library/RInside/lib -lRInside -Wl,-rpath,/usr/local/lib/R/site-library/RInside/lib -o sjb_simple_smle_with_Rinside
If I uncomment
#include "mpi.h"
and try to compile, I get the errors below. Is there any trick to compiling OpenMPI applications with the Intel compilers? The full source is listed below:
stevejb@ursamajor:~/Projects/big_data_sim_mle/simple_smle/R_inside_version$ mpic++ -I/usr/share/R/include -I/usr/lib/R/site-library/Rcpp/include -I/usr/local/lib/R/site-library/RInside/include -O3 -pipe -g -Wall -I/usr/local/lib/R/site-library/RcppEigen/include sjb_simple_smle_with_Rinside.cpp -L/usr/lib/R/lib -lR -lblas -llapack -L/usr/lib/R/site-library/Rcpp/lib -lRcpp -Wl,-rpath,/usr/lib/R/site-library/Rcpp/lib -L/usr/local/lib/R/site-library/RInside/lib -lRInside -Wl,-rpath,/usr/local/lib/R/site-library/RInside/lib -o sjb_simple_smle_with_Rinside -shared-intel -I/usr/local/include -pthread -L/usr/local/lib -lmpi_cxx -lmpi -ldl -lm -Wl,--export-dynamic -lrt -lnsl -lutil
/usr/local/include/openmpi/ompi/mpi/cxx/datatype.h(142): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/datatype.h(142): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/datatype.h(142): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/datatype.h(142): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/datatype.h(142): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/op.h(48): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/op.h(48): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/op.h(48): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/op.h(48): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/op.h(48): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/request.h(96): error: expected a type specifier
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/request.h(96): error: expected a ")"
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/request.h(96): error: expected an identifier
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/request.h(96): error: "virtual" is not allowed
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/request.h(96): error: expected a ";"
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/group.h(111): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/group.h(111): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/group.h(111): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/group.h(111): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/group.h(111): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/comm.h(264): error: expected a type specifier
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/comm.h(264): error: expected a ")"
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/comm.h(264): error: expected an identifier
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/comm.h(264): error: "virtual" is not allowed
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/comm.h(264): error: expected a ";"
virtual void Free(void);
^
/usr/local/include/openmpi/ompi/mpi/cxx/win.h(118): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/win.h(118): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/win.h(118): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/win.h(118): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/win.h(118): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/errhandler.h(59): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/errhandler.h(59): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/errhandler.h(59): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/errhandler.h(59): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/errhandler.h(59): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/info.h(84): error: expected a type specifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/info.h(84): error: expected a ")"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/info.h(84): error: expected an identifier
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/info.h(84): error: "virtual" is not allowed
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/info.h(84): error: expected a ";"
virtual void Free();
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(39): error: expected an identifier
MPI::Request::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(38): error: inline specifier allowed on function declarations only
inline void
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(39): error: incomplete type is not allowed
MPI::Request::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(39): error: a nonstatic member reference must be relative to a specific object
MPI::Request::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(39): error: expected an expression
MPI::Request::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(39): error: expected a ")"
MPI::Request::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/request_inln.h(40): error: expected a ";"
{
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(123): error: identifier "ompi_op_set_cxx_callback" is undefined
ompi_op_set_cxx_callback(mpi_op, (MPI_User_function*) func);
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(128): error: expected an identifier
MPI::Op::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(127): error: inline specifier allowed on function declarations only
inline void
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(128): error: incomplete type is not allowed
MPI::Op::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(128): error: a nonstatic member reference must be relative to a specific object
MPI::Op::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(128): error: expected an expression
MPI::Op::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(128): error: expected a ")"
MPI::Op::Free()
^
/usr/local/include/openmpi/ompi/mpi/cxx/op_inln.h(129): error: expected a ";"
{
^
sjb_simple_smle_with_Rinside.cpp(35): warning #12: parsing restarts here after previous syntax error
using namespace Rcpp;
^
sjb_simple_smle_with_Rinside.cpp(77): error: identifier "as" is undefined
const Map<MatrixXd> Xmat(as<Map<MatrixXd> >(Xmat_sexp));
^
sjb_simple_smle_with_Rinside.cpp(77): error: type name is not allowed
const Map<MatrixXd> Xmat(as<Map<MatrixXd> >(Xmat_sexp));
^
sjb_simple_smle_with_Rinside.cpp(80): error: type name is not allowed
const Map<MatrixXd> Ymat(as<Map<MatrixXd> >(Ymat_sexp));
^
sjb_simple_smle_with_Rinside.cpp(87): error: type name is not allowed
const Map<MatrixXd> ua(as<Map<MatrixXd> >(ua_sexp));
^
sjb_simple_smle_with_Rinside.cpp(89): error: type name is not allowed
const Map<MatrixXd> ub(as<Map<MatrixXd> >(ub_sexp));
^
sjb_simple_smle_with_Rinside.cpp(98): error: type name is not allowed
const Map<VectorXd> start_vector(as<Map<VectorXd> >(start_sexp));
^
compilation aborted for sjb_simple_smle_with_Rinside.cpp (code 2)
FULL SOURCE CODE:
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4; tab-width: 8; -*-
//
// SJB - first shot at RInside and Eigen combined
//
// Copyright (C) 2012 Stephen J. Barr
//
// GPL'ed
#include <iostream>
#include <sstream>
#include <iomanip>
#include <fstream>
#include "mkl.h"
#include "math.h"
#include <vector>
#include <cmath>
#include <string>
#include <cstdlib>
#include <fcntl.h>
#include <sys/stat.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <getopt.h>
#include <map>
#include <utility>
#include <RcppEigen.h>
#include <Rcpp.h>
#include <RInside.h> // for the embedded R via RInside
#include "mpi.h"
using namespace Rcpp;
using namespace Eigen;
using namespace std;
int main(int argc, char *argv[]) {
const int N_TRU_PARAMS = 5;
const int n = 1000;
const int t = 100;
const int nr = 500;
RInside R(argc, argv); // create an embedded R instance
stringstream ss;
ss << "n = " << n << "; t = " << t << ";" << " nr = " << nr << ";";
cout << ss.str() << endl;
R.parseEval(ss.str());
VectorXd tru = VectorXd(N_TRU_PARAMS);
tru << 2,1,-2,1,1;
// R.parseEval("n = 100;");
// R.parseEval("t = 100;");
R.parseEval("set.seed(123)");
string cmdstr = "tru = c(2,1,-2,1,1);"
"set.seed(123);"
"a = rnorm(n,tru[1],tru[2]);"
"b = rnorm(n,tru[3],tru[4]);"
// Make some data lists
"x = NULL;"
"y = NULL;"
// Generate some data for each firm
"for(i in 1:n) {"
" x[[i]] = rnorm(t,1,1);"
" y[[i]] = a[i]+b[i]*x[[i]]+ rnorm(t)*tru[5];"
"}";
R.parseEval(cmdstr);
SEXP Xmat_sexp = R.parseEval("Xmat = do.call(rbind, x)");
const Map<MatrixXd> Xmat(as<Map<MatrixXd> >(Xmat_sexp));
SEXP Ymat_sexp = R.parseEval("Ymat = do.call(rbind, y)");
const Map<MatrixXd> Ymat(as<Map<MatrixXd> >(Ymat_sexp));
cout << "X: " << Xmat.rows() << " x " << Xmat.cols() << endl;
cout << "Y: " << Ymat.rows() << " x " << Ymat.cols() << endl;
// Make matrices ua and ub
SEXP ua_sexp = R.parseEval( "ua = matrix(rnorm(nr*n),n,nr)");
const Map<MatrixXd> ua(as<Map<MatrixXd> >(ua_sexp));
SEXP ub_sexp = R.parseEval( "ub = matrix(rnorm(nr*n),n,nr)");
const Map<MatrixXd> ub(as<Map<MatrixXd> >(ub_sexp));
cmdstr = "track = 0;"
"Y = unlist(y);"
"X = unlist(x);"
"start = lm(Y~X)$coeff;"
"start =c(start[1],0,start[2],0);";
R.parseEval(cmdstr);
SEXP start_sexp = R.parseEval("start");
const Map<VectorXd> start_vector(as<Map<VectorXd> >(start_sexp));
cout << "Starting point: " << start_vector.transpose() << endl;
// THERE ARE TWO PAIRS OF FUNCTIONS, smle_init and smle_nll_mpi
// smle_init: distributes, using mpi broadcast, the necessary
// data to all machines.
//
// smle_nll_mpi: assuming smle_init has been successfully completed
// compute the negative log likelihood using MPI
//
// for now, these functions will be written out in the code
// // MPI INITIALIZATION
// int rank, size;
// const int root = 0;
// cout << "Before init" << endl;
// MPI::Init();
// rank = MPI::COMM_WORLD.Get_rank();
// size = MPI::COMM_WORLD.Get_size();
// double * ua_ptr = ua.data();
// double * ub_ptr = ub.data();
cout << "Initializating MPI Broadcast" << endl;
}
Solution
The RInside package comes with working examples for MPI:
edd@max:~$ ls -l /usr/local/lib/R/site-library/RInside/examples/mpi/
total 24
drwxr-xr-x 2 edd staff 4096 Jun 27 10:14 cmake
-rw-r--r-- 1 edd staff 1897 Jun 27 10:14 Makefile
-rw-r--r-- 1 edd staff 1224 Jun 27 10:14 rinside_mpi_sample0.cpp
-rw-r--r-- 1 edd staff 2896 Jun 27 10:14 rinside_mpi_sample1.cpp
-rw-r--r-- 1 edd staff 1137 Jun 27 10:14 rinside_mpi_sample2.cpp
-rw-r--r-- 1 edd staff 2836 Jun 27 10:14 rinside_mpi_sample3.cpp
edd@max:~$
with working support for make and CMake. I would start there. Adding support for RcppEigen is probably trivial; I just added another example directory for RcppArmadillo to the SVN of RInside a few days ago.
If the definition of Free() balks then you probably have a header file conflict which you may avoid by reordering include statements. I do not have the Intel compile so I cannot help.
Finally: the recommended way to get help for Rcpp and RInside is to ask on the rcpp-devel mailing list.
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
