MPI - first step

MPI - Start

Distributed memory computing.
MPI: Quick reference (C++), docu, home page, tutorial (LLNL, MPI-book)

Check:

mpirun not available?
> sudo apt-get install mpi-default-dev mpi-default-bin
also
> sudo apt-get install openmpi-bin openmpi-doc libopenmpi-dev
May I log onto the computer without password?

Does the ssh-deamon run?
>ps -ax |grep sshd
Install sshd if not available.

Create a secure authentication keys for your account.

> ssh-keygen
> cp  ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
> chmod go-rwx ~/.ssh/authorized_keys
> ssh-add

Compiling Code:

Compilig code mpicxx [compiler options] skalar.cpp -o main.GCC_
> mpicxx [compiler options] skalar.cpp -o main.GCC_
Running the code on 4 processes
> mpirun -np 4 ./main.GCC_
Run code on 4 processes on 4 different computers
> mpirun -np 4 --hostfile my_hostfile ./main.GCC_
where the ASCII-file my_hostfile contains the name of the computers involved per row.

Changing the underlying Compiler for OpenMPI (briefly):

Using INTEL: > export OMPI_CXX=icpc
Using PGI: > export OMPI_CXX=pgc++ and > export OMPI_CXXFLAGS="-I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi"
(otherwise the standard MPI option -pthread is not accepted by the PGI compiler,
the same as mpicxx -showme:compile |sed 's/-pthread//g')
Using CUDA-compiler: export OMPI_CXX=nvcc

How to MPI-parallelize the inner product:

Original code for inner product:

double scalar(const int N, const double x[], const double y[])
{
 double sum = 0.0;
 for (int i=0; i<N; ++i)
 {
    sum += x[i]*y[i];
 }
 return sum;
}

int main()
{
  ...
  double s = scalar(n,a,b);
  ... 
}

MPI code for inner product:

#include <mpi.h>

// local sequential inner product
double scalar(const int N, const double x[], const double y[])
{
 double sum = 0.0;
 for (int i=0; i<N; ++i)   
     sum += x[i]*y[i];
 return sum;
}

// MPI inner product
double scalar(const int n, const double x[], const double y[], const MPI_Comm icomm)
{
  const double s = scalar(n,x,y);                // call sequential inner product
        double sg;
  MPI_Allreduce(&s,&sg,1,MPI_DOUBLE,MPI_SUM,icomm);

  return(sg);
}

int main(int argc, char* argv[])
{
  ...
  MPI_Init(&argc,&argv);
  ...
  double s = scalar(n,a,b,MPI_COMM_WORLD);
  ...
  MPI_Finalize();
  ...
}

and

compile: > mpicxx skalar.cpp -o main.GCC_
run code on 4 MPI processes: > mpirun -np 4 main.GCC_ .