1. Documentation

Compiling with MPI and OpenMP

Status
Yes

We have OpenMPI in /opt/openmpi - which are normally set to use the Intel compilers as default. More information can be found here.

Make sure that MPI libraries are in your environment as well. For example, including the following lines in your .bashrc will add openmpi in your environment:

export PATH=/opt/openmpi/bin:$PATH

export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH

Example 1: OpenMP

Here is a simple example helloWorld.c

#include <stdio.h>
#include <omp.h>

int main(int argc, char *argv[]) {
   int iam = 0, np = 1;
   #pragma omp parallel default(shared) private(iam, np)
   {
   #if defined (_OPENMP)
     np = omp_get_num_threads();
     iam = omp_get_thread_num();
   #endif
   printf("Hello from thread %d out of %d\n", iam, np);
  }
  return 0;
}

This code uses OpenMP (shared memory) parallelization. To compile, you will need the -openmp flag (-fopenmp for GNU compiler)

icc -openmp -o helloWorld.x helloWorld.c

To run, include the following in your job submission file:

export OMP_NUM_THREADS = 12 # Desired number of parallel threads

./helloWorld.x

Example 2: MPI

Here is a simple helloWorld.c

#include "mpi.h"
#include <stdio.h>
int main(int argc, char *argv[]) {
    int numtasks, rank;
    MPI_Init(&argc,&argv);
    MPI_Comm_size(MPI_COMM_WORLD,&numtasks);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);
    printf ("Number of tasks= %d My rank= %d\n", numtasks,rank);
    MPI_Finalize();

    return 0;
}

This code uses MPI(distributed) parallelization and can be run across many nodes. To compile, you will need to use mpicc.

mpicc -o helloWorld.x helloWorld.c

To run this on two nodes with 24 processors, include the following in your job submission file:

mpirun -np 24 -machninefile $PBS_NODEFILE  ./helloWorld.x

Switching to GNU

If needed, you can switch to using Gnu if need be by switching some environment variables. You'll want to put this is your .bashrc so it happens every time you login (or run a job)

export OMPI_FC=gfortran
export  OMPI_F77=gfortran
export OMPI_CC=gcc
export OMPI_CXX=g++
export OMPI_CFLAGS="-O2"
export CXXFLAGS="-O3"