Sometimes you want to be able to set up a system exactly how you want, with your own specific software - and maybe with a different OS. For example, the CSC clusters run RedHat/CentOS, and maybe you want an ubuntu image with some different packages loaded. You can use 'containers' to make an image just how you want (think of it sort of as a virtual machine that you can run on the cluster).
To make the image, you need to be root/admin, so general you'd do it on your own system. Install singularity locally (E.g. on your linux desktop, yum install singularity) or download it from https://github.com/singularityware/singularity/releases
Once you have it installed you can pull docker images, etc (so for example, if a collaborator gives you a docker or singularity image).
Or, here's how to build your own basic one (for Ubuntu 16 - don't use v18, it's too new to run on our clusters). You're building an image, so you want to do this as root (so why you make you image on your local machine - you don't have root on 'knot').
singularity build --sandbox ubuntu-test docker://ubuntu:16.04
will make an image in your directory 'ubuntu-test', that contains your image.
You can now start it up with
/usr/bin/singularity shell --writable ubuntu-test
and that puts you in the container - it's no longer CentOS:
Singularity ubuntu-test:~> cat /etc/os-release NAME="Ubuntu" VERSION="16.04.5 LTS (Xenial Xerus)" ...
However, you might find it's missing some things you need, in this case 'gcc'.
Singularity ubuntu-test:~> gcc bash: gcc: command not found
First (for Ubuntu at least) need to first do a apt-get update
then after that, you can install the stuff you need
apt-get install gcc
and here's where it can get nice, maybe you want a bunch of other packages that are easily available under your guest OS, say 'bioperl' in this example. You can install it (remember, you're running it as root on your local machine at the moment, so you can do whatever you want).
apt-get install bioperl
after it's all dialed in, you build it into a (now unwriteable!!) image that you can transport to different systems
singularity build production.simg unbuntu-test
This image file you can then move to knot,braid,pod, etc. and run there with singularity
An example job script (which say runs some perl job in your container) would be
For PBS (knot,braid - at the moment)
#!/bin/bash -l #PBS -l nodes=1:ppn=4 cd $PBS_O_WORKDIR module load singularity singularity exec production.simg perl myfile.pl
Or in Slurm (Pod, soon, the other clusters too)
#!/bin/bash -l #SBATCH -N 1 --ntasks-per-node=4 cd $SLURM_BATCH_DIR module load singularity singularity exec production.simg perl myfile.pl