Here is a list of simple templates that you can use for submitting jobs to the queue. All scipts can be submitted by
#!/bin/bash #PBS -l nodes=2:ppn=12 #PBS -l walltime=2:00:00 #PBS -V cd $PBS_O_WORKDIR mpirun -np 24 -machinefile $PBS_NODEFILE ./run.x
This script is used for running the executable run.x which is compiler using MPI. This script enables a run across 2 nodes on 24 processors.
If you have a serial job, the following script can be used
#!/bin/bash #PBS -l nodes=1:ppn=12 #PBS -l walltime=2:00:00 #PBS -V cd $PBS_O_WORKDIR ./run.x
The serial executable run.x will run on a single compute node, making use of only one processor out of 12 but taking advantage of the whole memory of the node.
Job with Shared Memory Parallelizm (e.g. OpenMP)
If you have an executable with shared memory parallelization, you can use
#!/bin/bash #PBS -l nodes=1:ppn=12 #PBS -l walltime=2:00:00 #PBS -V cd $PBS_O_WORKDIR # You can use larger number of threads, but test speed before doing so export OMP_NUM_THREADS=12 ./run.x
The executable run.x will run on 12 cores on the shared memory of the node.
Running Multiple Parallel Jobs Sequentially:
Sometimes you may need to run several parallel jobs one after another (i.e. a case where the second run needs output from the first one). In this case you can use a script like the following one:
#!/bin/bash #PBS -l nodes=2:ppn=24 #PBS -l walltime=2:00:00 #PBS -V cd $PBS_O_WORKDIR mpirun -np 24 -machinefile $PBS_NODEFILE ./run1.x mpirun -np 24 -machinefile $PBS_NODEFILE ./run2.x mpirun -np 24 -machinefile $PBS_NODEFILE ./run3.x
run3.x will run only after run2.x is completed, and run2.x will only run after run1.x is completed.
Running Multiple Jobs Simultaneously
In certain applications, you may have a serial/parallel code that can execute multiple instances of data simultaneously. You can run several executables simultaneously by the following script
#!/bin/bash #PBS -l nodes=1:ppn=12 #PBS -l walltime=2:00:00 #PBS -V cd $PBS_O_WORKDIR ./run1.x & ./run2.x & ./run3.x & ./run4.x & wait
In this case four serial jobs will run simultaneously, sharing the resources of the 12 core node. The & sign makes sure that runs do not wait for others to end. The final wait command is essential to make sure that the job does not quit before all runs are completed. A similar script can be constructed for simultaneous parallel jobs (replace ./run1-4.x & with mpirun -machninefile $PBS_NODEFILE -np 12 ./run1-4.x &)
- Short queue: For jobs that run less than one hour, less waiting in the queue
qsub -q short name_of_your_script
- Large memory: For jobs that require very large memory; largemem: 256 GB/node and xlargemem: 512 GB/node.
qsub -q largemem name_of_your_script qsub -q xlargemem name_of_your_script
- GPU node: For Jobs that use GPUs
qsub -q gpuq name_of_your_script