Marcopolo's queueing system is set up to favour parallel jobs. Running serial jobs may be evicted and requeued if higher priority parallel jobs are submitted. There are short queues (24 hours), long queues (72 hours) and the test queue (30 minutes). Queues are named s1, s16, s32 and l1, l16, 132. The test queue is called 'test'. The number of processors available in the parallel queues is just a default and you can set it differently. Examples below.
qsub -q l16 # 16 cores on 1 node, the default for l16, which is probably what you want qsub -q l16 -l nodes=1:ppn=10 # 10 cores on 1 node qsub -q l32 # 32 cores, the default for l32, 16 on each of 2 nodes
Marcopolo is a cluster system and so message passing libraries are used to make code run in parallel. There are several different libraries installed. Shared memory parallelization (OpenMP, provided by the autoparallelizing options on some of the compilers) can also be used for up to sixteen cores but not further.
The available message passing libraries are MPI implementations: currently we have Intel MPI and OpenMPI.
To change your parallel environment you need to load the appropriate module. By default Intel MPI with Intel compilers is loaded. To switch you'd edit your .bashrc file and change the module add line.
Compile your code with the MPI compilers. These are usually called mpicc, mpicxx, mpif77 and mpif90.
To run MPI code you normally use a command such as mpirun to launch your program. On marcopolo there is a queueing system which assigns work to compute nodes, and so the MPI library has to interact with that which makes things more complicated. You use a special MPI launcher program to start your job. The launcher doesn't take any arguments but works out the correct number of cpus and which nodes to use from the queueing system. You must use the appropriate one for your MPI library, and you must have your environment configured for the library you want to use when you submit a job, as otherwise the job won't be able to find its library files. Here's a list of which launcher command goes with which library:
|MPI||Compiler Command Name||Launcher Command Name|
|Intel MPI with GNU compilers||mpicc /mpif90||mpirun|
|Intel MPI with Intel compilers||mpiicc /mpiifort||mpirun|