User Guide

Xseed User Manual
1.) How to compile and run the MPI program without queuing system
2.) How to submit a job to queuing system
3.) How to submit MPI program to queuing system
MPI
In /apps/MPI_users_help/MPI_examples/ there are some examples (simple mpi apps in c,
c++, and fortran). Copy the files from directory to your directory first using:
mkdir ~/MPI/
mkdir ~/MPI/examples/
cp /apps/MPI_users_help/MPI_examples/* ~/MPI/examples/
cd ~/MPI/examples
Now you have the examples in your directory: /home/$user/MPI/examples. And you are
in this directory too.
Setup environment: - not necessary with runmpi script
To launch the MPI applications from xseed1 the environment must be set up:
export MXMPI_MASTER=10.224.0.1
Make a machine file:
Machine file is a file with every node name you want use listed 1 per line. It would look like
this:
Node1
node1
node2
node2
...
node224
node224

Nodes are listed twice because there are 2 cpus per machine, if you only want 1 cpu per node,
just list them once.
To generate the machine file there was created a script. The script name is gen and it is
located in the directory:
/apps/MPI_users_help/ or /apps/MPI_users_help/MPI_examples/.
The script has two parameters – the first parameter is the first compute node and second
parameter is the last comp node. The script was already copied to your home directory with
the code examples.
Example of using the script:
./gen 1 20,
will generate the machinefile (machlist.mch) with nodes from 1 to 20.
Compile and run the examples
To compile the examples type:
make
To run one of the examples (cpi.c) use:
./runmpi

MPI – First program – helloworld.c
Copy the hello.c source code, Makefile and the rest of necessary files to your directory
using:
mkdir ~/MPI/
mkdir ~/MPI/helloworld/
cp /apps/MPI_users_help/helloworld/* ~/MPI/helloworld/
cd ~/MPI/helloworld
The content of the directory is:
gen
- machinefile generator
hello.c - helloworld source code
machlist.mch - machine file
Makefile - makefile – to compile to source code
runmpi
- program execution script
Edit the source code:
vi hello.c
Compile the program
Create (or edit) the Makefile to compile the source code:
vi Makefile
The Makefile looks like this:
CC
= /opt/mpich-mx/bin/mpicc
CCC
= /opt/mpich-mx/bin/mpicxx
F77
= /opt/mpich-mx/bin/mpif77
CLINKER = /opt/mpich-mx/bin/mpicc
CCLINKER = /opt/mpich-mx/bin/mpicxx
FLINKER = /opt/mpich-mx/bin/mpif77
F90
= /opt/mpich-mx/bin/mpif90
F90LINKER = /opt/mpich-mx/bin/mpif90
MAKE = make --no-print-directory
SHELL = /bin/sh
EXECS = hello #name of your source code without extension (e.g. hello.c)
all:
$(CC) -o $(EXECS) $(EXECS).c -lm
clean:
-rm *.o
All you need to change in the Makefile is the name of your program, which is after EXECS.
The name of the program in the example is hello.c (don’t change the file if you want to
compile hello.c)
If you create a Makefile, type
make
to compile the source code.
If you does not want to use the Makefile type
/opt/mpich-mx/bin/mpicc -o hello hello.c –lm
to compile the source code. The example shows how to compile source code hello.c

Run the program
/opt/mpich-mx/bin/mpirun -np -machinefile
./
Example : Script “runmpi”
The example shows the content of the script runmpi which executes the program hello .
export MXMPI_MASTER=10.224.0.1
/opt/mpich-mx/bin/mpirun -np 20 -machinefile machlist.mch ./hello
To run the script use:
./runmpi
This example runs program hello on 10 dual processor machines (20 processes = -np 20).
The machines are specified by machlist.mch file.

Queuing System on Xseed
The Xseed uses the Torque, Maui as queuing and resource management system. The
following manual will show how to use the system from user point of view. It means how to
submit simple or MPI jog to the queue.
How to submit a job to the queue
The job can be submitted to the queue using following script. Few parameters need to be
changed for different jobs. For example number of used compute nodes or estimated time it
will take to finish the job. All important parameters are described in the script. The
highlighted parts of the script should be changed by the user.
#bin!/bin/bash
## The below lines is how much real (wall clock)
## time it will take for your job to finish
## Format is HH:MM:SS
#PBS -lwalltime=00:05:00
## Define how many nodes your job will take
## by changing the number after -lnodes= leave :ppn=2 unchanged.
## In this example it is set to use 10 nodes and each node has 2 cpus.
#PBS -lnodes=10:ppn=2
## Below here enter the commands to start your job.
## cd $PBS_O_WORKDIR – the directory where the script is run
cd $PBS_O_WORKDIR
## The program you want to run. In this case it is only basic script.
date
echo "Hello world!"
sleep 120
date
## End of the program.
exit;
The output of the scripts will not be shown on the console. All outputs are saved to the two
files:
.e - error output
.o - program output

Run MPI program using queuing system
To run the MPI program using Torque and Maui the following script can be used. Before the
script can be run the MPI program must be compiled. The script does not need any machine
list because it takes the machine list from Torque. Highlighted parts of the script should be
changed by the user.
#!/bin/bash
## The below lines is how much real (wall clock)
## time it will take for your job to finish
## Format is HH:MM:SS
#PBS -lwalltime=00:05:00
## Define how many nodes your job will take
## by changing the number after -lnodes= leave :ppn=2 unchanged.
## In this example it is set to use 10 nodes and each node has 2 cpus.
#PBS -lnodes=10:ppn=2
## The following lines are for the MPI it self
## Gives to the MPI number of nodes which will be used
NUM_NODES=`cat $PBS_NODEFILE | wc -l | sed "s/ //g"`
## run the MPI program “/home/kutik/mpi/examples/cpi”
/opt/mpich-mx/bin/mpirun -np $NUM_NODES -machinefile $PBS_NODEFILE
/home/kutik/mpi/examples/cpi
Run the script
/apps/packages/torque-2.3.2/bin/qsub qsubtest.sh