Compiling METIS Libraries
First, enter the directory of METIS_source and edit the makefile pointing to your C compiler. It is recommended to use a C compiler from the same vender as your intended Fortran compiler. The optimizations -msse3 -O3 for Intel Compiler were activated.
- $ make
Edit makefile for building FVCOM
The following is an example for Intel Compiler:
- CPP = /usr/bin/cpp # GNU compiler may be required.
- CPPFLAGS = $(DEF_FLAGS) ?DINTEL
- FC = mpif90
- DEBFLAGS =
- OPT = -msse3 -O3 -ipo -no-prec-div # -msse3 is an option for HA8000
- CLIB =
Do not forget to uncomment “MULTI_PROCESSOR.”
- $ make
Then, you will obtain an executable file “fvcom.”
Make a bash job script file of mpi.sh as follows:
- #@$-q short # “parallel” for within 24 hours while “short” for within 1 hour
- #@$-N 4 # the number of nodes 4, 8, or 16 (dependent on your contract)
- #@$-J T16 # uncomment or T2, T4, T8, or T16. T16 or T8 seems the best choice for my case.
- #@$-lT 1:00:00 # job time limitation dependent on submitted job class
- cd ~/FVCOM_K/run # full path to the directory of executable file
- mpirun ./fvcom casename
Then, run fvcom using NQS.
- $ qsub mpi.sh
- $ qstat # you can check the status of the job.
- $ qstat ?b # you can check the status of all of the jobs.
Check mail and then check mpi.sh.e****** and mpi.sh.o******. In mpi.sh.e****** sometimes you may see errors starting with something
- =>> PBS: job killed: walltime 313 exceeded limit 300
which means the computational wall clock time exceeds the job time limitation.