Compiling METIS Libraries

First, enter the directory of METIS_source and edit the makefile pointing to your C compiler. It is recommended to use a C compiler from the same vender as your intended Fortran compiler. The optimizations -msse3 -O3 for Intel Compiler were activated.

  • $ make

Edit makefile for building FVCOM

The following is an example for Intel Compiler:

  • CPP = /usr/bin/cpp  # GNU compiler may be required.
  • CPPFLAGS = $(DEF_FLAGS) ?DINTEL
  • FC = mpif90
  • DEBFLAGS =
  • OPT = -msse3 -O3 -ipo -no-prec-div  # -msse3 is an option for HA8000
  • CLIB =

Do not forget to uncomment “MULTI_PROCESSOR.”

Compiling FVCOM

  • $ make

Then, you will obtain an executable file “fvcom.”

Run FVCOM

Make a bash job script file of mpi.sh as follows:

  • #!/bin/bash
  • #@$-q short          # “parallel” for within 24 hours while “short” for within 1 hour
  • #@$-N 4              # the number of nodes 4, 8, or 16 (dependent on your contract)
  • #@$-J T16            # uncomment or T2, T4, T8, or T16. T16 or T8 seems the best choice for my case.
  • #@$-lT 1:00:00     # job time limitation dependent on submitted job class
  • cd ~/FVCOM_K/run  # full path to the directory of executable file 
  • mpirun ./fvcom casename

Then, run fvcom using NQS.

  • $ qsub mpi.sh
  • $ qstat  # you can check the status of the job.
  • $ qstat ?b  # you can check the status of all of the jobs.

Check mail and then check mpi.sh.e****** and mpi.sh.o******. In mpi.sh.e****** sometimes you may see errors starting with something

  • =>> PBS: job killed: walltime 313 exceeded limit 300

which means the computational wall clock time exceeds the job time limitation.