Set up ITO-A@kyushu-u supercomputer

This is a memorandum report for seting up an environment on ITO-A at Kyushu University Supercomputer Center.

How to login

On Windows, you need a terminal emulator to login and MobaXterm is recommended to use, which also has a function of X11 forwarding for displaying X Window screen. Its installation method is introduced in this site (in Japanese).

Change file access permission

You may change the permission of your directories and files as follows so that group members can read and execute files and help you:

chmod -R 750 /home/usr_num/account

where the first, second, and third digits are types of permissions for user, group, and all the others, and the permission of each number is as follows:

0=none
1=execute
2=write
4=read
5=read/execute
6=read/write
7=read/write/execute

Warning: Do not change to 770 because group users can edit and change your files, which is very dangerous. In addition, you cannot login owing to a problem of inappropriate permission of ~/.ssh/.

Using Matlab

Matlab can be used in command line environment (CUI) or Graphical User Interface (GUI) through X Windows on ITO-A. Details are given in this site.

Batch job using Intel compilers

Computation on ITO-A is in general performed by submitting a bash job script to the batch queue system. Details is given in this site. A set of manual of Intel compilers can be referred in this site.
Some examples of bash script are shown below. More information is available in this site.

• rscgrp: Resource group. For example, 1 node and within 1 hour: rscgrp=ito-ss-dbg, within 4 nodes and within 1 hour ito-s-dbg, 1 node and within 96 hours: ito-ss, within 4 nodes and within 48 hours: ito-s.
• vnode: Number of nodes (virtual)
• vnode-core: Number of cores per node (virtual) (Max 36)
• elapse: Max elapsed time in HH:MM:SS (Less than or equal to rscgrp
• OMP_NUM_THREADS: Number of OpenMP (Number of threads)
• NUM_NODES:: Number of nodes (physical). See the following examples.
• NUM_CORES: Number of cores per node (physical)(=vnode-core)
• NUM_PROCS: Number of MPI processes (= NUM_CORES × vnode)
• I_MPI_PERHOST: Number of MPI processes per node (Physical)
• I_MPI_FABRICS: Communication method among and within nodes. See the following examples.

OpenMP or Automatic Parallelization (shared memory)

#!/bin/bash
#PJM -L "rscunit=ito-a" 
#PJM -L "rscgrp=ito-ss-dbg"
#PJM -L "vnode=1"
#PJM -L "vnode-core=36"
#PJM -L "elapse=01:00:00"
#PJM -j
#PJM -X

module load intel/2018
NUM_CORES=36
export OMP_NUM_THREADS=$NUM_CORES
./a.out

Flat MPI

#!/bin/bash
#PJM -L "rscunit=ito-a" 
#PJM -L "rscgrp=ito-s-dbg"
#PJM -L "vnode=4"
#PJM -L "vnode-core=36"
#PJM -L "elapse=02:00:00"
#PJM -j
#PJM -X

module load intel/2018

NUM_NODES=${PJM_VNODES}
NUM_CORES=36
NUM_PROCS=144

export I_MPI_PERHOST=$NUM_CORES
export I_MPI_FABRICS=shm:ofa

export I_MPI_HYDRA_BOOTSTRAP=rsh
export I_MPI_HYDRA_BOOTSTRAP_EXEC=/bin/pjrsh
export I_MPI_HYDRA_HOST_FILE=${PJM_O_NODEINF}

mpiexec.hydra -n $NUM_PROCS ./a.out

Install Miniconda

To keep the environment cleaner, I install Miniconda instead of Anaconda. Download the latest version of Miniconda3-latest-Linux-x86-64.sh from this site onto, e.g., ~/src/．After converting it to executable file, install as follows:

./Miniconda3-latest-Linux-x86-64.sh

Update the environment.

conda update conda
conda update --all

If you want to create a virtual environment, you may refer to this article.

Install basic packages

Install basic packages as follows:

conda install jupyter notebook pandas matplotlib netcdf4 holoviews geoviews
conda install selenium
conda install phantomjs
pip install holoext

You can get a list of installed packages by conda list. A package of holoext is an add-on for Holoviews. To upgrade holoext, first uninstall it and then install again:

pip uninstall holoext
pip install holoext

If you encounter the following error message, it may be attributed to using an outdated version of pip.

mkl-random 1.0.1 requires cython, which is not installed.
mkl-fft 1.0.4 requires cython, which is not installed.
distributed 1.22.0 requires msgpack, which is not installed.

Then, you can update pip by:

pip install --upgrade pip

Install PhantomJS

PhantomJS is required to use Holoviews. Referring to this site, you may proceed to install. First, install NVM:

wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.10/install.sh | bash
source .bashrc
command -v nvm

It is successful if nvm appears at the last.
Next, install NodeJS. You may get a list of available versions of NodeJS as follows:

nvm ls-remote

Install the latest version:

nvm install node

You may confirm the version of installed NodeJS:

$ nvm list
->      v10.8.0
default -> node (-> v10.8.0)
node -> stable (-> v10.8.0) (default)
stable -> 10.8 (-> v10.8.0) (default)
iojs -> N/A (default)
lts/* -> lts/carbon (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.14.3 (-> N/A)
lts/carbon -> v8.11.3 (-> N/A)

Or you may confirm using node command:

$ node -v
v10.8.0

Referring to this site, install PhantomJS:

npm install --save phantom phantomjs-prebuilt tough-cookie uuid