Set up ITO-A@kyushu-u supercomputer

Table of Content

This is a memorandum report for seting up an environment on ITO-A at Kyushu University Supercomputer Center.

How to login

On Windows, you need a terminal emulator to login and MobaXterm is recommended to use, which also has a function of X11 forwarding for displaying X Window screen. Its installation method is introduced in this site (in Japanese).

Change file access permission

You may change the permission of your directories and files as follows so that group members can read and execute files and help you:
[cc lang='bash']
$ chmod -R 750 /home/usr_num/account
[/cc]
where the first, second, and third digits are types of permissions for user, group, and all the others, and the permission of each number is as follows:
[cc]
0=none
1=execute
2=write
4=read
5=read/execute
6=read/write
7=read/write/execute
[/cc]
Warning: Do not change to [cci]770[/cci] because group users can edit and change your files, which is very dangerous. In addition, you cannot login owing to a problem of inappropriate permission of ~/.ssh/.

Using Matlab

Matlab can be used in command line environment (CUI) or Graphical User Interface (GUI) through X Windows on ITO-A. Details are given in this site.

Batch job using Intel compilers

Computation on ITO-A is in general performed by submitting a bash job script to the batch queue system. Details is given in this site. A set of manual of Intel compilers can be referred in this site.
Some examples of bash script are shown below. More information is available in this site.

  • [cci]rscgrp[/cci]: Resource group. For example, 1 node and within 1 hour: [cci]rscgrp=ito-ss-dbg[/cci], within 4 nodes and within 1 hour [cci]ito-s-dbg[/cci], 1 node and within 96 hours: [cci]ito-ss[/cci], within 4 nodes and within 48 hours: [cci]ito-s[/cci].
  • [cci]vnode[/cci]: Number of nodes (virtual)
  • [cci]vnode-core[/cci]: Number of cores per node (virtual) (Max 36)
  • [cci]elapse[/cci]: Max elapsed time in HH:MM:SS (Less than or equal to [cci]rscgrp[/cci]
  • [cci]OMP_NUM_THREADS[/cci]: Number of OpenMP (Number of threads)
  • [cci]NUM_NODES:[/cci]: Number of nodes (physical). See the following examples.
  • [cci]NUM_CORES[/cci]: Number of cores per node (physical)(=vnode-core)
  • [cci]NUM_PROCS[/cci]: Number of MPI processes (= NUM_CORES × vnode)
  • [cci]I_MPI_PERHOST[/cci]: Number of MPI processes per node (Physical)
  • [cci]I_MPI_FABRICS[/cci]: Communication method among and within nodes. See the following examples.

OpenMP or Automatic Parallelization (shared memory)

[cc lang='bash']
#!/bin/bash
#PJM -L "rscunit=ito-a"
#PJM -L "rscgrp=ito-ss-dbg"
#PJM -L "vnode=1"
#PJM -L "vnode-core=36"
#PJM -L "elapse=01:00:00"
#PJM -j
#PJM -X

module load intel/2018
NUM_CORES=36
export OMP_NUM_THREADS=$NUM_CORES
./a.out
[/cc]

Flat MPI

[cc lang='bash']
#!/bin/bash
#PJM -L "rscunit=ito-a"
#PJM -L "rscgrp=ito-s-dbg"
#PJM -L "vnode=4"
#PJM -L "vnode-core=36"
#PJM -L "elapse=02:00:00"
#PJM -j
#PJM -X

module load intel/2018

NUM_NODES=${PJM_VNODES}
NUM_CORES=36
NUM_PROCS=144

export I_MPI_PERHOST=$NUM_CORES
export I_MPI_FABRICS=shm:ofa

export I_MPI_HYDRA_BOOTSTRAP=rsh
export I_MPI_HYDRA_BOOTSTRAP_EXEC=/bin/pjrsh
export I_MPI_HYDRA_HOST_FILE=${PJM_O_NODEINF}

mpiexec.hydra -n $NUM_PROCS ./a.out
[/cc]

Install Miniconda

To keep the environment cleaner, I install Miniconda instead of Anaconda. Download the latest version of Miniconda3-latest-Linux-x86-64.sh from this site onto, e.g., [cci]~/src/[/cci].After converting it to executable file, install as follows:
[cc lang='bash']
~/src$ ./Miniconda3-latest-Linux-x86-64.sh
[/cc]
Update the environment.
[cc lang='python']
~$ conda update conda
~$ conda update --all
[/cc]
If you want to create a virtual environment, you may refer to this article.

Install basic packages

Install basic packages as follows:
[cc lang='bash']
~$ conda install jupyter notebook pandas matplotlib netcdf4 holoviews geoviews
~$ conda install selenium
~$ conda install phantomjs
~$ pip install holoext
[/cc]
You can get a list of installed packages by [cci]conda list[/cci]. A package of holoext is an add-on for Holoviews. To upgrade holoext, first uninstall it and then install again:
[cc lang='python']
~$ pip uninstall holoext
~$ pip install holoext
[/cc]
If you encounter the following error message, it may be attributed to using an outdated version of [cci]pip[/cci].
[cc lang='python']
mkl-random 1.0.1 requires cython, which is not installed.
mkl-fft 1.0.4 requires cython, which is not installed.
distributed 1.22.0 requires msgpack, which is not installed.
[/cc]
Then, you can update [cci]pip[/cci] by:
[cc lang='python']
~$ pip install --upgrade pip
[/cc]

Install PhantomJS

PhantomJS is required to use Holoviews. Referring to this site, you may proceed to install. First, install [cci]NVM[/cci]:
[cc lang='bash']
~$ wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.10/install.sh | bash
~$ source .bashrc
~$ command -v nvm
nvm
[/cc]
It is successful if [cci]nvm[/cci] appears at the last.
Next, install NodeJS. You may get a list of available versions of NodeJS as follows:
[cc lang='bash']
~$ nvm ls-remote
[/cc]
Install the latest version:
[cc lang='bash']
~$ nvm install node
[/cc]
You may confirm the version of installed NodeJS:
[cc lang='bash']
~$ nvm list
-> v10.8.0
default -> node (-> v10.8.0)
node -> stable (-> v10.8.0) (default)
stable -> 10.8 (-> v10.8.0) (default)
iojs -> N/A (default)
lts/* -> lts/carbon (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.14.3 (-> N/A)
lts/carbon -> v8.11.3 (-> N/A)
[/cc]
Or you may confirm using [cci]node[/cci] command:
[cc lang='bash']
~$ node -v
v10.8.0
[/cc]
Referring to this site, install [cci]PhantomJS[/cci]:
[cc lang='bash']
~$ npm install --save phantom phantomjs-prebuilt tough-cookie uuid
[/cc]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.