Others
Deep Learning Frameworks
To use Deep Learning Framework on the ABCI System, user must install it to home or group area. How to install Deep Learning Framework is following.
Caffe
To install Caffe, please follow the instructions below.
INSTALL_DIR : install path
[username@g0001 ~]$ cd INSTALL_DIR
[username@g0001 ~]$ module load python/2.7/2.7.15 cuda/9.1/9.1.85.3 cudnn/7.0/7.0.5
[username@g0001 ~]$ git clone https://github.com/BVLC/caffe
[username@g0001 ~]$ cd caffe
[username@g0001 caffe]$ cp Makefile.config.example Makefile.config
[username@g0001 caffe]$ vi Makefile.config
[username@g0001 caffe]$ make all 2>&1 > log_make-all.txt
[username@g0001 caffe]$ make test 2>&1 > log_make-test.txt
[username@g0001 caffe]$ make runtest 2>&1 > log_make-runtest.txt
[username@g0001 caffe]$ pip install -r python/requirements.txt
[username@g0001 caffe]$ make pycaffe
[username@g0001 caffe]$ make distibute
Caffe2
To install Caffe2, please follow the instructions below.
INSTALL_DIR : install path
[username@g0001 ~]$ export PREFIX=INSTALL_DIR
[username@g0001 ~]$ module load python/3.6.5 cuda/9.1/9.1.85.3 cudnn/7.0/7.0.5 nccl/2.1/2.1.15-1
[username@g0001 ~]$ git clone https://github.com/gflags/gflags.git
[username@g0001 ~]$ mkdir gflags/build && cd gflags/build
[username@g0001 build]$ cmake3 -DBUILD_SHARED_LIBS=ON -DCMAKE_CXX_FLAGS='-fPIC' -DCMAKE_INSTALL_PREFIX=$PREFIX ..
[username@g0001 build]$ make -j 8 2>&1 | tee make.log
[username@g0001 build]$ make install 2>&1 | tee make_install.log
[username@g0001 build]$ cd
[username@g0001 ~]$ git clone https://github.com/google/glog
[username@g0001 ~]$ cd glog
[username@g0001 glog]$ sh autogen.sh
[username@g0001 glog]$ CXXFLAGS="-fPIC -I$PREFIX/include" LDFLAGS="-L$PREFIX/lib" ./configure --prefix=$PREFIX 2>&1 | tee configure.log
[username@g0001 glog]$ make -j 8 2>&1 | tee make.log
[username@g0001 glog]$ make install 2>&1 | tee make_install.log
[username@g0001 glog]$ cd
[username@g0001 ~]$ pip3 install future graphviz hypothesis jupyter matplotlib numpy protobuf pydot python-nvd3 pyyaml requests scikit-image scipy six --prefix=$PREFIX
[username@g0001 ~]$ export CUDNN_INCLUDE_DIR=$CUDNN_HOME/include
[username@g0001 ~]$ export CUDNN_LIBRARY=$CUDNN_HOME/lib64/libcudnn.so.7.0.5
[username@g0001 ~]$ export NCCL_INCLUDE_DIR=$NCCL_HOME/include
[username@g0001 ~]$ export NCCL_LIBRARY=$NCCL_HOME/lib/libnccl.so.2.1.15
[username@g0001 ~]$ git clone --recursive https://github.com/pytorch/pytorch.git
[username@g0001 ~]$ cd pytorch && git submodule update --init
[username@g0001 pytorch]$ mkdir build && cd build
[username@g0001 build]$ cmake3 -DPYTHON_INCLUDE_DIR=/apps/python/3.6.5/include/python3.6m -DPYTHON_EXECUTABLE=/apps/python/3.6.5/bin/python3 -DPYTHON_LIBRARY=/apps/python/3.6.5/lib -DNCCL_INCLUDE_DIR=$NCCL_INCLUDE_DIR -DNCCL_LIBRARY=$NCCL_LIBRARY -DUSE_OPENCV=ON -DCMAKE_INSTALL_PREFIX=INSTALL_DIR .
[username@g0001 build]$ make install 2>&1 | tee make_install.log
TensorFlow
To install TensorFlow, please follow the instructions below.
NEW_VENV : python virtual environment or path to be installed
[username@g0001 ~]$ module load python/3.6/3.6.5 cuda/9.0/9.0.176.4 cudnn/7.2/7.2.1
[username@g0001 ~]$ export LD_LIBRARY_PATH=$CUDA_HOME/extras/CUPTI/lib64:$LD_LIBRARY_PATH
[username@g0001 ~]$ python3 -m venv NEW_VENV
[username@g0001 ~]$ source NEW_VENV/bin/activate
(NEW_VENV) [username@g0001 ~]$ pip3 install tensorflow-gpu
Theano
Please refer to following page for how to install Theano.
Torch
To install Torch, please follow the instructions below.
INSTALL_DIR : install path
INSTALL_DIR_OPENBLAS : install path (OpenBLAS)
[username@g0001 ~]$ module load cuda/9.1/9.1.85.3
[username@g0001 ~]$ git clone https://github.com/xianyi/OpenBLAS.git
[username@g0001 ~]$ make TARGET=HASWELL NO_AFFINITY=1 USE_OPENMP=1 > log_make_20180621-00.txt 2>&1
[username@g0001 ~]$ make install PREFIX=INSTALL_DIR_OPENBLAS > log_make_inst_20180621-00.txt 2>&1
[username@g0001 ~]$ export LD_LIBRARY_PATH=INSTALL_DIR_OPENBLAS/lib:$LD_LIBRARY_PATH
[username@g0001 ~]$ git clone https://github.com/torch/distro.git ./torch --recursive
[username@g0001 ~]$ export TORCH_NVCC_FLAGS="-D__CUDA_NO_HALF_OPERATORS__"
[username@g0001 ~]$ TORCH_LUA_VERSION=LUA51 PREFIX=INSTALL_DIR ./install.sh
PyTorch
To install PyTorch, please follow the instructions below.
NEW_VENV : python virtual environment or path to be installed
[username@g0001 ~]$ module load python/3.6/3.6.5 cuda/9.1/9.1.85.3
[username@g0001 ~]$ python3 -m venv NEW_VENV
[username@g0001 ~]$ source NEW_VENV/bin/activate
(NEW_VENV) [username@g0001 ~]$ pip3 install torch torchvision
CNTK
Please refer to following page for how to install CNTK.
MXNet
To install MXNet, please follow the instructions below.
NEW_VENV : python virtual environment or path to be installed
[username@g0001 ~]$ module load python/3.6/3.6.5 cuda/9.2/9.2.148.1
[username@g0001 ~]$ python3 -m venv NEW_VENV
[username@g0001 ~]$ source NEW_VENV/bin/activate
(NEW_VENV) [username@g0001 ~]$ pip3 install mxnet-cu92
Chainer
To install Chainer, please follow the instructions below.
NEW_VENV : python virtual environment or path to be installed
[username@g0001 ~]$ module load python/3.6/3.6.5 cuda/9.1/9.1.85.3 cudnn/7.0/7.0.5
[username@g0001 ~]$ python3 -m venv NEW_VENV
[username@g0001 ~]$ source NEW_VENV/bin/activate
(NEW_VENV) [username@g0001 ~]$ pip3 install cupy-cuda91 chainer
Keras
To install Keras with TensorFlow backend, please follow the instructions below.
NEW_VENV : python virtual environment or path to be installed
[username@g0001 ~]$ module load python/3.6/3.6.5 cuda/9.0/9.0.176.4 cudnn/7.2/7.2.1
[username@g0001 ~]$ export LD_LIBRARY_PATH=$CUDA_HOME/extras/CUPTI/lib64:$LD_LIBRARY_PATH
[username@g0001 ~]$ python3 -m venv NEW_VENV
[username@g0001 ~]$ source NEW_VENV/bin/activate
(NEW_VENV) [username@g0001 ~]$ pip3 install tensorflow-gpu
(NEW_VENV) [username@g0001 ~]$ pip3 install keras
More details can be found in Keras.
Big Data Analytics Frameworks
Hadoop
Hadoop is available for ABCI System. When you use this framework, you need to set up user environment by module command.
Setting commands for Hadoop are the following.
$ module load openjdk/1.8.0.131
$ module load hadoop/2.9.1
Example) Running Hadoop on compute nodes.
[username@es1 ~]$ qrsh -l rt_F=1 -l h_rt=1:00:00
[username@g0001~]$ module load openjdk/1.8.0.131
[username@g0001~]$ module load hadoop/2.9.1
[username@g0001~]$ mkdir input
[username@g0001~]$ cp /apps/hadoop/2.9.1/etc/hadoop/*.xml input
[username@g0001~]$ hadoop jar /apps/hadoop/2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+'
[username@g0001~]$ cat output/part-r-00000
1 dfsadmin