Outlines:

  • Adding your Public Key to the server
  • Adding essential variables to ~/.bashrc
  • Installing conda on Big data storage
  • Changing the vscode installing path for extensions
  • Scheduling a job with tmux and slurm

Adding your Public Key to the server

If you don’t want to enter your password each time you connect to the server you can do the following steps: follow this link. (linux9 is an example)

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub <user_name>@linux9

Adding essential variables to ~/.bashrc

You can add these variables to ~/.bashrc to prevent exceeding the memory limit. Note: Remember to change to your username!

# Torch and Python
export TORCH_HOME=/research/.../<user_name>/.cache/
export PIP_CACHE_DIR=/research/.../<user_name>/.cache/
export PYTHONUSERBASE=/research/.../<user_name>/.cache/python
export XDG_CACHE_HOME=/research/.../<user_name>/.cache/

# Slurm Confs
export SLURM_CONF=/opt1/slurm/gpu-slurm.conf

Installing conda on Big data storage

You need to install conda in order to work with different python versions on the server and the ~ directory space is limited. You can follow this link

  1. go to your big data storage directory (sth like /research/d2/.../<user_name>)
  2. mkdir -p /research/.../<user_name>/miniconda3
  3. wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /research/.../<user_name>/miniconda3/miniconda.sh
  4. bash /research/.../<user_name>/miniconda3/miniconda.sh
  5. rm -rf /research/.../<user_name>/miniconda3/miniconda.sh
  6. ./miniconda3/bin/conda init bash

Changing the vscode installing path for extensions

Due to the limited space in the home directory, if you want to install some VS code extensions, you need to change the installing path. You can see this link for more information or just follow these steps:

  1. Go to the settings
  2. Search Remote.SSH: Server Install Path in the settings.
  3. add the name of the server as the key and /research/d2/rshr/<user_name>/ as the path. (For example)

Scheduling a job with tmux and slurm

  1. First make a tmux session
  2. Allocate the gpu using slurm commands. (Follow this link)
  3. Run your code!
  4. Detach from tmux using (cntrl + b + d)
  5. You can attach again using tmux a command.

If you had any question, you can just contact me!