Well, you have got a server (most probably linux) which you want to run your tasks on it. How should you do that? Here I list all the challenges I’ve encountered.
First of all, you know that you should connect to server via
SSH: (1914 is just a dummpy number for port. Just select something that linux hasn’t used)
ssh -L 1914:localhost:1914 pourmand@ServerAddress
After that you will need to enter your password for the user
pourmand in the server.
You may also want the server to know your computer i.e. it does not ask for password every time you want to connect to it. If that’s the case, I suggest that you read this tutorial. To put it in the nutshell, you sould copy your public ssh key to the server using
Then you can run your commands on the server, e.g.:
python train.py ...
This is especially good for commands that do not take long time to run. If you have a model that need to be trained for days or even longer, you can not use this. That’s because the moment you close your
ssh, all your normal processes would die. So, you will need to use a terminal multiplexer like tmux or
screen to handle this problem. Both of them are free and open source and both have easy commands to achieve your needs. You can read full pros and cons here. I use
tmux because the syntax is easier to use.
Here is a simple tmux session I have created. Note that the session has 6 open tabs. Each of them may do something different.
If you install tmux, out of the box, it won’t look like this. You have to install Oh-my-tmux to make it look pretty. It is pretty easy to install. Just copy the lines and you’re set.
Also, if you are using
bash, I recommend installing Oh-my-bash and if you are working with
zsh, I recommend installing Oh-my-zsh. Actually, we live in the terminal. It shouldn’t look like 80’s anymore. It should look pretty.
Then, you want to install python packages in your environment. But hold on. You can not and should not install them system-wide. This is because multiple projects would require different versions of packages. They may even require different python versions other than preinstalled ones. We need a solution to handle all of them.
Miniconda in my opinion is a very good solution to all of these problems. You can install it using this guide. Once installed, you basically need these commands.
# create a new env conda create -n venv # create a new env with specified python version conda create -n venv python=3.8 # to activate the env conda activate venv
Now for package installation, you can use both
conda install and
pip install. Some people say that you shouldn’t use both of them because it may cause problems. I just tend to use conda for creating and activating environment. Then, installing everything else with
pip and that works just fine.
I can’t think anyone not using this tool. It shows you all processes and their memory usuage and what not.
You can see GPU information using
nvidia-smi. But most of the time, it doesn’t tell you all the information you need at once. You want to undestand who is using GPU or how much time GPU is being used and what not. You can see all you need using
nvidia-htop. For installation it is indeed very easy.
pip install nvidia-htop # then run nvidia-htop.py