Pop!_OS for Machine Learning

I recently switched my desktop OS from Fedora to Pop!_OS and I've been really impressed. Here, I've written down my experiences with it and included notes on how I set it up for machine learning engineering.

Background

Pop!_OS is an Ubuntu-based Linux distribution created by System76. I decided to switch because I'd read that it has really great, almost seamless support for desktop and laptop systems that have NVIDIA GPU. Because it's based on Ubuntu, anything you need from Ubuntu is also available in Pop!_OS.

My desktop has a NVIDIA GeForce GTX 1080 Ti GPU, and while this excellent guide was really helpful in setting it up in Fedora 30/31, I just wanted to try something new while I wait for the Adder WS I ordered recently to be delivered.

Dotfiles

I use stow to manage my dotfiles, and zsh as my $SHELL. Plugins for zsh are managed using zinit.

stow

Install stow:

sudo apt update && sudo apt install -y stow

2. Clone the dotfiles repo locally to $HOME:

cd && git clone git@gitlab.com:bitjockey/dotfiles.git

3. Stow zsh configurations to place them to your home folder:

cd $HOME/dotfiles
stow zsh

zsh

Install zsh:

## Install zsh and set as default SHELL for current user
sudo apt update && sudo apt install -y zsh
chsh -s $(which zsh)

2. Start zsh and install zinit:

zsh
sh -c "$(curl -fsSL https://raw.githubusercontent.com/zdharma/zinit/master/doc/install.sh)"

Python 3

pyenv

I use pyenv to manage Python versions. Here's what I did to set it up.

Install the build dependencies:

sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev python-openssl git

2. Install pyenv:

curl https://pyenv.run | bash

3. Add this to $HOME/.zshrc if using zsh or $HOME/.bashrc if using bash:

export PATH="/home/aj/.pyenv/bin:$PATH"
eval "$(pyenv init -)"

4. Install and set a default Python version:

pyenv install 3.6.9
pyenv global 3.6.9

poetry

For dependency management, I use poetry.

Install poetry:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

2. Add this to $HOME/.zshrc if using zsh or $HOME/.bashrc if using bash:

source $HOME/.poetry/env

3. Restart the shell and see if poetry works:

exec $SHELL
poetry

Tensorman

Tensorman is a tool built by Pop!_OS developers to manage TensorFlow containers.

Installation

Install tensorman:

sudo apt update && sudo apt install -y tensorman nvidia-container-runtime
sudo usermod -aG docker $USER

2. Usage:

# Pull container with latest version of TensorFlow
tensorman pull latest

# Pull container with specific version of TensorFlow
tensorman pull 1.14.0

# Run container with GPU support
tensorman run --gpu --python3 --jupyter bash

Creating a custom container:

Start a named container as root to configure it:

tensorman run -p 8888:8888 --gpu --python3 --jupyter --root --name my_custom_container bash

2. Then inside the container:

# Inside the container:
## Install poetry
pip install poetry

## Disable virtual env creation since we're running inside a container.
poetry config virtualenvs.create false
poetry run pip install pip==20.0.2
poetry install

3. On your host machine, in a different terminal window:

## Fix permissions
sudo chown -R $USER {.config,.cache} && sudo chgrp -R $USER {.config,.cache}

## Save the container as a custom image
tensorman save my_custom_container my_custom_image

4. Run a container based on the new custom image:

tensorman =my_custom_image run -p 8888:8888 --gpu --python3 --jupyter bash

LaTeX

For reports/papers, I use LaTeX and use gummi as an editor.

Install LaTeX and gummi:

sudo apt update && sudo apt install -y texlive-latex-extra gummi

Zotero

I use Zotero to manage research papers and create bibliographies.

On Ubuntu and its derivatives, the recommend installation process is documented here.

Conclusion

See my project here for an example of how to use the tensorman and poetry tools.