I follow some simple rules for Python packages.

• Never touch the OS site-packages with pip
• Never run pip with escalated privileges (sudo)
• Always use a local ($HOME) pip
• Use the OS package manager when installing to site-packages to avoid competing package managers (pip and yum, for example)

Where’s site-packages located?

python3 -m site
python3 -m user-site

Read up on the Python docs and the pip docs to get a better understanding.

how to win

Assuming you have no traces of pip left.

Grab get-pip.py from the official pip website and install it under $HOME:

python3 get-pip.py --user --ignore-installed

After this command succeeds, a pip binary will be placed in ~/.local/bin. Make sure you have this folder prepended to your $PATH:

export PATH=$HOME/.local/bin:$PATH

Once pip is installed and set up we will install only one more package in the user-environment: pipx.

Make sure you are actually using the local pip by running which pip.

pip install --user pipx

pipx installs each package into ~/.local/pipx/venvs/PKGNAME and then symlinks the scripts (CLI entry points) into ~/.local/bin. This makes it so you don’t have to activate the virtual environments manually to use the programs.

We can use pipx to install pew (a virtualenv manager):

pipx install pew

I prefer pipx over pipsi as it lets you easily upgrade the virtual environments after a system upgrade of Python (see #146):

pipx reinstall-all

overview

pipx --version
pipx list
pip -V

jupyter notebooks

I like to follow the same principles for Jupyter notebooks.

pipsi install notebook

Create a virtualenv that can be used as a notebook kernel:

pew new pandas3 --python=python3
pip install ipykernel
pip install pandas

Create the kernel spec:

ipykernel install --user --name 'python3-pandas' --display-name 'Python 3 (venv pandas3)'

It will be placed in ~/.local/share/jupyter/kernels/python3-pandas/kernel.json

Whenever you want to use pandas in Jupyter you create a new notebook with this kernel.

other

I use the OS package manager to install some packages, for example python3-neovim and python2-neovim in the Debian repos. Hopefully they are optimized to work with the neovim package.

inspiration

Hitchhiker’s guide to Python

Now, tell me how to do it even better.

updates

• <2020-02-16 Sun> - Recommend pipx over pipsi
• <2020-02-16 Sun> - Don’t install virtualenv (use python3 -m venv instead)

Back to posts