Managing Multiple Python Versions With pyenv

Why Not Use System Python?

“System Python” is the Python that comes installed on your operating system. If you’re on Mac or Linux, then by default, when you type python in your terminal, you get a nice Python REPL.

So, why not use it? One way to look at it is that this Python really belongs to the operating system. After all, it came installed with the operating system. That’s even reflected when you run which:

$ which python
/usr/bin/python

Here, python is available to all users as evidenced by its location /usr/bin/python. Chances are, this isn’t the version of Python you want either:

$ python -V
Pyhton 2.7.12

To install a package into your system Python, you have to run sudo pip install. That’s because you’re installing the Python package globally, which is a real problem if another user comes along and wants to install a slightly older version of the package.

Problems with multiple versions of the same package tend to creep up on you and bite you when you least expect it. One common way this problem presents itself is a popular and stable package suddenly misbehaving on your system. After hours of troubleshooting and Googling, you may find that you’ve installed the wrong version of a dependency, and it’s ruining your day.

Even if your Python version is installed in /usr/local/bin/python3, you’re still not safe. You will run into the same permissions and flexibility problems described above.

In addition, you don’t really have much control over what version of Python comes installed on your OS. If you want to use the latest features in Python, and you’re on Ubuntu for example, you might just be out of luck. The default versions might be too old, which means you’ll just have to wait for a new OS to come out.

Finally, some operating systems actually use the packaged Python for operation. Take yum for example, which makes heavy use of Python to do its job. If you install a new version of Python and aren’t careful to install it into your user space, you could seriously damage your ability to use your OS.

What About a Package Manager?

The next logical place to look is package managers. Programs such as apt, yum, brew, or port are typical next options. After all, this is how you install most packages to your system. Unfortunately, you’ll find some of the same problems using a package manager.

By default, package managers tend to install their packages into the global system space instead of the user space. Again, these system level packages pollute your development environment and make it hard to share a workspace with others.

Once again, you still don’t have control over what version of Python you can install. It’s true some repositories give you a greater selection, but by default, you’re looking at whatever version of Python your particular vendor is up to on any given day.

Even if you do install Python from a package manager, consider what would happen if you’re writing a package and want to support and test on Python 3.4 - 3.7.

What would happen on your system when you type python3? How would you switch quickly between the different versions? You can certainly do it, but it is tedious and prone to error. Nevermind the fact that if you want PyPy, Jython, or Miniconda, then you’re probably just out of luck with your package manager.

With these constraints in mind, let’s recap the criteria that would let you install and manage Python versions easily and flexibly:

1. Install Python in your user space
2. Install multiple versions of Python
3. Specify the exact Python version you want
4. Switch between the installed versions

pyenv lets you do all of these things and more.

Build Dependencies

pyenv builds Python from source, which means you’ll need build dependencies to actually use pyenv. The build dependencies vary by platform. If you are on Ubuntu/Debian and want to install the build dependencies, you could use the following:

$ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev \
libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl

This uses Apt to install all the build dependencies. Let this run, and you’ll be ready to go for Debian systems.

If you use Fedora/CentOS/RHEL, you could use yum to install your build dependencies:

$ sudo yum install gcc zlib-devel bzip2 bzip2-devel readline-devel sqlite \
sqlite-devel openssl-devel xz xz-devel libffi-devel

This command will install all the build dependencies for Python using yum.

macOS users can use the following command:

$ brew install openssl readline sqlite3 xz zlib

This command relies on Homebrew and installs the few dependencies for macOS users.

$ sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

If you’re instead using openSUSE then you would run the following:

$ zypper in zlib-devel bzip2 libbz2-devel libffi-devel \
libopenssl-devel readline-devel sqlite3 sqlite3-devel xz xz-devel

Once again, this command installs all the Python build dependencies for your system.

Finally, for Alpine users, you can use this:

$ apk add libffi-dev ncurses-dev openssl-dev readline-dev \
tk-dev xz-dev zlib-dev

This command uses apk as the package manager and will install all build dependencies for Python on Alpine.

Using the pyenv-installer

After you’ve installed the build dependencies, you’re ready to install pyenv itself. I recommend using the pyenv-installer project:

$ curl https://pyenv.run | bash

This will install pyenv along with a few plugins that are useful:

1. pyenv: The actual pyenv application
2. pyenv-virtualenv: Plugin for pyenv and virtual environments
3. pyenv-update: Plugin for updating pyenv
4. pyenv-doctor: Plugin to verify that pyenv and build dependencies are installed
5. pyenv-which-ext: Plugin to automatically lookup system commands

At the end of the run, you should see something like this:

WARNING: seems you still have not added 'pyenv' to the load path.

Load pyenv automatically by adding
the following to ~/.bashrc:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

The output will be based on your shell. But you should follow the instructions to add pyenv to your path and to initialize pyenv/pyenv-virtualenv auto completion. Once you’ve done this, you need to reload your shell:

$ exec "$SHELL" # Or just restart your terminal

That’s it. You now have pyenv and four useful plugins installed.

Installation Location

As mentioned before, pyenv works by building Python from source. Each version that you have installed is located nicely in your pyenv root directory:

$ ls ~/.pyenv/versions/
2.7.15  3.6.8  3.8-dev

All of your versions will be located here. This is handy because removing these versions is trivial:

$ rm -rf ~/.pyenv/versions/2.7.15

Of course pyenv also provides a command to uninstall a particular Python version:

$ pyenv uninstall 2.7.15

Using Your New Python

Now that you’ve installed a couple of different Python versions, let’s see some basics on how to use them. First, check what versions of Python you have available:

$ pyenv versions
* system (set by /home/realpython/.pyenv/version)
  2.7.15
  3.6.8
  3.8-dev

The * indicates that the system Python version is active currently. You’ll also notice that this is set by a file in your root pyenv directory. This means that, by default, you are still using your system Python:

$ python -V
Python 2.7.12

If you try to confirm this using which, you’ll see this:

$ which python
/home/realpython/.pyenv/shims/python

This might be surprising, but this is how pyenv works. pyenv inserts itself into your PATH and from your OS’s perspective is the executable that is getting called. If you want to see the actual path, you can run the following:

$ pyenv which python
/usr/bin/python

If, for example, you wanted to use version 2.7.15, then you can use the global command:

$ pyenv global 2.7.15
$ python -V
Python 2.7.15

$ pyenv versions
  system
* 2.7.15 (set by /home/realpython/.pyenv/version)
  3.6.8
  3.8-dev

$ pyenv global 3.8-dev
$ python -m test

If you ever want to go back to the system version of Python as the default, you can run this:

$ pyenv global system
$ python -V
Python 2.7.12

You can now switch between different versions of Python with ease. This is just the beginning. If you have many versions that you want to switch between, typing these commands consistently is tedious. This section goes over the basics, but a better workflow is described in working with multiple environments.

install

You’ve already seen the install command above. This command can be used to install a specific version of Python. For example, if you wanted to install 3.6.8 you would use this:

$ pyenv install 3.6.8

The output shows us pyenv downloading and installing Python. Some of the common flags you may want to use are the following:

versions

The versions command displays all currently installed Python versions:

$ pyenv versions
* system (set by /home/realpython/.pyenv/version)
  2.7.15
  3.6.8
  3.8-dev

This output shows not only that 2.7.15, 3.6.8, 3.8-dev, and your system Python are installed, but also shows you that the system Python is active. If you only care about the current active version, you can use the following command:

$ pyenv version
system (set by /home/realpython/.pyenv/version)

This command is similar to versions but only shows you the current active Python version.

which

The which command is helpful for determining the full path to a system executable. Because pyenv works by using shims, this command allows you to see the full path to the executable pyenv is running. For example, if you wanted to see where pip is installed, you could run this:

$ pyenv which pip
/home/realpython/.pyenv/versions/3.6.8/bin/pip

The output displays the full system path for pip. This can be helpful when you’ve installed command-line applications.

global

The global command sets the global Python version. This can be overridden with other commands, but is useful for ensuring you use a particular Python version by default. If you wanted to use 3.6.8 by default, then you could run this:

$ pyenv global 3.6.8

This command sets the ~/.pyenv/version to 3.6.8. For more information, see the section on specifying your Python version.

local

The local command is often used to set an application-specific Python version. You could use it to set the version to 2.7.15:

$ pyenv local 2.7.15

This command creates a .python-version file in your current directory. If you have pyenv active in your environment, this file will automatically activate this version for you.

shell

The shell command is used to set a shell-specific Python version. For example, if you wanted to test out the 3.8-dev version of Python, you can do this:

$ pyenv shell 3.8-dev

This command activates the version specified by setting the PYENV_VERSION environment variable. This command overwrites any applications or global settings you may have. If you want to deactivate the version, you can use the --unset flag.

Creating Virtual Environments

Creating a virtual environment is a single command:

$ pyenv virtualenv <python_version> <environment_name>

Technically, the <python_version> is optional, but you should consider always specifying it so that you’re certain of what Python version you’re using.

The <environment_name> is just a name for you to help keep your environments separate. A good practice is to name your environments the same name as your project. For example, if you were working on myproject and wanted to develop against Python 3.6.8, you would run this:

$ pyenv virtualenv 3.6.8 myproject

The output includes messages that show a couple of extra Python packages getting installed, namely wheel, pip, and setuptools. This is strictly for convenience and just sets up a more full featured environment for each of your virtual environments.

Activating Your Versions

Now that you’ve created your virtual environment, using it is the next step. Normally, you should activate your environments by running the following:

$ pyenv local myproject

You’ve seen the pyenv local command before, but this time, instead of specifying a Python version, you specify an environment. This creates a .python-version file in your current working directory and because you ran eval "$(pyenv virtualenv-init -)" in your environment, the environment will automatically be activated.

You can verify this by running the following:

$ pyenv which python
/home/realpython/.pyenv/versions/myproject/bin/python

You can see a new version has been created called myproject and the python executable is pointing to that version. If you look at any executable this environment provides, you will see the same thing. Take, for example, pip:

$ pyenv which pip
/home/realpython/.pyenv/versions/myproject/bin/pip

If you did not configure eval "$(pyenv virtualenv-init -)" to run in your shell, you can manually activate/deactivate your Python versions with this:

$ pyenv activate <environment_name>
$ pyenv deactivate

The above is what pyenv-virtualenv is doing when it enters or exits a directory with a .python-version file in it.