How to install multiple versions of Python on Red Hat Enterprise Linux

August 13, 2018
Cedric Clyburn Rob Terzi
Related topics:
LinuxPython
Related products:
Red Hat Enterprise Linux

This article shows how to install Python on Red Hat Enterprise Linux (RHEL), including both versions 3 and 2 of Python along with the pip, vent, virtualenv, and pipenv utilities. The article starts with RHEL 9 and continues with RHEL 8 and RHEL 7, including plenty of tips useful in all versions. The article also covers virtual environments, which help you ensure that your applications have the correct versions of Perl and related modules.

Installing Python 3 on RHEL 9

The default Python implementation for RHEL 9 is Python 3.9. It is not always already installed, however, so make sure to check by using a simple command: python --version. The following command installs Python 3.9 on a RHEL 9 machine:

$ dnf install python3

Now, if that's all you need, you are set. If you want to update to a later version of Python, however, you can install several versions of Python and run them side-by-side. For example, you can run Python versions 3.6, 3.8, and 3.9 by simply using a different command : python3.6, python3.8 or python3.9. All of this is possible by taking advantage of Application Streams. Let's go ahead and install Python 3.11:

$ dnf install python3.11

In order to install additional Python 3 packages, use an additional -requests or -pip like so:

$ dnf install python3-pip

Note: The collection you enable last is the one that will be first in your path, which determines the version you get when you type a command such as python or pip without an explicit version number.

We recommend installing the developer tools such as automaker, GCC, perl, debuggers, and more. For a RHEL 9 system, use the following command to firstly enable the CodeReady Builder repository if you haven't already, then install the Development Tools:

$ sudo subscription-manager repos --enable codeready-builder-for-rhel-9-x86_64-rpms
$ yum group install "Development Tools"

Installing Python 3 on RHEL 8

In Red Hat Enterprise Linux 8, Python 3 is distributed in versions 3.6 (default), 3.8, and 3.9, provided by the python36, python38, and python39 modules, which can be installed in parallel as well. For example, the following command installs Python 3.6 on a RHEL 8 machine:

$ yum install python3

Now, this module stream is enabled automatically, and you can verify this using: python3 --version. Let's go ahead and install Python 3.9 on RHEL 8:

$ yum install python39

To use a newer version of Python, for example Python 3.11, use the following command:

$ yum install python3.11

Verify this specific installation using: python3.11 --version. In addition, for installing the pip package installer, add on the -pip extension to the version of python being installed, for example:

$ yum install python39-pip

 

 Similar to RHEL 9, to install the development tools for RHEL 8, follow the below command:

$ sudo subscription-manager repos --enable codeready-builder-for-rhel-8-x86_64-rpms
$ yum group install "Development Tools"

Installing Python 3 on RHEL 7

While Python 2.7 is the default implementation for RHEL 7, newer versions of the Python interpreter and libraries are available as Red Hat Software Collections packages. Let's go ahead and install Python 2.7 with the following command:

$ yum install python27

Similarly, we can install newer versions of Python for RHEL 7, such as Python 3.6 and 3.8, with the following command:

$ yum install rh-python36

On RHEL 7, to install the development tools, simply use:

$ yum group install "Development Tools"

Using Python 3 on RHEL

To use Python on Red Hat Enterprise Linux, simply use the command that corresponds to the version you wish to use. For example:

  • python3
  • python3.6
  • python3.9

You can use the python command as well, which invokes the default version of Python. You can alter this behavior by using the Linux alternatives command. Better yet, use Python virtual environments, which are discussed later in this article.

Why Develop Python Applications on RHEL?

In general, there are a number of different ways to get Python installed on RHEL. This article introduces Application Streams and Red Hat Software Collections because these give you a current Python installation that is built and supported by Red Hat. During development, support might not seem that important to you. However, support is important to those who have to deploy and operate the applications you write. To understand the importance of having support, consider what happens when your application is in production and a critical security vulnerability in a core library (for example, SSL/TLS) is discovered. This type of scenario is why many enterprises use Red Hat.

Using Python 3 on RHEL 7

This section of this article shows how to install Python 3, pip, venv, virtualenv, and pipenv on Red Hat Enterprise Linux 7. After following the steps in this article, you should be in a good position to follow many Python guides and tutorials using RHEL. Other tips and FAQs for working with Python and software collections on RHEL 7 are also covered. Using Python virtual environments is a best practice to isolate project-specific dependencies and create reproducible environments. With these considerations in mind, the remainder of this article that pertains to Python best practices is applicable, with some nuances.

Virtual Environments on RHEL 7

With Python 3.3 and later, the venv module is now included in the default Python standard library, allowing us to create virtual environments alongside our base Python installation.

Follow these steps:

  1. Under your normal user ID, run scl enable to add python 3 to your path.
  2. Create a Python virtual environment and activate it. (Note: Your prompt changes to show the virtual environment.)
  3. Install whatever additional modules you need with pip in an isolated environment without being root.

The steps can be summarized in the following commands:

$ python3 -m venv py36-venv
$ source py36-venv/bin/activate

(py36-venv) $ python3 -m pip install ...some modules...

If you start a new session, here are the steps for using your virtual environment:

$ python3 -m venv python-venv

RHEL 7: Why use Red Hat Software Collections

The benefit of using Red Hat Software Collections is that you can have multiple versions of Python installed at the same time along with the base Python 2.7 that shipped with RHEL 7. You can easily switch between versions with scl enable.

Note: The latest stable packages for .NET Core, Go, Rust, PHP 7, Ruby 2.5, GCC, Clang/LLVM, Nginx, MongoDB, MariaDB, PostgreSQL, and more can be installed as software collections using yum. So you should take the time to get comfortable with software collections.

Using software collections requires an extra step because you have to enable the collection you want to use. Enabling just adds the necessary paths (PATH, MANPATH, LD_LIBRARY_PATH) to your environment. Once you get the hang of it, software collections are fairly easy to use.

It helps to understand how environment variable changes work in Linux and UNIX. Changes can be made only to the current process. When a child process is created, it inherits the environment of the parent. Any environment changes made in the parent after the child has been created will have no effect on the child. Correspondingly, changes to environment variables in the child have no effect on the parent.

Therefore, the changes made by scl enable affect only the current terminal session or anything started from it. This article also shows how you can permanently enable a software collection for your user account.

RHEL 7: Installation prerequisites

Installing development tools, including GCC, make, and git

If you install modules that depend on compiled C or C++ code, you'll need the tools to compile them. If you haven't already installed development tools, run the following command:

$ su -
# yum install @development

Enabling repositories with additional developer tools

While the default, base RHEL software repositories have many development tools, these are the older versions that are shipped with the operating system (OS) and are supported for the full 10-year life of the OS. Packages that are updated more frequently and have a different support lifecycle are distributed in other repositories that aren't enabled by default.

Red Hat Software Collections are in the rhscl repo. Red Hat Software Collections packages have some dependencies on packages in the optional-rpms repository, so you need to enable both.

To enable the additional repositories, run the following commands as root:

$ su -
# subscription-manager repos \
 --enable rhel-7-server-optional-rpms \
 --enable rhel-server-rhscl-7-rpms

Notes:

  • If you are using the workstation variant of RHEL, change -server- to -workstation-.
  • This command needs to be run only once. The repositories will stay enabled. All of the enabled repositories will be searched by yum when installing or updating software.
  • The no-cost RHEL subscription for developers includes access to all of these repositories and the server variant of RHEL. The server variant is a superset.
  • For more information, see the FAQ for the no-cost subscription.

To see which repositories are available for your current subscription, run the following command:

# subscription-manager repos --list

To see which repositories are enabled, use --list-enabled:

# subscription-manager repos --list-enabled

RHEL 7: Installing Python 3

You can now install Python 3.6 (or other versions in RHSCL) with yum:

# yum install rh-python36

Notes:

  • These packages will be installed in /opt/rh/.
  • The commands are not added to your path until you run scl enable, described later.
  • For other versions of Python, use the following as the package or collection name:
    • Python 3.5: rh-python35
    • Python 3.4: rh-python34
    • Python 2.7.13: python27
  • A number of additional packages are installed as dependencies. These include python-devel, pip, setuptools, and virtualenv.
  • The python-devel package contains the files needed to build modules that dynamically link into Python (such as C/C++ code).

Installing additional packages

Optionally, you may want to install the following RPM packages that are part of the software collection:

  • Python tools (rh-python36-python-tools): Tools included with Python 3, 2to3, and idle3
  • Numpy (rh-python36-numpy): A fast multidimensional array facility for Python
  • Scipy (rh-python36-scipy): Scientific tools for Python
  • Six (rh-python36-python-six): Utilities compatible with both Python 2 and 3
  • Sqlalchemy (rh-python36-python-sqlalchemy): A modular and flexible object-relational mapping (ORM) library for Python
  • PyYAML (rh-python36-PyYAML): A YAML parser and emitter for Python
  • Simplejson (rh-python36-python-simplejson): A simple, fast, extensible JSON encoder/decoder for Python

Example:

# yum install rh-python36-numpy \
 rh-python36-scipy \
 rh-python36-python-tools \
 rh-python36-python-six

Note: By default, system modules are not used with Python virtual environments. Use the --system-site-packages option when creating the virtual environment in order to include system modules.

RHEL 7: How to use Python 3 (scl enable)

Python 3 is now installed. You no longer need to run under the root user ID. The rest of the commands should be executed using your normal user account.

As previously mentioned, software collections are installed under /opt/rh and aren't automatically added to your PATH, MANPATH, and LD_LIBRARY_PATH environment variables. The command scl enable makes the necessary changes and runs a specified command. Because of the way environment variables work in Linux (and UNIX), the changes take effect only for the command run by scl enable. You can use bash as the command to start an interactive session. This is one of the most common ways (but not the only way) of working with software collections.

$ scl enable rh-python36 bash
$ python3 -V
Python 3.6.3

$ python -V # python now points to Python 3
Python 3.6.3

$ which python
/opt/rh/rh-python36/root/usr/bin/python

Note: Enabling the Python collection makes the python in your path, with no version number, point to Python 3. /usr/bin/python is still Python 2. You can run Python 2 by typing python2, python2.7, or /usr/bin/python. It is recommended that you use a version number to avoid any ambiguity about what python means. This also applies to other Python commands such as pip, pydoc, python-config, pyvenv, and virtualenv. For more information, see PEP 394.

See the section How to permanently enable a software collection later in this article to permanently put Python 3 in your path.

RHEL 7 and RHEL 8: Creating a Python virtual environment (best practices)

Using Python virtual environments is a best practice to isolate project-specific dependencies and create reproducible environments. Virtual environments avoid conflicting dependencies that lead to dependency hell. Using a virtual environment lets you use pip to install whatever modules you need for your project in an isolated directory under your normal user ID.

You can easily have multiple projects with different dependencies. To work on a specific project, activate the virtual environment, which adds the right directories to your path.

Using virtual environments along with pip list, pip freeze, and a requirements.txt file gives you a reproducible environment in which to run your code. Other people who need to run your code can use the requirements.txt file you generate to create a matching environment.

By default, virtual environments don't use any system installed modules or modules installed under your home directory. From an isolation perspective, and for creating reproducible environments, this is generally considered the correct behavior. However, you can override that isolation by using the argument --system-site-packages.

Should I use venv, virtualenv, or something else?

When you install Python 3 from Red Hat Software Collections, venv, virtualenv, and pip are installed, so you are ready to install whatever modules you choose. Installing Python Modules in the current Python documentation says this:

  • venv is the standard tool for creating virtual environments, and has been part of Python since Python 3.3.
  • virtualenv is a third-party alternative (and predecessor) to venv. It allows virtual environments to be used on versions of Python prior to 3.4, which either don't provide venv at all or aren't able to automatically install pip into created environments.

So for all the recent versions of Python 3, venv is preferred. If you work with Python 2.7, you'll need to use virtualenv.

The commands to create the virtual environments differ only in the module name used. Once created, the command to activate the virtual environment is the same.

Note: For virtualenv, using python3.6 -m virtualenv is recommended instead of using the virtualenv command. See the section Avoiding Python wrapper scripts such as virtualenv: Use the module name later in this article for more information.

RHEL 7 and RHEL 8: Creating and activating a virtual environment with venv

On RHEL 7, if you haven't already done so, enable the rh-python36 collection:

$ scl enable rh-python36 bash

On RHEL 8, the scl enable step is not necessary.

Now create the virtual environment. To avoid any surprises, use an explicit version number for running Python. Here are two examples, using Python 3.6 and Python 3.9. Choose only one of them if you're following along on your machine:

$ python3.6 -m venv myproject1 # for Python 3.6
$ python3.9 -m venv myproject1 # for Python 3.9

Anytime you need to activate the virtual environment, run the following command:

$ source myproject1/bin/activate

Once you've activated a virtual environment, your prompt changes to remind you that you are working in a virtual environment:

(myproject1) $

Note: When you log in again or start a new session, you need to activate the virtual environment using the source command again. You should already have run scl enable on RHEL 7 before activating the virtual environment.

For more information, see Virtual Environments and Packages in the Python 3 tutorial at docs.python.org.

RHEL 7, RHEL 8: Creating and activating a virtual environment with virtualenv

On RHEL 7, if you haven't already done so, enable the rh-python36 collection:

$ scl enable rh-python36 bash

On RHEL 8, the scl enable step is not necessary.

Now create the virtual environment. To avoid any surprises, use an explicit version number for running Python:

$ python3.6 -m virtualenv myproject1

Anytime you need to activate the virtual environment, run the following command. You should already have run scl enable on RHEL 7 before activating the virtual environment:

$ source myproject1/bin/activate

Once you've activated a virtual environment, your prompt changes to remind you that you are working in a virtual environment:

(myproject1) $

Note: When you log in again or start a new session, you need to activate the virtual environment using the source command again. You should already have run scl enable before activating the virtual environment.

For more information, see Installing packages using pip and virtualenv in the Python Packaging User Guide.

RHEL 7 and RHEL 8: Managing application dependencies with pipenv

From the Python Packaging User Guide tutorial, Managing Application Dependencies:

"Pipenv is a dependency manager for Python projects. If you're familiar with Node.js' npm or Ruby's bundler, it is similar in spirit to those tools. While pip alone is often sufficient for personal use, Pipenv is recommended for collaborative projects as it's a higher-level tool that simplifies dependency management for common use cases."

With pipenv you no longer need to use pip and virtualenv separately. pipenv isn't currently part of the standard Python 3 library or Red Hat Software Collections. You can install pipenv using pip. (But see the section Do not run pip install as root (or with sudo) later in the article.) Since pipenv uses virtualenv to manage environments, you should install pipenv without having any virtual environment activated. However, don't forget to enable the Python 3 software collection first if you are running RHEL 7:

Note: The scl enable command is not necessary for RHEL 8.

$ scl enable rh-python36 bash # if you haven't already done so
$ python3.6 -m pip install --user pipenv

Creating and using isolated environments with pipenv work a bit differently from venv or virtualenv. A virtual environment is automatically created if no Pipfile exists in the current directory when you install the first package. However, it's a good practice to explicitly create an environment with the specific version of Python you want to use:

$ scl enable rh-python36 bash # if you haven't already done so
$ mkdir -p ~/pydev/myproject2
$ cd ~/pydev/myproject2
$ pipenv --python 3.6
$ pipenv install requests

To activate a pipenv environment, change into that directory and run pipenv shell:

$ scl enable rh-python36 bash # if you haven't already done so
$ cd ~/pydev/myproject2
$ pipenv shell

pipenv is similar to scl enable in that it doesn't try to modify the current environment with source. Instead, pipenv starts a new shell. To deactivate the new environment, exit the shell. You can also run a command in the pipenv environment by using pipenv run command.

For more information see:

General tips for working with Python

This section summarizes the advice in this article and provides simple practices that can save you a lot of trouble.

The python command: Avoid surprises by using a version number

To avoid surprises, don't invoke the python command. Use an explicit version number in the command, such as python3.6 or python2.7. At a minimum, always use python3 or python2.

If you have followed the steps in this article, you've got more than one version of Python installed on your system. Depending on your path, you might get different versions. Activating and deactivating virtual environments, as well as enabling a software collection, changes your path. So it can be easy to get confused about what version you'll get from typing python.

The same problem occurs with any of the Python utilities such as pip or pydoc. Using version numbers, for example, pip3.6, is recommended. At a minimum use the major version number: pip3. Some more robust alternatives are also described in this article.

Scripts that start with "#!/usr/bin/env python" might break

For many years, the advice was to start scripts with #!/usr/bin/env python to avoid hard-coding paths like /usr/bin or /usr/local/bin in the script. The /usr/bin/env construct searches your path to find Python. Unfortunately, enabling software collections and activating virtual environments can change what's in your path. So a Python 2 script that starts with this construct might suddenly break when your path changes. As the use of virtual environments increases, it is best to no longer use the /usr/bin/env command, because you might get a different installation of Python with different modules.

Avoiding Python wrapper scripts such as virtualenv: Use the module name

Some Python utilities are put in your path as a wrapper script in a bin directory. This is convenient because you can just type pip or virtualenv. Most Python utilities are actually just Python modules with wrapper scripts to start Python and run the code in the module.

The problem with wrapper scripts is the same ambiguity that happens when typing python. Which version of pip or virtualenv do you get when you type the command without a version number? An additional complication is that the utility needs to match the version of Python you intend to be using for the command to run properly. Some subtle (hard to diagnose) problems can occur if you wind up unintentionally mixing versions.

Note: There are several directories where wrapper scripts can reside. Which version you get is dependent on your path, which changes when you enable software collections or activate virtual environments. Modules installed with pip --user put their wrapper scripts in ~/.local/bin, which can get obscured by activating the software collection or a virtual environment.

You can avoid the surprises caused by path issues by running the module directly from a specific version of Python using -m <modulename>. Although this construct involves more typing, it is much safer.

Summary of recommendations in this section:

  • Instead of pip, use python3.6 -m pip.
  • Instead of pyvenv, use python3.6 -m venv.
  • Instead of virtualenv, use python3.6 -m virtualenv.

Do not run pip install as root (or with sudo)

Running pip install as root either directly or by using sudo is a bad idea and will cause you problems at some point. Some of the problems that you could encounter are:

  • Conflicts between the RPM packages and packages installed by pip. The conflicts will most likely show up when you need to install a fixed or upgraded package or module. The installation might fail or, worse, you might wind up with a broken installation. It's best to let yum be the exclusive manager of the files in the system directories.
  • Runtime environments that can't be easily reproduced. It can be difficult to determine which modules were installed via an RPM package or via pip. When you want to run your Python code on another system, what needs to be installed? Does it need to be installed system-wide? Will you get the same versions of the modules you tested your code under?
  • Upgrading modules to solve one dependency can break some other code. Unfortunately, there are many cases where code needs a specific version of a module and newer versions might be incompatible. Running pip install as root means all modules get installed in a system-wide directory, making it hard to determine which modules were installed for a specific application.

In case you think these warnings are overly dire, see this xkcd comic. (Hover your cursor over the picture so you see the alt text.)

Virtual environments allow you to isolate the modules you install for each project from the modules that are part of the Python installation from Red Hat. Using virtual environments is considered a best practice to create isolated environments that provide the dependencies needed for a specific purpose. You don't need to use the --user option when running pip in a virtual environment because it defaults to installing in the virtual environment, which you should have write access to.

If you aren't using virtual environments, or need a module or tool to be available outside of a virtual environment, use pip --user to install modules under your home directory.

Use virtual environments instead of pip --user

Some guides recommend using pip --user. Although this practice is better than running pip as root, using virtual environments is much better for properly isolating the modules you need for a given project or set of projects. pip --user installations use ~/.local, which can be obscured by enabling software collections and activating virtual environments. For modules that install wrapper scripts in ~/.local/bin, running pip --user can cause a mismatch between the wrapper script and the module.

The exception to this advice involves modules and tools that you need to use outside of virtual environments. The primary example is pipenv. You should use pip install --user pipenv to install pipenv. That way, you'll have pipenv in your path without any virtual environments.

Don't use the system Python binary for your own projects

The Python version installed in /usr/bin/python and /usr/bin/python2 is part of the operating system. RHEL was tested with a specific Python release (2.7.5) that will be maintained for the full ten-year supported life of the OS. Many of the built-in administration tools are actually written in Python. Trying to change the version of Python in /usr/bin might actually break some of the OS's functionality.

At some point, you might want to run your code on a different version of the OS. That OS will likely have a different version of Python installed as /usr/bin/python, /usr/bin/python2, or even /usr/bin/python3. The code you write might have dependencies on a specific version that can be best managed through virtual environments and software collections.

The one exception to this advice comes if you are writing system administration tools. In that case, you should use the Python binary in /usr/bin because it has the correct modules and libraries installed for the APIs in the OS.

Note: If you are writing system administration or management tools in Python, you might want to take a look at Ansible. Ansible is written in Python, uses Jinja2 for templating, and provides higher-level abstractions for many system tasks.

Tip: If you need to work with Python 2.7, install the python27 software collection. Follow the installation steps described earlier in this article, but substitute python27 for rh-python36. You can enable both collections at the same time, so you'll have both python2.7 and python3.6 in your path.

Don't change or overwrite /usr/bin/python, /usr/bin/python2, or /usr/bin/python2.7

As explained in the previous section, the system Python is part of Red Hat Enterprise Linux 7 and is used by critical system utilities such as yum. (Yes, yum is written in Python.) So overwriting the system Python is likely to break your system—badly. If you try to compile Python from source, do not run make install (as root) without using a different prefix or you will overwrite /usr/bin/python.

RHEL 7: Software collection tips

These practices are specific to version 7 of Red Hat Enterprise Linux.

Enable the Python collection before the virtual environment

You should always enable the Python software collection before using any of Python virtual environment utilities to create or activate an environment. In order for things to work correctly, you need to have your desired version of Python in your path because it will be needed by the Python virtual environment. A number of problems, some of which are subtle, come up if you try to enable and activate in the wrong order.

Example for venv:

$ scl enable rh-python36 bash
$ python3.6 -m venv myproject1
$ source myproject1/bin/activate

When reactivating later in a new shell:

$ scl enable rh-python36 bash
$ source myproject1/bin/activate

Example for virtualenv:

$ scl enable rh-python36 bash
$ python3.6 -m virtualenv myproject1
$ source myproject1/bin/activate

When reactivating later in a new shell:

$ scl enable rh-python36 bash
$ source myproject1/bin/activate

How to permanently enable a software collection

To permanently add Python 3 to your path, you can add an scl_source command to the "dot files" for your specific user ID. The benefit of this approach is that the collection is already enabled at every login. If you are using a graphical desktop, everything that you start from the menu will already have the collection enabled.

There are a few caveats with this approach:

  • When you type python with no version number, you will get Python 3 instead of Python 2. You can still get Python 2 by typing python2 or python2.7. Using an explicit version number is strongly recommended.
  • The advice in this section also applies to other Python commands such as pip, pydoc, python-config, pyvenv, and virtualenv. Use a version number to avoid surprises.
  • There is no scl disable command. Everything is in environment variables, so you can work around the lack of this command, but it would be a manual process. You can, however, enable a different software collection that then takes precedence over the collection in your profile.

Using your preferred text editor, add the following line to your ~/.bashrc:

# Add RHSCL Python 3 to my login environment
source scl_source enable rh-python36

Note: You could also add the scl_source line to the start of a build script to select the desired Python for the build. If your build procedure isn't written as a shell script, you could just wrap it in a shell script that has the source scl_source command and that then runs your build script.

How to use Python 3 from RHSCL in the #! (shebang) line of a script

You can create a script that uses Python from the software collection requiring scl enable to be manually run first. Use /usr/bin/scl enable as the interpreter for the script:

#!/usr/bin/scl enable rh-python36 -- python3
import sys

version = "Python %d.%d" % (sys.version_info.major, sys.version_info.minor)
print("You are running Python",version)

Note: You might be tempted to try using just the full path to root/usr/bin/python without the scl enable command. In many cases, this won't work. The behavior is dependent on the specific software collection. For most collections, the command fails with a shared library error, since LD_LIBRARY_PATH isn't set correctly. The python27 collection doesn't give an error but it finds the wrong shared library, so you get the wrong version of Python, which can be surprising. You can call rh-python36 directly without setting LD_LIBRARY_PATH, but that Python collection is currently the only one that works that way. There is no guarantee that future collections will work the same way.

How to see which software collections are installed

You can use the scl -l command to see which software collections are installed. The command shows all software collections that are installed, whether they are enabled or not:

$ scl -l
python27
rh-python36

How to tell which software collections are enabled

The environment variable X_SCLS contains a list of the software collections that are currently enabled:

$ echo $X_SCLS
$ for scl in $X_SCLS; do echo $scl; done
rh-python36
python27

In scripts, you can use scl_enabled <collection-name> to test whether a specific collection is enabled.

How to find a list of Red Hat Software Collections and how long they are supported

The Red Hat Software Collections Product Life Cycle page on the Red Hat Customer Portal contains a list of Red Hat Software Collections packages and support information. You can also check the release notes for the most recent release of Red Hat Software Collections.

Finding additional RPM packages and seeing other available versions

You can use yum search to search for additional packages and see the other versions that are available. Thus, to search for packages that are part of the rh-python36 collection, enter:

# yum search rh-python36

Starting with the Python 3.4 collection, the collection and package names are all prefixed with rh-. So you can use the following command to see all of the rh-python packages and, therefore, which collections are available:

# yum search rh-python

To see the available packages in the Python 2.7 collection, search for python27:

# yum search python27

You can, of course, just search for python and get a list of every available RPM that has python in the name or description. You will get a very long list, so it's best to redirect the output to a file and use grep or a text editor to search the file. The packages that start with python- (without a version number) are part of the base RHEL Python 2.7.5 packages that are installed in /usr/bin.

Troubleshooting

This section lists common problems that developers see when using Python on RHEL.

Python: error while loading shared libraries

This error occurs when you are trying to run a binary that can't find the shared libraries it depends on. Typically this occurs when trying to run python from a software collection without enabling the collection first. In addition to setting PATH, scl enable also sets LD_LIBRARY_PATH. This adds the directory containing the software collection's shared objects to the library search path.

To see what environment variables are modified, take a look at the /opt/rh/rh-python/enable file:

$ cat /opt/rh/rh-python36/enable
export PATH=/opt/rh/rh-python36/root/usr/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/opt/rh/rh-python36/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/rh/rh-python36/root/usr/share/man:$MANPATH
export PKG_CONFIG_PATH=/opt/rh/rh-python36/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}
export XDG_DATA_DIRS="/opt/rh/rh-python36/root/usr/share:${XDG_DATA_DIRS:-/usr/local/share:/usr/share}"

Wrong version of Python when running the python command

First, running python with no version number is likely to give you an unexpected version of Python at some point. The result is dependent on your PATH, which depends on whether you've enabled the software collection and activated the virtual environment. If you use a version number such as python3.6 and you haven't enabled and activated the right environment, you'll get a simple "command not found" error.

You can get the wrong version if you've forgotten to enable the software collection. Enabling the software collection puts the collection's /bin directory in your path first, hiding all of the other versions of commands with the same name.

The software collection needs to be enabled even if you give the full path to the python binary. For most of the collections, you'll get a shared library error ( described earlier) if the library path isn't set correctly. However, if you try running the python27 collection, you'll get Python 2.7.5 (the default version) instead of Python 2.7.13 as you'd expect. This is because the shared library dependency is satisfied out of /lib instead of from the software collection, so you pick up the system Python.

Error running pip: ImportError cannot import name 'main'

If you run pip upgrade --user pip, as some guides suggest, the pip command no longer works. The problem is a path issue combined with an incompatibility between versions. The user installation of pip places a new pip command in ~/.local/bin. However, ~/.local/bin comes after the software collection in your path. So you get the older wrapper script that is incompatible with the newer module.

You can work around this error in several ways:

  • Use virtual environments. Once you create or activate a virtual environment, you'll get the correct pip wrapper script in the bin directory of the virtual environment.
  • Run pip as a module: python3.6 -m pip install ... (See the section Avoiding Python wrapper scripts such as virtualenv: Use the module name earlier in this article.)
  • Don't upgrade pip outside of virtual environments.
  • Use the full path to the pip wrapper script: ~/.local/bin/pip3.6.
  • Add ~/.local/bin as the first directory in your PATH after enabling the Python software collection.

To uninstall the upgraded pip that was installed in ~/.local, run the following command under your regular user ID (not root):

$ python3.6 -m pip uninstall pip

Can't find virtualenv3.6

The rh-python36 software collection includes the virtualenv wrapper script but does not have a link for virtualenv3.6. There are two workarounds for this, but first I should point out that venv is now the Python 3 preferred tool for virtual environments.

The preferred workaround is to avoid the wrapper script entirely and invoke the module directly:

$ python3.6 -m virtualenv myproject1

Alternatively, you can create your own symbolic link in your ~/bin directory:

$ ln -s /opt/rh/rh-python36/root/usr/bin/virtualenv ~/bin/virtualenv3.6

More information: Developing in Python on Red Hat platforms

Nick Coghlan and Graham Dumpleton gave a talk Developing in Python on Red Hat Platforms at DevNation 2016. The talk is chock full of information and still very relevant. They include information on building Python applications using containers, using Source-to-Image (S2I) builds, and deploying to Red Hat OpenShift. I recommend watching the video or at least reviewing the slides.

Summary

After reading this article, you've learned:

  • How to install Python 3 and other versions of Python that are supported by Red Hat using Red Hat Software Collections and Application Streams on Red Hat Enterprise Linux.
  • Why Python virtual environments are a best practice for installing Python modules while isolating dependencies in order to avoid conflicts. You can create and activate virtual environments with venv and virtualenv. Both tools are installed as part of the software collection.
  • Using pipenv, a tool that is similar to npm, which is recommended by the Python Packaging Guide for managing application dependencies, especially on shared projects. pipenv provides one command that integrates pip and virtualenv.
  • Practices to avoid, such as:
    • Running pip install as root, which creates conflicts with RPM packages installed by yum
    • Typing python without a version number, which creates ambiguity about which version is running and possible surprises caused by that confusion
    • Modifying /usr/bin/python, because many system management tools such as yum depend on it and might break
  • Tips for working with Red Hat Software Collections:
    • Enabling the Python software collection before using virtual environments
    • How to permanently enable a software collection so you'll always have Python 3 in your path
    • How to use Python 3 from RHSCL in the #! (shebang) line of a script
  • How to troubleshoot common problems such as:
    • Python: error while loading shared libraries
    • pip upgrade breaks pip with: ImportError cannot import name 'main'
    • Wrong version of Python when typing python
Last updated: February 27, 2024