Using Python's Virtualenv with RHSCL

I’ve been getting more and more questions about using Python’s virtualenv with python27 and python33 collections from RHSCL, so I decided to write a very short tutorial about this topic. The “tl;dr” version is: everything works perfectly fine as long as you remember to enable the collection first.

What is Virtualenv

Citing Virtualenv official documentation: “virtualenv is a tool to create isolated Python environments”. In short, Virtualenv allows you to setup multiple runtime environments with different sets of Python extension packages on a single machine. Unlike Ruby’s RVM (Ruby Virtual Machine), it can’t install the language interpreter itself – just the extension libraries.

When you create a new virtual environment “foo“, a few things happen:

  • The “foo” directory is created with a few subdirectories: bin, lib, lib64 and include.
  • The bin directory contains python, pythonX and pythonX.Y executables. These are basically aliases for the system Python interpreter executable. This directory also contains activate script (in few variants for different shells) – this is used to activate the environment in the current shell session.
  • Extension packages are installed into the lib directory, lib64 is a symlink that points to lib.
  • Python header files are located in include/pythonX.Y, which is a symlink that points to the include directory of system Python installation.

Creating a Virtual Environment

Creating a virtual environment is easy and it works in the same way for both python27 and python33 collections. Both of these collections contain python-virtualenv RPM, so the only thing you need to do is install the desired collection with yum: yum install python27 or yum install python33. I’m going to show an example using the python33 collection:

# run scl-enabled shell and create the virtual environment
scl enable python33 bash
virtualenv foo
cd foo
source bin/activate

# test your virtualenv by installing Django and printing its version
pip install django
python -c "import django; print(django.__file__)"

# now just run "deactivate" to deactivate the environment
# in current shell session
deactivate
# or just "exit" the current shell, which both terminates
# the virtual environment and SCL-enabled shell
exit

The first four instructions above are all that you need to do to create and activate your virtual environment – the rest of the lines just demonstrate that the environment works properly by installing Django and printing the location from where it was imported. If you have ever worked with Virtualenv before, you’ve probably already noticed that the only difference is that an SCL-enabled bash was run first, all other steps stay the same.

Wrap Up

The only thing you need to remember is to run “scl enable pythonXY bash” before activating the virtual environment. This is the only difference from working with non-SCL Virtualenv. Another nice thing is, that exactly the same commands work for both python27 and python33 collections from RHSCL. I also recommend creating virtual environments with --system-site-packages option, which will allow you to import RPM packaged modules from RHSCL collection.

And that’s all you need to know to work with RHSCL Virtualenv.

Share

  1. Any thoughts on setting the scl version of python as a default for a user’s interactive shell? For example `source /opt/rh/python27/enable` in your .bashrc? I’ve noticed most commonly used tools such as yum or sealert specify /usr/bin/python explicitly in the hashbang and work fine. I guess it would not be safe to assume that this would be true across all RHEL packages?

    1. Sourcing the enable scriptlet works (or you can do this in a script in /etc/profile.d/ to make it systemwide). If you want to make this bulletproof, there’s also an “internal” variable of current scl-utils, X_SCLS, that you should set accordingly, something along the lines of:

      export X_SCLS=”`scl enable python27 ‘echo $X_SCLS’`”

      (also in your bashrc/profiled script). This prevents scl-utils from enabling a collection twice. (not that this is considered to be an implementation detail and can change in future versions of scl-utils). Also note, that if you do this, you won’t be able to disable the collection in your shell session.

      Personally, I think that all system Python executables should point to /usr/bin/python, since they’re built/tested/meant to work with it, but I’m not sure whether it’s always the case (hopefully it is :)).

    1. Python 2.6 is the “base” python in RHEL 6, so there’s no software collection version of it. So if you have RHEL 6, you can use the base python 2.6 and RHSCL versions of 2.7 and 3.3. HTH.

  2. What has worked for us is to export everything you need for pythonXY software collection in .bashrc. That way we don’t have to run scl enable python27….

    export PATH=/opt/rh/python27/root/usr/bin:$PATH:$HOME/bin:${ORACLE_HOME}/bin
    export XDG_DATA_DIRS=/opt/rh/python27/root/usr/share:
    export MANPATH=/opt/rh/python27/root/usr/share/man:
    export PKG_CONFIG_PATH=/opt/rh/python27/root/usr/lib64/pkgconfig:
    export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64:$LD_LIBRARY_PATH:$ORACLE_HOME/lib

  3. what’s worked for us is adding the exports to .bashrc. This allows us to source pythonXY without having to run scl enable pythonXY all the time:

    export PATH=/opt/rh/python27/root/usr/bin:$PATH:$HOME/bin:${ORACLE_HOME}/bin
    export XDG_DATA_DIRS=/opt/rh/python27/root/usr/share:
    export MANPATH=/opt/rh/python27/root/usr/share/man:
    export PKG_CONFIG_PATH=/opt/rh/python27/root/usr/lib64/pkgconfig:
    export LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64:$LD_LIBRARY_PATH:$ORACLE_HOME/lib

  4. this [stinks] because it breaks tons of compilation in pip, also breaks the executing model of virtualen. Try install python2.7 + django1.8 + pymysql on a clean centos6.5. The $PATH does not include the libs path so compilation will fail.

Leave a Reply