Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

How to add packages to Python 2.7 Software Collection

December 4, 2014
Bohuslav Kabrda Tomas Orsava
Related topics:
Linux
Related products:
Red Hat Enterprise Linux

Share:

    As Software Collections are getting popular, there are more and more people asking how they can build their own collections and/or extend collections in RHSCL. In this article, I will demonstrate how to extend python27 collection from RHSCL 1.2, adding a simple Python extension library. (Note that the same steps can be applied to the python33 collection.) I'm going to work on a RHEL 6 machine throughout this whole tutorial. I'm assuming that readers have basic knowledge of RPM building and Software Collections concept.        

    General Notes

    This tutorial uses a technique that we call "dependent collections". Dependent collections are a systematic approach to the "how can I extend SCLs?" problem, not a new technology, so don't get scared. We will be extending python27 collection by building another collection that depends on it. This is a general approach that can be applied to most of RHSCL 1.2 collections, although some specific steps may differ from collection to collection (read the details).

    Please note, that the "dependent collections" approach as demonstrated here won't work with collections from RHSCL 1.0, so update to the latest version.

    Preparing the Environment

    At first, we must install some development tools to be able to actually build RPMs and SCLs:

    sudo yum install rpmdevtools scl-utils-build

    Then we need to install the python27 collection and its "-scldevel" package (make sure you're subscribed to the correct RHSCL channel):

    sudo yum install python27 python27-scldevel

    The python27-scldevel package contains some macros useful for building collections dependent on python27.

    As the last step, we will create ~/rpmbuild directory with all necessary subdirectories, so that we have a place to do the builds:

    rpmdev-setuptree

    Building a New Collection

    Building a new Software Collection consists of two steps:

    • creating a metapackage
    • converting and building all packages we want to be in the new SCL

    Creating a Metapackage

    A Software Collection metapackage is a package that defines the Software Collection's name (since we'll be packaging Versiontools 1.9.1, let's go with vt191) and holds RPM macros related to the collection and the "enable" scriptlet. Create

    ~/rpmbuild/SPECS/vt191.spec

    with this content (see comments inline for explanation):

    # define name of the scl
    %global scl vt191
    %scl_package %scl
    
    # Defaults for the values for the python27 Software Collection. These
    # will be used when python27-scldevel is not in the buildroot
    %{!?scl_python:%global scl_python python27}
    %{!?scl_prefix_python:%global scl_prefix_python %{scl_python}-}
    
    # Only for this build, we need to override default __os_install_post, because
    # the default one would find /opt/.../lib/python2.7/ and rpmbuild would
    # try to bytecompile with the system /usr/bin/python2.7 (which doesn't exist)
    %global __os_install_post %{%{scl_python}_os_install_post}
    # Similarly, override __python_requires for automatic dependency generator
    %global __python_requires %{%{scl_python}_python_requires}
    
    # The directory for site packages for this Software Collection
    %global vt191_sitelib %(echo %{python27python_sitelib} | sed 's|%{scl_python}|%{scl}|')
    
    Summary: Package that installs %scl
    Name: %scl_name
    Version: 1
    Release: 1%{?dist}
    License: GPLv2+
    BuildRequires: scl-utils-build
    # Always make sure that there is the python27-sclbuild
    # package in the buildroot
    BuildRequires: %{scl_prefix_python}scldevel
    # Require python27-python-devel, we will need macros from that package
    BuildRequires: %{scl_prefix_python}python-devel
    Requires: %{scl_prefix}python-versiontools
    
    %description
    This is the main package for %scl Software Collection.
    
    %package runtime
    Summary: Package that handles %scl Software Collection.
    Requires: scl-utils
    Requires: %{scl_prefix_python}runtime
    
    %description runtime
    Package shipping essential scripts to work with %scl Software Collection.
    
    %package build
    Summary: Package shipping basic build configuration
    Requires: scl-utils-build
    # Require python27-scldevel so that there is always access
    # to the %%scl_python and %%scl_prefix_python macros in builds for this Software
    # Collection
    Requires: %{scl_prefix_python}scldevel
    
    %description build
    Package shipping essential configuration macros to build %scl Software Collection.
    
    %prep
    %setup -c -T
    
    %install
    %scl_install
    
    # Create the enable scriptlet that:
    # - Adds an additional load path for the Python interpreter.
    # - Runs scl_source so that you can run:
    # scl enable vt191 "bash"
    # instead of:
    # scl enable python27 vt191 "bash"
    
    cat >> %{buildroot}%{_scl_scripts}/enable << EOF
    . scl_source enable %{scl_python}
    export PYTHONPATH=%{vt191_sitelib}${PYTHONPATH:+:${PYTHONPATH}}
    EOF
    
    mkdir -p %{buildroot}%{vt191_sitelib}
    
    # - Enable Software Collection-specific bytecompilation macros from
    # the python27-python-devel package.
    # - Also override the %%python_sitelib macro to point to the vt191 Software
    # Collection.
    # - If you plan to add some architecture-dependent packages, you will also need to override
    # the %%python_sitearch macro.
    
    cat >> %{buildroot}%{_root_sysconfdir}/rpm/macros.%{scl}-config << EOF
    %%scl_package_override() %%{expand:%{?python27_os_install_post:%%global __os_install_post %%python27_os_install_post}
    %%global __python_requires %%python27_python_requires
    %%global __python_provides %%python27_python_provides
    %%global __python %python27__python
    %%global __python2 %python27__python
    %%global python_sitelib %vt191_sitelib
    %%global python2_sitelib %vt191_sitelib
    }
    EOF
    
    %files
    
    %files runtime
    %scl_files
    
    %scl_files
    %vt191_sitelib
    
    %files build
    %{_root_sysconfdir}/rpm/macros.%{scl}-config
    
    %changelog
    * Wed Jan 22 2014 John Doe <jdoe@example.com> - 1-1
    - Initial package.

    Build this package by running:

    cd ~/rpmbuild/SPECS
    rpmbuild -ba vt191.spec

    And then install vt191-build and vt191-runtime so that we can build packages for vt191 collections:

    sudo yum install ~/rpmbuild/RPMS/x86_64/vt191-{build,runtime}-1-1.el6.x86_64.rpm

    Converting and Building Packages

    We will be building Versiontools 1.9.1 as the only package for our collection. The specfile -

    ~/rpmbuild/SPECS/python-versiontools.spec
    • looks like this (again, see comments inline):
    %{?scl:%scl_package python-versiontools}
    %{!?scl:%global pkg_name %{name}}
    
    %global pypi_name versiontools
    
    Name: %{?scl_prefix}python-versiontools
    Version: 1.9.1
    Release: 1%{?dist}
    Summary: Smart replacement for plain tuple used in __version__
    
    License: LGPLv3
    URL: https://launchpad.net/versiontools
    Source0: http://pypi.python.org/packages/source/v/versiontools/versiontools-1.9.1.tar.gz
    
    BuildArch: noarch
    BuildRequires: %{?scl_prefix_python}python-devel
    BuildRequires: %{?scl_prefix_python}python-setuptools
    %{?scl:BuildRequires: %{scl}-build %{scl}-runtime}
    %{?scl:Requires: %{scl}-runtime}
    
    %description
    Smart replacement for plain tuple used in __version__
    
    %prep
    %setup -q -n %{pypi_name}-%{version}
    
    %build
    %{?scl:scl enable %{scl} "}
    %{__python} setup.py build
    %{?scl:"}
    
    %install
    # Explicitly specify --install-purelib %{python_sitelib}, which is now overriden
    # to point to vt191, otherwise Python will try to install into the python27
    # Software Collection site-packages directory
    %{?scl:scl enable %{scl} "}
    %{__python} setup.py install -O1 --skip-build --root %{buildroot} --install-purelib %{python_sitelib}
    %{?scl:"}
    
    %files
    %{python_sitelib}/%{pypi_name}*
    
    %changelog
    * Wed Jan 22 2014 John Doe <jdoe@example.com> - 1.9.1-1
    - Built for vt191 SCL.

    If you're creating lots of packages, try using pyp2rpm for creating specfiles from Python packages and spec2scl to make them SCL ready. Also note, that when converted properly, specfiles can be used for both SCL and non-SCL builds.

    You'll need to download the actual source file from PyPI:

    wget -p --no-directories -P ~/rpmbuild/SOURCES https://pypi.python.org/packages/source/v/versiontools/versiontools-1.9.1.tar.gz

    Now you can build Versiontools:

    rpmbuild -ba python-versiontools.spec

    Then install them:

    sudo yum install ~/rpmbuild/RPMS/noarch/vt191-python-versiontools-1.9.1-1.el6.noarch.rpm

    And see that everything actually works:

    scl enable vt191 "python -c 'import versiontools; print(versiontools.__file__)'"

    The above command will print out the full path to the file that Versiontools are loaded from:

    /opt/rh/vt191/root/usr/lib/python2.7/site-packages/versiontools/__init__.pyc

    Note, that precisely the same approach also works for python33 collection from RHSCL 1.2. This means that you can rebuild vt191 collection as e.g. vt191-p33 without any modifications other than replacing the collection name and interpreter version in macros from vt191.spec.

    Last updated: February 23, 2024

    Recent Posts

    • Meet the Red Hat Node.js team at PowerUP 2025

    • How to use pipelines for AI/ML automation at the edge

    • What's new in network observability 1.8

    • LLM Compressor: Optimize LLMs for low-latency deployments

    • How to set up NVIDIA NIM on Red Hat OpenShift AI

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue