Have you wanted to use software collections but found packaging has kept you at bay? Tried rebuilding a package only to find it give you weird errors you've not seen before? In this blog post we'll learn how to configure and use mock to build RPM packages for the Python 2.7 Software Collection. Along the way we'll learn why we can't use standard mock configurations, and what makes Software Collections (SCL) mock configurations different.
For readers unfamiliar with mock, I'll introduce it by borrowing a quote from its Fedora Wiki page:
[Mock] can build packages for different architectures and different Fedora or RHEL versions than the build host has. Mock creates chroots and builds packages in them. Its only task is to reliably populate a chroot and attempt to build a package in that chroot.
As we've learned from the other fantastic Software Collections related posts on this blog, in practice SCL is most often used to provide newer versions of software on version-locked platforms where major updates are no longer incorporated into the standard package channels. For example, as we'll cover in this blog post, SCL allows us to have a supported Python 2.7 installation available on a Red Hat Enterprise Linux 6 server. Normally such a server would have been limited to Python version 2.6.
General Notes
Any user running mock commands must be in the mock system group. You can check if you're in the group by running the command groups $USER and looking for "mock" in the printed list. See man 1 mock for group adding instructions.
Why
Before we begin, let's get one thing straight and answer the question of why we can't just use the existing epel-6-x86_64 chroot. What's it missing anyway?
There are three factors prohibiting us from using a default mock chroot:
- Building SCL packages requires special rpm macros
- The SCL packages which provide these macros aren't installed
- The repository providing the packages aren't included in the chroot
As it was for me, some of this information may be new to you, even if you're an experienced packager. Without digressing too far, I'll try to summarize what all of this means.
Building SCL packages requires special RPM macros specific to your target SCL. We can think of the RPM macro system as a templating language, and each macro being used for the convenience of writing simpler-to-read RPM spec files. Our standard epel6 chroot does not have these macros installed.
Typically RPM macros are distributed via packages with the string -macros somewhere in the name. SCL macro packages use a slightly different convention. SCL macro package names end with -build. For example, building a Python27 SCL package requires the scl-utils-build and python27-build packages. As you may have already guessed, these packages are not installed in our epel6 chroot.
Finally, the reason we cannot simply install these new packages in our existing chroot is because they aren't available in repositories included in the standard chroot.
To enable us to have these macros available we'll have to configure the chroots setup command to install the required packages. In addition to updating the setup command, we'll need to include a repository which provides these packages.
Configuring Our New chroot
We begin creating our new mock EPEL6 SCL chroot by making a copy of the existing epel-6-x86_64 definition. We'll label it epel-6-sclpy27-x86_64:
$ cp /etc/mock/epel-6-x86_64.cfg ./epel-6-sclpy27-x86_64.cfg
Now we'll customize the new chroot to fit our needs. To review, this means:
- The SCL and python27
-buildpackages are installed - The repository these packages live in is available for yum
Open epel-6-sclpy27-x86_64.cfg in your favorite text editor and let's get to work!
If you're familiar with the Python language you should be very at-home editing this file. What we're seeing is actually a simple dictionary (or whatever it's called in you lingua franca [language of choice]) variable named config_opts with some keys being set. If your editor includes a Python editing mode, I recommend switching into it now.
The three keys of particular interest to us are:
root- the label of this chrootchroot_setup_cmd- the yum command to install necessary build packagesyum.conf- a multi-line string which is copied into /etc/yum.conf
First we set therootkey to the name of this configuration file without the.cfgsuffix. Assuming you followed the examples above, then this value will beepel-6-sclpy27-x86_64:
config_opts['root'] = 'epel-6-sclpy27-x86_64'
Next we update the chroot_setup_cmd key to install the scl-utils-build and python27-build packages as well:
config_opts['chroot_setup_cmd'] = 'install @buildsys-build scl-utils-build python27-build'
Because -build subpackages were not distributed as part of RHSCL 1.0, the easiest way to make them available (without getting the source and rebuilding them yourself) is by adding the EPEL6 Python27 SCL repository to the yum.conf key.
Take special care when updating this key. For readers not familiar with Python syntax the triple-double-quotes (""") after the assignment operator (=) indicate a multi-line string. This means the value of the config_opts['yum.conf'] key doesn't stop until where the second """ appears at the end of the file.
To keep things simple, let's add our new repository right before the second triple-double-quotes appear:
config_opts['yum.conf'] = """
...
[python27scl]
name=Python27 - epel-6-x86_64
baseurl=https://www.softwarecollections.org/repos/rhscl/python27/epel-6-x86_64
enabled=1
gpgcheck=0
"""
Now we've finished configuring our new mock chroot. For reference, I have posted the full contents of both my original, and updated, chroot configuration files as a Gist on GitHub.
Building and Initializing Our Mock chroot
The simplest way for mock to be able to use our new chroot is to copy the definition into the default chroot directory, /etc/mock/:
$ sudo cp epel-6-sclpy27-x86_64.cfg /etc/mock/
If you don't have super-user rights on your system you can leave the file right where it is for now. We'll review how to get around this later.
Our chroot must be initialized before we can use it for building SCL packages. If your definition is in the global chroot directory:
$ mock -r epel-6-sclpy27-x86_64 --init
If your definition is in another directory we'll have to copy two required files to the that directory first:
$ cp /etc/mock/{site-defaults.cfg,logging.ini} .
Now we can initialize the chroot, just don't forget to use the --configdir option from now on when you run mock commands. This option argument should be the directory you saved the configuration files in.
$ mock --configdir=. -r epel-6-sclpy27-x86_64 --init
Quick Verification
We can verify our chroot's yum.conf file was installed correctly by opening a shell in the chroot and examining the file directly (don't forget to use --configdir if you used it in the last step):
$ mock -r epel-6-sclpy27-x86_64 --shell
INFO: mock.py version 1.1.41 starting...
Start: init plugins
INFO: selinux disabled
Finish: init plugins
Start: run
Start: lock buildroot
Start: device setup
Finish: device setup
Start: shell
[root@deepfryer /]# tail -5 /etc/yum.conf
[python27scl]
name=Python27 - epel-6-x86_64
baseurl=https://www.softwarecollections.org/repos/rhscl/python27/epel-6-x86_64
enabled=1
gpgcheck=0
[root@deepfryer /]#
Build an SCL Package
Let's test this new chroot out. We'll take an SCL-aware source RPM and use mock's --rebuild command to build the binary RPM. For reader convenience, I have posted the srpm used in this example online.
sha256sum: bc9b2dbc5d2e296ea13977b9a753206946d157c23a35c84eabb8f19ac228ae4b
The actual build command is highlighted in line 1 in the following example:
$ mock --rebuild -r epel-6-sclpy27-x86_64 re-core-0.0.6-8.fc20.src.rpm
INFO: mock.py version 1.1.41 starting...
Start: init plugins
INFO: selinux disabled
Finish: init plugins
Start: run
INFO: Start(./rpm-build/re-core-0.0.6-8.fc20.src.rpm) Config(epel-6-sclpy27-x86_64)
...
Finish: rpmbuild -bb re-core-0.0.6-8.fc20.src.rpm
Finish: build phase for re-core-0.0.6-8.fc20.src.rpm
INFO: Done(./rpm-build/re-core-0.0.6-8.fc20.src.rpm) Config(epel-6-sclpy27-x86_64) 0 minutes 9 seconds
INFO: Results and/or logs in: /var/lib/mock/epel-6-sclpy27-x86_64/result
Finish: run
To verify the package was built correctly, let's list the contents of our freshly built RPM and print out the package requirements.
$ rpm -qpl /var/lib/mock/epel-6-sclpy27-x86_64/result/python27-re-core-0.0.6-8.el6.noarch.rpm
/opt/rh/python27/root/usr/bin/re-core
/opt/rh/python27/root/usr/lib/python2.7/site-packages/re_core-0.0.6-py2.7.egg-info
/opt/rh/python27/root/usr/lib/python2.7/site-packages/re_core-0.0.6-py2.7.egg-info/PKG-INFO
/opt/rh/python27/root/usr/lib/python2.7/site-packages/re_core-0.0.6-py2.7.egg-info/SOURCES.txt
/opt/rh/python27/root/usr/lib/python2.7/site-packages/re_core-0.0.6-py2.7.egg-info/dependency_links.txt
/opt/rh/python27/root/usr/lib/python2.7/site-packages/re_core-0.0.6-py2.7.egg-info/top_level.txt
...
/opt/rh/python27/root/usr/share/doc/python27-re-core-0.0.6/AUTHORS
/opt/rh/python27/root/usr/share/doc/python27-re-core-0.0.6/LICENSE
/opt/rh/python27/root/usr/share/doc/python27-re-core-0.0.6/README.md
/opt/rh/python27/root/usr/share/doc/python27-re-core-0.0.6/settings-example.json
The /opt/rh/ path prefixes are our first success indicator! Now let's check out those package requirements. We want to see "python27-" prefixes here:
$ rpm -qpR /var/lib/mock/epel-6-sclpy27-x86_64/result/python27-re-core-0.0.6-8.el6.noarch.rpm
/opt/rh/python27/root/usr/bin/python2
python27-python(abi) = 2.7
python27-python-pika
python27-python-pymongo
python27-python2-devel
python27-pytz
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PartialHardlinkSets) <= 4.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1
Perfect! Now we've successfully created a new mock chroot for building Python27 SCL packages. The next time we need to build a package for this SCL we can use the same chroot.
Closing Notes
If you're building new packages for an SCL and find yourself having to build one or more new SCL enabled dependencies for that SCL, you may find value in creating a yum repository locally and including that in the new chroot as well. This SCL mailing list post has an example of how to do exactly that.
For more information on packaging with SCL macros, check out the SCL Packaging Guidelines available on the Fedora Project wiki.
Last updated: July 28, 2022