Extending software collections with additional packages

Recent Red Hat Developer Blog articles have explained what Software Collections are, what they're good for, and how you can leverage them to easily enhance your development environment leveraging the latest and greatest versions of your favorite programming languages and relational database backends.

But the software collection packages available for each language comprise only a subset of what is available — usually the core and some popular additions. Until that day when repositories have sprung up with all the additional SCL builds of libraries, modules, extensions to your language collection, you'll need to build those yourself.

Fortunately, it's not very difficult.

This article assumes you've built RPM packages before using the rpmbuild utility and any prior experience with the mock tool is also beneficial before diving in. If you want some more background, check out this article and this post in the Red Hat Customer Portal.

Before building any SCL packages, there are a few packages you'll want to install with yum:

sudo yum install scl-utils-build python-setuptools

The scl-utils-build package adds, among some other things, several important RPM macros in /etc/rpm/macros.scl.

Overview of the SCL build process

Here are the steps required for building a package for a software collection.

1. Obtain/create a source RPM or a spec file and source tarball
2. Modify the spec file with SCL directives
3. Build source and binary RPMs for a specific collection

SCL-enabling a spec-file

The Red Hat Software Collections documentation offers detailed information on what specifically needs to be added or modified in a spec file so that you may build a package for a software collection. It's usually easier to just use a handy script called spec2scl.

The spec2scl script is installable with yum via the EPEL repository, but the developer has been actively updating this tool and, as a result, the version available in the EPEL repository is somewhat dated. It is advised that you use Python's easy_install utility to install the most up-to-date version of spec2scl:

sudo easy_install spec2scl

Building additional Perl CPAN modules for the perl516 collection

As an example and an illustration, this article covers building additional packages for the perl516 collection.

The perl516 collection contains the basic packages you will need to use Perl 5.16 on an EL6 system including modules like DBI, DBD::Pg, FCGI, and YAML, but leaves out a large number of popular CPAN modules. For example, Text::CSV_XS is commonly used for reading and writing comma-delimited text files and is not included in the perl516 collection, but that's okay because we can build it ourselves.

We'll want to use the cpanspec tool to download CPAN modules and construct spec files, so install that.

sudo yum install cpanspec

Next, you can use the cpanspec tool to download a module's source tarball and construct a basic spec file:

cpanspec Text::CSV_XS

One notable behavior of cpanspec is it stores the source tarball it downloads and the spec file it builds in the current working directory. If you're working from your SPECS directory, you'll probably want to move the source tarball downloaded by cpanspec into the ../SOURCES directory.

Next, run spec2scl to add the necessary SCL macros to the spec file.

spec2scl -i -m perl-Text-CSV_XS.spec

The -i option specifies that spec2scl should make in-place modifications to the spec file. The -m option specifies that the resulting RPM should depend on the SCL runtime package.

You can then run rpmbuild with the spec file. Add -D 'scl perl516' to activate the conditional SCL macros in the spec file.

rpmbuild -ba perl-Text-CSV_XS.spec -D 'scl perl516'

This should build an installable RPM package.

Using mock to build and maintain a repository of software collection packages

The process described above is effective when you're building packages to be installed on the host you're building on and any dependent packages are already installed. Otherwise, you want to use a system that automatically resolves and installs dependencies as part of the build process. That's where mock comes in to play.

mock builds packages in an isolated chroot environment and, because it's isolated from the rest of the system, can be used to build packages for other distributions and architectures. This makes it trivial (and very useful,) for example, to use mock to build packages for EL6 on a Fedora workstation.

As you build packages for your software collection you will undoubtedly encounter packages which depend on others. By placing the dependent packages into a local repository, mock can install them in the chroot environment to build the package that has the dependencies.

You will still want to use spec2scl to prepare a modified spec file for your package, but instead of using rpmbuild you will hand off package building to mock.

First, install mock.

sudo yum install mock

Create a directory that will serve as your local repository for the packages you build.

sudo mkdir -p /var/spool/local_yum_repos/perl516/{source,packages}

Create a copy of a mock configuration file (these are found in /etc/mock) and extend with your base software repository information and local repository information.

sudo cp /etc/mock/epel-6-x86_64.cfg /etc/mock/epel-6-x86_64-perl516-scl.cfg

The lower portion of this file contains yum repository information, similar to what you'd fine in .repo files under /etc/yum.repos.d. For a mock environment that builds on top of the perl516 collection, add the following:

[perl516]
name=perl516
baseurl=https://cdn.redhat.com/content/dist/rhel/server/6/x86_64/rhscl/1/os
enabled=1
[perl516_local]
name=perl516_local
baseurl=file:///var/spool/local_yum_repos/perl516
enabled=1

You'll also want to add some packages to always be installed in the chroot environment that are necessary for building SCL packages. To do this, modify the line near the top of the configuration file that defines the config_opts['chroot_setup_cmd'] option.

config_opts['chroot_setup_cmd'] = 'install @buildsys-build scl-utils-build perl516-build perl516-perl-core'

Run this mock command to initialize the mock root filesystem.

mock -r epel-6-x86_64-perl516-scl --init

Run this mock command to build a source RPM from a spec file.

mock -r epel-6-x86_64-perl516-scl --buildsrpm --sources ~/rpm/SOURCES --spec ~/rpm/SPECS/perl-Text-CSV_XS.spec

Copy the newly built source RPM to your local repository.

cp /var/lib/mock/epel-6-x86_64-perl516-scl/result/perl516-perl-Text-CSV_XS-1.01-1.el6.src.rpm /var/spool/local_yum_repos/perl516/source

Build binary RPM(s) from the source RPM.

mock -r epel-6-x86_64-perl516-scl -D "scl perl516" /var/spool/local_yum_repos/perl516/source/perl516-perl-Text-CSV_XS-0.99-1.el6.src.rpm

Copy the binary RPM(s) into the packages directory of your local repository.

cp /var/lib/mock/epel-6-x86_64-perl516-scl/result/perl516-perl-Text-CSV_XS-1.01-1.el6.x86_64.rpm /var/spool/local_yum_repos/perl516/packages

Regenerate the repository metadata.

cd /var/spool/local_yum_repos/perl516
createrepo .

Now that this newly built package is in the directory created for the local package repository, any time mock builds a package that depends on it, it will be able to install it from the local repository to satisfy the dependency.

Doran Barton is a senior developer at Bluehost. He is a Red Hat Certified Engineer and has worked with Linux since 1995 as a system administrator and as a software developer.