Continuous Integration Strategies (Part 3 of 3)

This is part 3 of a three part article on Continuous Integration Strategies:  Simplifying application development on Red Hat® Enterprise Linux®

SOFTWARE COLLECTIONS

The testing environment requirements are simplified somewhat if your software is built against a Red Hat Software Collection (RHSCL).

Red Hat Software Collections provide three primary features: the ability to have multiple versions of software on one OS, certainty that the software is the same across versions of the OS, and the option to refresh components more frequently than the underlying OS.
As a result, testing need only be performed against the software collection for which the application has been built—so an application developed to run on the software collection for Python 2.7 should be testable on that software collection running on any version of Red Hat Enterprise Linux. The test results should then be valid for all versions of Red Hat Enterprise Linux that run that software collection.

REQUIREMENTS FOR TESTS

Test coverage within the CI environment should be considered no differently than any other integration and testing regime. Your tests should provide an appropriate breadth of coverage and depth in test scenarios to ensure the necessary level of quality in the delivered software.

Suitable test coverage tools may include CoverMe for Ruby 1.9 or RCov for versions of Ruby other than 1.9, any of the tools described in the Wikipedia Java Code Coverage Tools article, COVTOOL for C++, or coverage.py or by using test.regrtest for Python.

An area where you might have some flexibility in the breadth and depth of testing is where the soft- ware uses features covered by API/ABI guarantees, but care should be taken to ensure the testing strategy doesn’t become over-reliant on these guarantees. Comprehensive baseline testing should be performed and automation should trivialize execution of the baseline tests across multiple OS versions.

And this is the key point: CI environment testing must be automated so that tests can be executed unattended with each build of the application software or OS update.

There are a number of automation tools that can be used with Red Hat Enterprise Linux, including Selenium, Beaker, and Autotest, among others.

MANAGING YOUR TESTING ENVIRONMENT

It’s possible that the environment needed to support comprehensive testing will be relatively complex: multiple versions of Red Hat Enterprise Linux, coupled with the need to test when changes occur in each OS version, in addition to changes in your software.
The process of managing and implementing the necessary test environments can be simplified by the use of provisioning tools such as Red Hat Satellite or Cobbler.

Red Hat Satellite is a system designed for the distribution of generic content, although it is used principally to distribute OS updates from the Red Hat network, updates to applications created within your organization, or for software delivered by third parties.

With Red Hat Satellite, you create channels for content you wish to deliver, where each channel offers the ability to apply update streams. Then the software can be used to instantiate the content on hardware. In addition, Red Hat Satellite can merge channels—such as your applications and an OS version as they are instantiated on hardware—enabling a single OS configuration to be used independently by multiple applications. And the process can be automated through an API or scripting.

For more information on setting up your test environments using Red Hat Satellite, refer to “Part 1— Setting up your Environment—Staging Content” of the Best Practices for Deploying and Managing   Linux with Red Hat Network white paper. You may also find the information in RHN Satellite:  Best Practices For Multiple Organizations on enabling each development group to manage its own content set using multi-organization support useful too.

CONCLUSION

As an ISV or independent application developer, the flexibility of Red Hat Enterprise Linux and Red Hat’s service provisioning over an extended time period for multiple major and minor OS releases could encourage you to lock down your software to a specific build of the OS. Doing so would either limit your market or limit the options of your customers to take advantage of the latest OS improvements and innovations.

By adopting a robust CI process—one underpinned by a solid repository for application assets, and an automated build and test process against a well defined and managed set of Red Hat Enterprise Linux environments— it will become possible to give your clients the assurance that they can use your software on almost any OS version. At the same time, a CI environment can potentially reduce your development/integration costs and eliminate the lead time normally needed to effect integration.

Following the guidelines in this paper and making use of the tools and techniques suggested should ease the path to a robust and effective CI environment. Your Red Hat sales and support team stands ready to assist you with the setup and commissioning of this environment.

In addition, you can also look towards completing a more comprehensive self certification of your software, thereby offering current and future customers a high level of comfort that your software will work well on whatever Red Hat Enterprise Linux environment they have.

For more information on assistance with CI setup, Red Hat Satellite, or self certification, please contact developer@redhat.com.