Article

Refine your Linux package maintenance process with 'rebase-helper'

 

April 7, 2016
CI/CD Linux
Petr Hracek

Rebase-helper is a tool that helps Linux package maintainers update their to the latest upstream version. New maintainers and experts alike will find value in the convenience this tool can offer. Rebase-helper applies patches, packages, and reports differences between the patched and un-patched versions.In summary, it automates some of the manual tasks usually performed by package maintainers when a new upstream version of that package is released. This article will explain how to get started using rebase-helper, and also demonstrate how it can be integrated with upstream monitoring services.

How does rebase-helper work?

The workflow can be summarized in just a few steps:

  • Rebase-helper downloads an archive with new sources.
  • It rebases downstream patches on top of the latest upstream version using the `git rebase` command.
  • If patching was successful, rebase-helper builds two sets of RPMs. One using the “old” sources and a second one using the “new” sources.Available build tools are `mock`, `rpmbuild`, `fedpkg`.
    • If the “new” RPMs build fails, it downloads logs like build.log and root.log
    • If the build is successful, it downloads all available RPM packages.
  • If both builds succeed, rebase-helper runs several analyzers, including: `pkgdiff`, `rpmdiff`, `abipkgdiff`, then compares the old and new packages and reports results.

How to install rebase-helper

Begin installation on Fedora using the dnf command:

dnf install -y rebase-helper

Rebase-helper requires several other programs, including abipkgdiff, rpmdiff, mock, fedpkg, meld, etc. These will be installed automatically as dependencies, and I will talk more about them individually about later on.

Once rebase-helper is installed, it can be run from the git repository root for the desired package by typing the following command (you might also consider adding it to your PATH):

repo_root user$ /path_to_rebase-helper/rebase-helper.py <new_upstream_version> (e.g. /home/user/phracek/rebase-helper/rebase-helper.py "3.1.10")

RHEL & CentOS: rebase-helper is also available as EPEL-7 package, so it may also be used on CentOS and Red Hat Enterprise Linux systems (RHEL is now free for developers).

What has changed recently?

Since last year, we have implemented several new features needed primarily for automated and upstream release monitoring services:

  • Non-interactive mode - This mode means that the package maintainer is never prompted for input. Patches are either rebased automatically, or in case of a failure, the patch is skipped and reported at the end of the process. Builds are also performed automatically, and in case of build failure, rebase-helper tries to analyze the `build.log` and correct the SPEC file. After this is done, the build is retried once more.
  • Using git for rebasing downstream patches - In earlier versions of rebase-helper, we used `patch` and `meld` tools for rebasing patches onto new upstream releases; however, we now use the `git rebase` workflow for applying patches. This allows the user to specify custom behavior in their `.gitconfig` file. For example, they can specify the diff tool for comparing or merging patches (in case of patches that don’t apply cleanly.)
  • fedpkg build support - Rebase-helper is able to build package on koji as scratch builds.
  • abipkgdiff support - Compare old and new sets of packages with abipkgdiff, and report to the user any changes detected by the tool.
  • Upstream Release Monitoring support - The Anitya release-monitoring project checks new upstream releases and opens new bugzilla tickets (or updates existing), and reports the changes since the last version (as detected by rebase-helper.)

What are the other rebase-helper --options?

  • buildtool - select a build tool to be used when building packages. (Available choices are `mock` [the default], `fedpkg` and `rpmbuild`.)
  • pkgcomparetool - select a tool for comparing the old and new sets of RPMs. (Available choices include `pkgdiff`, `rpmdiff` or `abipkgdiff`. By default all are executed.)
  • outputtool - select a tool used for presenting the output. (Only `text` is currently supported.)
  • not-download-sources - do not download the package sources before building the package.
  • builds-nowait - do not wait for builds. (Valid only for `--buildtool fedpkg`. Otherwise it is ignored.)
  • fedpkg-build-tasks - specify `fedpkg` build task ids. (The logs and RPMs will be downloaded from the task ids. This is the second step which is called as part of the upstream release monitoring workflow.)
  • results-dir - specify where the results/output will be stored.

Integration with an upstream monitoring service

This chapter demonstrates how to include rebase-helper in your upstream monitoring service. (This has already ben included in Fedora's monitoring services.)

Before rebase-helper was integrated into the Fedora release monitoring service integration, bugs reported by the Fedora upstream release monitoring system used to look like this:

upstream_without_rebasehelper

In the case of a failed scratch build, the package maintainer would not see if patches were applied properly or if anything else failed; however, after the integration of rebase-helper, the Bugzilla issue note now includes the result of the package debug output (as an attachment), whether patching failed, the actual scratch build failed, etc:

rebase_helper_upstream

Rebase-helper the tool for upstream monitoring services

If you would like to integrate rebase-helper into your upstream monitoring services, this chapter is for you. Rebase-helper provides an API which you can use either directly from Python, or directly from the command line.

Patch new upstream version and start scratch builds

Example of patching new sources and starting scratch builds with fedpkg. This returns task_ids. The bash equivalents are included for comparison:

  • Python API
from rebasehelper.application import Application
cli = CLI([‘--non-interactive, ‘--builds-nowait’, ‘-buildtool’, ‘fedpkg’, upstream_version])
rh = Application(cli)
rh.set_upstream_monitoring() # Switch rebase-helper to upstream release monitoring mode.
rh.run()
  • Bash
rebase-helper --non-interactive --builds-nowait --buildtool fedpkg upstream_version

Download logs and RPMs and compare with tools like abipkgdiff

  • Python API
cli = CLI([‘--non-interactive, ‘--builds-nowait’, ‘--fedpkg-build-tasks old_id,new-id])
rh.run() # Downloads RPMs, logs and runs checkers and provides logs.
rh.get_rebasehelper_data() # Get all information about the results
  • Bash
rebase-helper --non-interactive --builds-nowait --fedpkg-build-tasks old_id,new-id

Project Roadmap

  • Implement csbuild.
  • Download old release directly from `koji`, instead of building locally.
  • Improve building Source1 GitHub issue 161.
  • Attach ChangeLog, News etc. to Bugzilla issues.
  • Comment in the Bugzilla issue if `soname bump` is changed.
  • More output tools like XML, JSON.

Links & Additional References:

Last updated: May 31, 2023