Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

End-to-end testing with self-hosted runners in GitHub Actions

July 25, 2023
Jianzhu Zhang Andrew Kiselev Daniel Kostecki
Related topics:
CI/CDContainersKubernetes
Related products:
Red Hat OpenShift

Share:

Accelerating the software development life cycle while ensuring the quality and performance of applications is a challenging task. GitHub Actions makes it easy to automate all required CI software workflows for your GitHub repository.

GitHub Actions Runner is an application that runs a job from a GitHub Actions workflow. GitHub Actions also provides a self-hosted runner that allows you to run continuous integration (CI) tests that require actual hardware. End-to-end testing (E2E testing) is a popular methodology to test an application's functionality and performance under real-life conditions. Still, it often demands actual hardware, rendering it infeasible to run on the public cloud.

In this article, we'll delve into our experience performing E2E testing for an open source project on-premises using a containerized self-hosted runner. The self-hosted runner container image used in this tutorial is available for download from the quay.io registry.

Self-hosted runner container on Red Hat Enterprise Linux

First, we'll create a self-hosted runner container on Red Hat Enterprise Linux (RHEL).

Build and download the containerized runner image

The whole procedure is covered in https://github.com/redhat-eets/gitaction.

To build the containerized runner for a given runner version, enter the following:

podman build --build-arg RUNNER_VERSION=2.301.1 --tag quay.io/gitaction/runner:2.301.1

To see what runner releases are available to use for RUNNER_VERSION, check on https://github.com/actions/runner/releases.

Alternatively, you can download a specific self-hosted runner container image for the 2.301.1 release:

podman pull quay.io/gitaction/runner:2.301.1

The GitHub runner will check if a newer version is available on startup. It will self-update and restart with the latest version. However, it is worth using the latest version for the container image. Note that the runner's self-update takes time and may not always be successful.

GitHub token protection

In order to generate a registration token, the container requires you to enter a GitHub personal access token (PAT) when starting. The PAT has to belong to the target repository owner for the container to register successfully.

From a security perspective, using the PAT directly with the Podman command is not a good idea. Instead, a Podman secret should be created for the PAT:

echo "your github access token" > token && podman secret create github_token token && rm -rf token

In the above step, github_token must be used as the secret name, as this is the default secret filename that the script inside the container will look for. If you want to choose a different secret name, you can use the environment variable GH_TOKEN_PATH to specify the secret file path when running Podman to start the container.

With all the information we have so far, run the self-hosted runner with Podman:

podman run --secret github_token --name runner -it --rm --privileged -e GH_OWNER='<your github id>' -e GH_REPOSITORY='<repo name>' quay.io/gitaction/runner:2.301.1

If using a different Podman secret name, say some_github_token, use the extra environment variable GH_TOKEN_PATH:

podman run --secret some_github_token --name runner -it --rm --privileged -e GH_OWNER='<your github id>' -e GH_REPOSITORY='<repo name>' -e GH_TOKEN_PATH=/run/secrets/some_github_token quay.io/gitaction/runner:2.301.1

Pass in extra information

In reality, E2E CI workflows often require extra information outside of the target GitHub repository.

For illustrative purposes, we use the RHEL SR-IOV test suite as an example throughout this article. Its E2E CI workflow requires testbed information. The testbed information is not checked into the GitHub repository. For the runner container to access this information, a volume mount can be used. You can apply the same technique in Red Hat OpenShift.

To set up the volume mount for this purpose, first create a folder on the host and copy the required files into this folder. For the RHEL SR-IOV E2E CI, the required files are testbed.yaml and config.yaml, so copy these files into the folder and start the container with the volume mount:

sudo mkdir -p /opt/E2E-config
sudo cp testbed.yaml  /opt/E2E-config
sudo cp config.yaml  /opt/E2E-config
sudo chown -R nobody:nobody  /opt/E2E-config
podman run --secret github_token --name runner -it --rm --privileged -e GH_OWNER='redhat-partner-solutions' -e GH_REPOSITORY='rhel-sriov-test' -v /opt/E2E-config:/config quay.io/gitaction/runner:2.301.1

In the above sample step, the volume is mounted to /config inside the container. That means the E2E CI workflow needs to go to this folder to retrieve these YAML files. Interested readers can take a look at the following E2E CI workflow for reference.

Label the runner

A GitHub repo can have multiple containerized runners on the same server using different labels. This is useful if various tests have different hardware requirements; for example, 800-series and 700-series Intel NICs:

podman run --secret github_token --name runner810 -it --rm --privileged -e RUNNER_LABEL='810' -e GH_OWNER='redhat-partner-solutions' -e GH_REPOSITORY='rhel-sriov-test' -v /opt/E2E-config-810:/config quay.io/gitaction/runner:2.301.1
podman run --secret github_token --name runner710 -it --rm --privileged -e RUNNER_LABEL='710' -e GH_OWNER='redhat-partner-solutions' -e GH_REPOSITORY='rhel-sriov-test' -v /opt/E2E-config-710:/config quay.io/gitaction/runner:2.301.1

You can refer to the runners above as runs-on: [self-hosted, 810] or runs-on: [self-hosted, 710]  in a GitHub workflow. 

How to use the runner label will be explained later in the How to trigger the CI section.

Self-hosted runner as a systemd service

Directly using the Podman command line to start the runner container primarily serves the purpose of proof of concept. For production use, you can use a systemd service to manage the self-hosted runner. Here is the systemd unit file that was used by the RHEL SR-IOV E2E CI:

[Unit]
Description=github self runner in container
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/podman run --secret github_token --name runner --rm --privileged -e GH_OWNER='redhat-partner-solutions' -e GH_REPOSITORY='rhel-sriov-test' -v /opt/E2E-config:/config quay.io/gitaction/runner:2.301.1
ExecStop=/usr/bin/podman stop runner

[Install]
WantedBy=multi-user.target

After the container starts and successfully registers with the target GitHub repository, the self-hosted runner can be found under the target repository's Actions/Runners, as shown in Figure 1:

Runners
Figure 1: The self-hosted runner listed in the repository.

How to trigger the CI

Here is the sample code for using the runner label:

name: sriov-e2e-test
run-name: sriov-e2e-test initiated by ${{ github.actor }}
on:
  pull_request:
    types: [ labeled ]
  workflow_dispatch:
    inputs:
      tag:
        description: 'NIC hardware'
        required: true
        default: '810'
        type: choice
        options:
        - 810
        - 710
jobs:
  prepare-label:
    runs-on: ubuntu-latest
    outputs:
      label: ${{ steps.step1.outputs.label }}
    steps:
      - name: Check label
        id: step1
        run: |
          if [ ${{ github.event.label.name }} == 'e2e-test' ]; then
            echo "label=810" >> $GITHUB_OUTPUT
          elif [ ${{ github.event.label.name }} == 'e2e-test-710' ]; then
            echo "label=710" >> $GITHUB_OUTPUT
          elif [ -n ${{ github.event.inputs.tag }} ]; then
           echo "label=${{ github.event.inputs.tag }}" >> $GITHUB_OUTPUT
          fi

Using a label to trigger an E2E CI action

As illustrated in the above sample code, one option to trigger the E2E test is to use the appropriate label, e2e-test or e2e-test-710 (see Figure 2). This labeling mechanism serves as a way to limit who can trigger the E2E runs due to hardware resource constraints. Only repo users with write permission can set a pull request label and trigger the E2E test execution.  

Labels
Figure 2: Triggering a E2E test run with the e2e-test label.

The labels double as a flag showing which PRs have been tested. 

On-demand triggering

In addition to the labeling above, we can also trigger this E2E action on demand. NIC hardware labels (810 or 710) are collected from the user input, in this case, and used to trigger the appropriate runner.

On demand

Self-hosted runner as an OpenShift Workload

If the test environment already has an OpenShift/Kubernetes cluster installed, and the user does not plan to add an extra RHEL server to host the runner systemd service, the runner container can be hosted on the OpenShift/Kubernetes cluster instead. In this situation, the self-hosted runner will be a workload in the pod format.

To use the runner container as an OpenShift workload for controlling an on-premise E2E CI testbed, the OpenShift cluster needs to be on-premise and have connectivity to the E2E CI testbed.

We will need to take steps to protect the user's PAT and pass in extra test configuration, similar to the runner container.

GitHub token protection

In OpenShift, create a secret for the PAT:

kubectl create secret generic gh-token --from-literal=github_token=<your github token>

In the above command, the name github_token is used for the same reason explained earlier in the podman usage.

The secret gh-token will be mounted as a volume later in the runner pod YAML spec.

Pass in extra information

Once again using the RHEL SR-IOV test suite repository for demo purposes, its E2E CI workflow requires testbed.yaml and config.yaml files, which can be passed to the runner pod via a volume map.

First, create a folder and store the required files under this folder:

ls /opt/E2E-config
config.yaml
testbed.yaml

Create a ConfigMap from this folder:

oc create configmap test-config --from-file=/opt/E2E-config

This ConfigMap, test-config, will be used in the volume map of the runner pod YAML spec.

Self-hosted runner in deployment

We will let OpenShift take care of the runner pod lifecycle management using a deployment. Here is the self-hosted runner deployment YAML spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: runner-deployment
  labels:
    app: runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: runner
  template:
    metadata:
      labels:
        app: runner
    spec:
      volumes:
      - name: secret-volume
        secret:
          secretName: gh-token
      - name: config-volume
        configMap:
          name: test-config
      containers:
      - name: runner
        image: quay.io/gitaction/runner:2.301.1
        securityContext:
          privileged: true
        env:
        - name: GH_OWNER
          value: "redhat-partner-solutions"
        - name: GH_REPOSITORY
          value: "rhel-sriov-test"
        - name: GH_TOKEN_PATH
          value: "/etc/gh_secrets/github_token"
        volumeMounts:
        - name: secret-volume
          readOnly: true
          mountPath: "/etc/gh_secrets"
        - name: config-volume
          mountPath: "/config"

Notice in the above YAML spec the OpenShift secret gh-token is mounted to the path /etc/gh_secrets, so the environment variable GH_TOKEN_PATH is used to tell the container to retrieve the secret from this path.

The extra information for the E2E CI testbed is mounted under /config. As explained earlier, the E2E workflow will look for the extra information in that folder inside the container.

Summary

We reviewed E2E CI building blocks which allow real hardware test execution for a GitHub open source project. Lightweight GitHub Actions CI, along with the containerized GitHub Actions runner, allow minimizing system footprint while maintaining CI functionality. Furthermore, this CI implementation fits well into corporate IT security policy for lab access: nothing extra gets exposed to the internet.

Feel free to comment below if you have questions. We welcome your feedback!

Last updated: September 19, 2023

Related Posts

  • Deploy self-hosted GitHub Actions runners for Red Hat OpenShift

  • Test GitHub projects with GitHub Actions and Testing Farm

  • Schedule tests the GitOps way with Testing Farm as GitHub Action

  • Automate dependency analytics with GitHub Actions

  • Leveraging Kubernetes and OpenShift for automated performance tests (part 1)

Recent Posts

  • Storage considerations for OpenShift Virtualization

  • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

  • EE Builder with Ansible Automation Platform on OpenShift

  • How to debug confidential containers securely

  • Announcing self-service access to Red Hat Enterprise Linux for Business Developers

What’s up next?

Podman in action e-book share image

Read Podman in Action for easy-to-follow examples to help you learn Podman quickly, including steps to deploy a complete containerized web service.

Get the e-book
Red Hat Developers logo LinkedIn YouTube Twitter Facebook

Products

  • Red Hat Enterprise Linux
  • Red Hat OpenShift
  • Red Hat Ansible Automation Platform

Build

  • Developer Sandbox
  • Developer Tools
  • Interactive Tutorials
  • API Catalog

Quicklinks

  • Learning Resources
  • E-books
  • Cheat Sheets
  • Blog
  • Events
  • Newsletter

Communicate

  • About us
  • Contact sales
  • Find a partner
  • Report a website issue
  • Site Status Dashboard
  • Report a security problem

RED HAT DEVELOPER

Build here. Go anywhere.

We serve the builders. The problem solvers who create careers with code.

Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

Sign me up

Red Hat legal and privacy links

  • About Red Hat
  • Jobs
  • Events
  • Locations
  • Contact Red Hat
  • Red Hat Blog
  • Inclusion at Red Hat
  • Cool Stuff Store
  • Red Hat Summit
© 2025 Red Hat

Red Hat legal and privacy links

  • Privacy statement
  • Terms of use
  • All policies and guidelines
  • Digital accessibility

Report a website issue