Static analysis with KubeAudit for Red Hat OpenShift

In this article, we introduce a new utility for developers who want to ensure that their code transitions cleanly from upstream Kubernetes to Red Hat OpenShift. OpenShiftKubeAudit (KubeAudit) is a static analyzer that semantically checks a user's code for known incompatibilities so you can fix them before bringing the code into OpenShift. KubeAudit is also simple to use and easy to extend.

Running an audit

This being the first release, KubeAudit currently offers only a handful of audits, but they're easy to write. We're looking for feedback and additional use cases from the community to help make the tool more comprehensive.

To start, find KubeAudit on GitHub.

You'll want to clone the git repo somewhere on the same machine where the manifests to audit reside, and set up a Python virtual environment:

$ git clone https://github.com/AICoE/OpenShiftKubeAudit.git
$ cd OpenShiftKubeAudit
$ python3 -m virtualenv venv
$ source venv/bin/activate
$ python3 -m pip install -r requirements.txt

Now, let's run the auditing tool:

$ ./audit.py /path/to/yaml/directory

It should produce similar output to this:

$ ./audit.py ./testaudits   
Audit Results:

Issue: Pod Security Policies in manifests
Severity: 1 - High
Resolution: In OpenShift PodSecurityPolicies are replaced by SecurityContextConstraints. See https://docs.openshift.com/container-platform/4.5/authentication/managing-security-context-constraints.html
Affected Files:
  Matched regex:
  testaudits/yaml/PodSecurityPolicies.yaml

Issue: NetworkPolicy has an IPBlock
Severity: 2 - Medium
Resolution: The Kubernetes v1 NetworkPolicy features are available in OpenShift Container Platform except for egress policy types and IPBlock.
Affected Files:
  Matched regex:
  testaudits/yaml/NetworkPolicies.yaml

Issue: Egress network policy set
Severity: 2 - Medium
Resolution: The Kubernetes v1 NetworkPolicy features are available in OpenShift Container Platform except for egress policy types and IPBlock.
Affected Files:
  Matched regex:
  testaudits/yaml/NetworkPolicies.yaml

Now with these issues identified, a developer could fix them before they become issues in production. However, since the tool is new, there are many use cases that have not yet been identified, which is why the tool is easy to extend.

Adding a custom audit

KubeAudit uses a simple initialization (INI) syntax to define an audit with a few fields:

• Name: A name or title to identify the audit.
• Query (optional): This is a jq-syntax query, which the auditing tool uses for semantic analysis of parts of a manifest. Note that the query is optional if you are using regex. This type of query is useful when you need a more comprehensive analysis.
• Regex (optional): You can specify a regex to search for in each file. Although regex is a simpler check than query, the utility can ignore regexes found in the code comments.
• Regexes section (optional): Use this section when you need multiple regexes. Once again, the utility ignores regex matches found in comments, and it matches the results in the order that they appear. If all of the regexes are found in the corresponding order, it is considered a match.
• Severity: This message indicates the relative severity of the issue. It needs to start with a number, and can then be followed by a text description of the severity.
• Message: The diagnostic message to output to the user.

Below is an example audit:

[Default]
# The name/title of the issue, e.g., "Pod Security Policies in manifests"
name = RunAsUser is set outside range, has some known issues
# A jq-syntax query to be applied to the yaml files. The query must return
# either True or False, but can be otherwise any valid jq string
query = ((.. | .runAsUser? | numbers) != 0) and ((.. | .runAsUser? | numbers) < 10000000 or (.. | .runAsUser? | numbers) > 20000000)
# Python-compatible regex to search for. The script automatically ignores
# commented lines
regex = runAsUser:
# Severity to help the user prioritize fixes
severity = 4 - Warning
# Message to output to the user, usually a resolution or more information
message = Setting runAsUser ID explicitly is not recommended. Additionally setting runAsUser explicitly outside of the expected range in OpenShift (10000000 - 20000000) has known incompatibilities

# This section can be used to construct a multiline regex for the files.
# Each regex is searched for in the order that it appears and automatically
# ignores comments. Any number of regexes can be added this way
[regexes]
regex1 =
regex2 =

Conclusion

This auditing tool currently only supports Kubernetes manifests, but we plan to expand it to include Helm charts and Go code, as well. The tool is in very early stages, but is looking for community input to help add use cases.