Building images for cloud deployments or on-premises servers provides a number of challenges. These include reducing the attack surface, ensuring the images are hardened, and monitoring their compliance on an ongoing basis. We will explore how Red Hat Insights can achieve what no other service can by leveraging Red Hat’s open hybrid cloud strategy. You can use Red Hat Insights image builder to deploy pre-hardened images then monitor the systems with our compliance tool, compared to using custom-built testing frameworks leveraging tools such as AWS Security Hub.
Building images
Insights image builder makes the process of hardening images much simpler because it leverages OpenSCAP remediations at build time before the image has even been launched. This is desirable since some options, such as filesystem customizations, are hard to remediate once an image has been built and booted.
There are two ways we can build images using image builder on console.redhat.com. The first way is making an API request to the backend image builder application and then deploying the resulting image to Amazon Web Services (AWS). Alternatively, we can use the front end to build pre-hardened images. This guide will take you through both methods.
Building images via the UI
Since April 2024, it is possible to build pre-hardened images using OpenSCAP in the https://console.redhat.com environment. To get started building an image, navigate to the image wizard to create an image. We will skip ahead a few steps through the wizard with the following assumptions:
- A RHEL 9 image was selected.
- An AWS AMI was selected as the target environment.
- The system was registered with an Access Key.
From the OpenSCAP compliance step, choose an OpenSCAP profile from the dropdown menu, and select the PCI-DSS profile. Once selected, a summary of the profile is displayed, as shown in Figure 1. The summary also shows the additional kernel arguments that the profile requires, the services to be enabled, and the services to be disabled.
The PCI-DSS profile doesn’t have any partitioning requirements. For profiles that do, the filesystem step in the wizard will be pre-populated with recommended partition sizes based on the requirements of the selected profile. These sizes can be amended and additional mount points can be added. See Figure 2.
Similarly, the required packages for the selected profile are pre-populated; these packages may be removed and additional packages can be added as desired.
Note that amending the suggested file system layout or removing suggested packages may impact your image’s compliance score or break the image build if the package is one of the packages in the list of services to be enabled. Figure 3 depicts this.
Click through the remaining steps, select the options as desired, then start the image build. Once the build is complete, we can launch the image and move on to the next step. See Figure 4.
Building images via the API
To start off with, we will build an image for AWS using the Insights image builder’s API, as shown in Figure 5. Head to our API developer docs.
The example request body will need to be updated, since the example shows all of the available customizations. Here is an example of a request with the PCI-DDS profile:
distributionimage_namearchitectureimage_typetype,share_with_accountsprofile_id
{
"distribution": "rhel-93",
"image_name": "pci-dss-compliant-image",
"image_requests": [
{
"architecture": "x86_64",
"image_type": "ami",
"upload_request": {
"type": "aws",
"options": {
"share_with_accounts": [ "AWS_Account_ID" ]
}
}
}
],
"customizations": {
"packages": [ "zsh" ],
"openscap": {
"profile_id": "xccdf_org.ssgproject.content_profile_pci-dss"
}
}
}The image build can be monitored through the API by using the compose ID generated by the create compose request. Figure 6 shows an example of this in the API catalog.
Once the image build has completed successfully, the image can be launched or, if desired, shared to a new region and then launched. Please note that sharing an image to a new region will start a new image build which will need to be monitored. See the API catalog for an example. See Figure 7.
Compliance monitoring
The choice to use the PCI-DSS profile was not an arbitrary one. Due to limitations with Amazon Security Hub, we are not able to run scans for the CIS profile. This is because Amazon only recognized images built with AWS when attempting to scan and report compliance for CIS.
Insights compliance
To get started with Insights compliance, we need to make sure we have the correct permissions to access the feature. We need either the Red Hat Insights compliance viewer role or the administrator role. We can check our access by clicking the user icon in the top right corner of the screen and then selecting the My User Access option.
You should see the "Compliance administrator" and "Compliance viewer" roles; if not, you should contact your organization administrator to arrange access. See Figure 8.
Register the instance
For a more seamless experience, we recommend pre-registering the image with Red Hat Insights during the image creation. We can pre-register the image in the Create Image Wizard using Activation Keys in Insights image builder. See Getting started with activation keys on the Hybrid Cloud Console for further guidance.
Additionally, you will need to ensure that you have Simple Content Access (SCA) enabled on your account. This provides the simplest experience to ensure subscriptions are automatically usable on the system. The default is "enabled" for new accounts. For older accounts that may not have this option enabled, you may need to contact your company's org admin for the Red Hat account to enable SCA.
Create the policy and attach it
We now need to create and assign a policy and attach it to the instance from Insights compliance in https://console.redhat.com. To do this, navigate to https://console.redhat.com/insights/compliance/scappolicies.
You can select an existing policy, if available, or create a new policy. Depending on your organization, some profiles may have already been created by the org admin or another user. For the purpose of this demo, we will create a new one since there are none available to us.
For the policy, since we built a RHEL 9 image, we will select RHEL 9 for the operating system. Next, we need to provide a description of the policy, the business objective, and compliance threshold. For the business object, we are trying to ensure that a customer’s base image meets the PCI-DSS compliance standards—so we’ll use that as the objective.
The compliance threshold describes the margin of error with which we are satisfied to consider the system as compliant with the standard. We will choose 95% for this demonstration since there are still some remediation steps that need to be run manually in order to get a 100% compliant image built with Insights image builder. Figure 9 depicts this menu option.
We then need to register our system with the policy we are creating in Insights compliance; otherwise, running the compliance report generation will fail. We can select the system from the dropdown. The name of the system should match the IP address of the EC2 instance we have created and launched, as shown in Figure 10.
We will leave the default rules for the policy since tailoring profiles in Insights image builder is not yet available at the time of writing. Finally, we can review the policy and create it. We should now see the created policy in the list of SCAP policies.
Run the scan
We are now ready to run a scan and generate a report on the instance. To do so, we can run the following command:
sudo insights-client –-complianceView the report
We can now navigate to the reports page under insights compliance, as seen in Figure 11:
We can see in the above screenshot that the image is 100% compliant since it met the compliance threshold of 95%. We can click on the report, get additional information and see which rules are non-compliant in order to perform manual intervention.
Custom testing framework using AWS Security Hub
We will be building a custom testing framework using AWS Security Hub. Security Hub enables users to monitor the security and compliance of instances in AWS. Before we start, please note that Security Hub and AWS configuration must be enabled prior to running this script, or else the dashboard won’t be populated. To test an AWS AMI created by Red Hat Insights image builder, we’re going to build out the following testing framework (Figure 12 and Figure 13):
The framework operates by raising a RHEL AMI on an EC2 instance, then AWS Systems Manager (SSM) is deployed to run a set of commands that will:
- Install the OpenSCAP scanning utility
- Perform the vulnerability scan to see if it’s PCI-DSS compliant
- It will then publish the results into both an XML and html file and push them to an S3 bucket
Once S3 detects that an object has been pushed to the designated bucket, it will then trigger a lambda function to scan the bucket for the XML file. Once it has found the XML, the lambda function will pull it from S3, process the file, and push the results to Security Hub. At that point, the results of the scan should be visible in the Security Hub dashboard. The cloudwatch component of the framework is there purely for monitoring and troubleshooting purposes.
Now, access the AWS web console and locate your AMI in the EC2 section. Click the Images tab -> AMI -> Private Images. Locate the AMI ID and sub that value into the Terraform code found at https://github.com/oezeakachi/open_scap_aws to raise the framework. Once the build is complete, go to the Security Hub in the AWS web console and check the PCI dashboard to see how compliant the AMI is, as shown in Figures 14, 15, and 16.
Further information can be found by clicking on the Findings tab and searching for the EC2 instance that’s using the AMI via its ID.
Summary
The custom OpenSCAP testing framework generates a thorough vulnerability report that can be viewed as an HTML file in an S3 bucket. The framework can also push the results to a dashboard in Security Hub (Security Hub is pre-configured to display results for a set list of security standards specifically for AWS-based images). However, a custom framework has the additional overhead of implementation and maintenance.
On the other hand, Red Hat Insights succeeds in all the areas where the custom frameworks fail. The custom framework was also limited by the fact that AWS only supports a subset of the profiles that are available in Red Hat Insights. It offers a clear and concise scanning process and a simple way of viewing the subsequent results. In addition to this, the tool is not limited to scanning images from a single cloud service and can scan any subscribed image in all formats (AWS, Azure, on premise, etc). We feel it makes a great tool for scanning and ensuring compliance with the images we build.
