Introducing atomic scan – Container vulnerability detection

In the world of containers, there is a desperate need to be able to scan container images for known vulnerabilities and configuration problems, and as we proliferate containers and bundled applications into the enterprise, many groups and companies have started to build container scanning tools. Even Red Hat has been building a scanning tool based on the tried and true OpenSCAP project, but there were several problems we saw time and again.

The problems with existing container scanning tools included:

  • Either these tools were required to be installed on the host, or if they were installed via a container they would have to be very privileged; therefore, the containers themselves could use that authority on the host machine.
  • They were being hard coded to only support a single container framework – docker.
  • We saw some very slow implementations, where the container scanning tools were actually copying all of the content out of the container image, then scanning the content, and finally removing the content.  You can image how inefficient this could be for hundreds or thousands of images or containers.

We decided we wanted to build a tool, atomic scan, that understood the underlying architecture and could download and run containers with scanning tools in them. But, we also didn’t want the scanning tools to be privileged; the atomic tool would be able to mount up read only rootfs from the host’s file system. These mounted file systems could then be passed onto the scanning container, along with a writeable directory for the scanner to place its output.

Familiarizing yourself with atomic scan

The atomic application is pre-installed on all atomic images.  If you want to test on a non-atomic system, like Fedora or Centos, you can simply install the atomic application with yum.

# yum install atomic

If you install atomic, it will include a full set of man pages.  Usually the man pages are grouped by the atomic sub-commands.  These sub-commands can be easily discovered with the –help command line switch.  You can also browse the atomic man pages in its git repository on-line.

The quickest way to familiarize yourself with the atomic scan sub-command is to run it with the –help option.  Note that atomic requires sudo privileges to run properly.  Depending on the atomic version, will output something like the following:

# sudo atomic scan --help
usage: atomic scan [-h] [--scanner {openscap,example_plugin}]
 [--scan_type SCAN_TYPE] [--list]
 [--all | --images | --containers]
 [scan_targets [scan_targets ...]]

positional arguments:
 scan_targets container image

optional arguments:
 -h, --help show this help message and exit
 --scanner {openscap,example_plugin}
 define the intended scanner
 --scan_type SCAN_TYPE
 define the intended scanner
 --list List available scanners
 --all scan all images (excluding intermediate layers) and
 containers
 --images scan all images (excluding intermediate layers)
 --containers scan all containers

atomic scan <input> scans a container or image for CVEs

There are a few important command line switches that are important to point out:

  • By default, atomic ships with the OpenSCAP project set as its default scanner.  This is set in the /etc/atomic.conf configuration file.
  • When using the –help command line switch, atomic will show which scanners are set up properly.  In the above example, it is aware of two scanners: openscap and example_plugin.
  • The –scanner options allows you to choose which scanner you want to use and the –scan_type allows you to select the different scan types each scanner has to offer.
  • You can use the –list option to get more detail on the scanners and scan types available.
  • There are several options to tell the scanners what content you want scanned (–all, –images, –containers, or a list of containers and images).

Running atomic scan with the default scanner

As mentioned earlier, the default scanner shipped with atomic is based on the OpenSCAP project,  and the default scan_type for the openscap scanner is to scan for vulnerabilities.  Suppose I want to check for vulnerabilities in the an image, for example the rhel-tools images, I simply issue the atomic scan command with the rhel-tools image name:

(Editor’s note: Several lines have been cut from this output in order to shorten the example. Those cuts have been marked with “(… truncated)”.)

# sudo atomic scan registry.access.redhat.com/rhel7/rhel-tools
 docker run -it --rm -v /etc/localtime:/etc/localtime -v /run/atomic/2016-04-17-09-14-28-278584:/scanin -v /var/lib/atomic/openscap/2016-04-17-09-14-28-278584:/scanout atomic-registry.usersys.redhat.com:5000/openscap/openscap oscapd-evaluate scan --no-standard-compliance --targets chroots-in-dir:///scanin --output /scanout
 Unable to find image 'atomic-registry.usersys.redhat.com:5000/openscap/openscap:latest' locally
 Trying to pull repository atomic-registry.usersys.redhat.com:5000/openscap/openscap ... latest: Pulling from openscap/openscap
 45764f98d75b: Pull complete
 85036e292fea: Pull complete 
 cad225283687: Pull complete (... truncated)
 Digest: sha256:f36009bb430a61d1a6ef7f32c221fdc691081e8bb0a82c10b27c6c9ebbefb023
 Status: Downloaded newer image for atomic-registry.usersys.redhat.com:5000/openscap/openscap:latest
registry.access.redhat.com/rhel7/rhel-tools (c28fabd46c7457b)
The following issues were found:
RHSA-2016:0534: mariadb security and bug fix update (Moderate)
 Severity: Moderate
 RHSA URL: https://rhn.redhat.com/errata/RHSA-2016-0534.html
 RHSA ID: RHSA-2016:0534-00
 Associated CVEs:
 CVE ID: CVE-2015-4792
 CVE URL: https://access.redhat.com/security/cve/CVE-2015-4792
 CVE ID: CVE-2015-4802
 CVE URL: https://access.redhat.com/security/cve/CVE-2015-4802
 ... (truncated)
RHSA-2016:0459: bind security update (Important)
 Severity: Important
 RHSA URL: https://rhn.redhat.com/errata/RHSA-2016-0459.html
 RHSA ID: RHSA-2016:0459-00
 Associated CVEs:
 CVE ID: CVE-2016-1285
 CVE URL: https://access.redhat.com/security/cve/CVE-2016-1285
 CVE ID: CVE-2016-1286
 CVE URL: https://access.redhat.com/security/cve/CVE-2016-1286

 ... (truncated)
RHSA-2016:0370: nss-util security update (Critical)
 Severity: Critical
 RHSA URL: https://rhn.redhat.com/errata/RHSA-2016-0370.html
 RHSA ID: RHSA-2016:0370-00
 Associated CVEs:
 CVE ID: CVE-2016-1950
 CVE URL: https://access.redhat.com/security/cve/CVE-2016-1950

Files associated with this scan are in /var/lib/atomic/openscap/2016-04-17-09-14-28-278584.

As you can see in the above example, if the scanning image (in this case the openscap image) is not already on the host, atomic will pull it from the registry first.  When the scanner has completed, it is required to leave a JSON-based file containing its results on the host’s file system.  In this case, the results reside in /var/lib/atomic/openscap/2016-04-17-09-14-28-278584.

The location in the file system is always /var/lib/atomic plus the scanner name plus a time stamp.  This ensures differentiation between multiple scans.  Atomic will always display some level of feedback about the scan so that users have some immediate feedback.  Further details about the results, however, are always stored in the aforementioned place in the file system.

Note: The OpenSCAP CVE scan only works on RHEL based containers and images because it relies on CVE input that no other distribution is currently creating.

Atomic’s configuration file

Earlier when I mentioned that the default atomic scanner is defined in its configuration file, I referred to /etc/atomic.conf.  If you add more scanners to atomic, you can change the default here as well.  As of the writing of this blog, the configuration file appears as:

# Atomic CLI configuration file
default_scanner: openscap
default_docker: docker

 

Querying atomic about its scanners

If you pass the –list option to atomic scan, atomic will display its known, configured scanners.  Remember, atomic ships only with the openscap scanner configured at the time of this writing.  But if we query atomic about it’s scanners, it will reveal that the openscap scanner actually has two scan types:

# sudo atomic scan --list
Scanner: openscap * 
 Image Name: atomic-registry.usersys.redhat.com:5000/openscap/openscap
 Scan type: cve * 
 Description: Performs a CVE scan based on known CVE data

 Scan type: standards_compliance 
 Description: Performs a standards scan


* denotes defaults

A second scan type called standards_compliance is defined.  If you want to perform that scan, you simple would pass atomic scan the –scan_type standards_compliance switch.

Performing a scan with a non-default scan type

Lets take a look at the a standards_compliance scan.  By passing the –scan_type switch, the scan appears as such:

atomic scan --scan_type standards_compliance registry.access.redhat.com/rhel7/rhel-tools
 docker run -it --rm -v /etc/localtime:/etc/localtime -v /run/atomic/2016-04-18-14-08-02-813729:/scanin -v /var/lib/atomic/openscap/2016-04-18-14-08-02-813729:/scanout atomic-registry.usersys.redhat.com:5000/openscap/openscap oscapd-evaluate scan --targets chroots-in-dir:///scanin --output /scanout --no-cve-scan
registry.access.redhat.com/rhel7/rhel-tools (c28fabd46c7457b)
The following issues were found:
Ensure Software Patches Installed
 Severity: Important
 XCCDF result: notchecked
 Files associated with this scan are in /var/lib/atomic/openscap/2016-04-18-14-08-02-813729.

The non-default scan type is not very verbose yet as it is a work in progress.  But if you check the json file in the /var/lib/atomic/openscap/2016-04-18-14-08-02-813729/<ID>/ directory, you will find the openscap outfile in XML format which is very verbose and informative.

Adding additional scanners

In the future, we expect to have additional scanners that can be added to your system to be subsequently used by the atomic command.  The additional scanners are actually images and can be simply installed with the atomic install command.  For example, suppose a new scanner is available called new_scanner.  You would install the new scanner like so:

# sudo atomic install localhost:5000/new_scanner
Using default tag: latest
Trying to pull repository localhost:5000/new_scanner ... latest: Pulling from new_scanner
158758914368: Pull complete 
f859fa8adeb2: Pull complete 
b1592788a639: Pull complete 
Digest: sha256:d0a2dccc6ba33e4167c1c3ceb1edf366257d99ee9edde62adf8b3049a5571bc1
Status: Downloaded newer image for localhost:5000/new_scanner:latest

docker run -it --rm --privileged localhost:5000/new_scanner sh /install.sh
Added 'new_scanner' to /etc/atomic.d/

new_scanner is now ready for use and can be used like:

atomic scan --scanner new_scanner <image_name>

You can set new_scanner as the atomic scan default by editing /etc/atomic.conf.

And once the new scanner is installed, you can use atomic scan –list to see it:

# sudo atomic scan --list
Scanner: openscap * 
 Image Name: atomic-registry.usersys.redhat.com:5000/openscap/openscap
 Scan type: cve * 
 Description: Performs a CVE scan based on known CVE data

 Scan type: standards_compliance 
 Description: Performs a standard scan

Scanner: new_scanner 
 Image Name: new_scanner
 Scan type: scan1 * 
 Description: scan1


* denotes defaults

Conclusion

One of the exciting features of atomic scan is that the current design is pluggable.  You can pull in more scanners or design your own scanner for nearly any purpose. In my next blog post, I will cover how to make a custom scanner plug-in.

 

Share
  • Dharmit

    I’m trying this on Fedora 23 and CentOS 7 systems. Seems like `–list` option is no longer available. Even `atomic scan -h` doesn’t show that option.

    Also, how long should I expect a command like `atomic scan centos:7` to take to show me some results? I’ve the command running since over 30 minutes with no output and ps says it’s in S+ state.

    Probably not a valid question for this blog, but asking it anyway — since you say that CVE database for distro other than RHEL doesn’t exist, and since CentOS is a clone of RHEL, should atomic scan run against CentOS container images?

  • baude

    On your first comment, which version of atomic are you using? You will need to be running a very recent version if not even the upstream version.

    The scans are generally very quick so this indicates the likelihood of a problem.. probably due to version not new enough.

    While the CVE database might technically be valid for CentOS, the scanner does not see it that way. I have created a scanning image based on fedora. It is docker.io/fedora/atomic_scan_openscap

    • Dharmit

      I’m using atomic-1.9-4.gitff44c6a.el7.x86_64 on CentOS 7.2.1511.

      Thanks for the fedora Docker image. I’ll give it a spin.

  • Julien

    Hello,
    it does’nt seem to work for me using the same example (pulls steps have been cut):

    [root@slv-fr-poc-docker1 Tomcat7]# atomic scan registry.access.redhat.com/rhel7/rhel-tools
    docker run -it –rm -v /etc/localtime:/etc/localtime -v /run/atomic/2016-08-04-11-41-29-123413:/scanin -v /var/lib/atomic/openscap/2016-08-04-11-41-29-123413:/scanout:rw,Z -v /etc/oscapd:/etc/oscapd:ro rhel7/openscap oscapd-evaluate scan –no-standard-compliance –targets chroots-in-dir:///scanin –output /scanout

    registry.access.redhat.com/rhel7/rhel-tools (sha256:00bfb974)

    registry.access.redhat.com/rhel7/rhel-tools is not supported for this scan.

    Files associated with this scan are in /var/lib/atomic/openscap/2016-08-04-11-41-29-123413.

    [root@slv-fr-poc-docker1 Tomcat7]# atomic scan –list
    Scanner: openscap
    Image Name: rhel7/openscap
    Scan type: cve *
    Description: Performs a CVE scan based on known CVE data

    Scan type: standards_compliance
    Description: Performs a standard scan

    Did i forgot something ?
    I’m using RHEL 7.2 and not RHEL Atomic.

    Very nice blog, very nice article, i’ll put it in my favorites !

    Regards

    Julien

    • Brent Baude

      Hi Julien,

      Thanks for the feedback. Can you provide the version of atomic, rhel-tools, and the openscap image?

  • chuanchang

    could we ignore the following defect result for standards_compliance of scan type? thanks.

    Ensure Software Patches Installed
    Severity: Important
    XCCDF result: notchecked