This article introduces you to using Thoth's security recommender to scan for flaws in your Python applications. Security checks were recently added in Project Thoth, a cloud-based resolver for Python applications.
Why we need security recommendations
Software engineering history has proven countless times that shipping secure software is not easy, and that security problems often lie not in the application developer's code but in imported packages. Software commonly uses libraries written by other developers, and an application's security is dependent on those libraries. If a library has vulnerabilities, the dependent application will inherit them when the library code is executed. Given the complexity and variety of libraries available in the open source world, it is not possible to review all the source code publicly available with rigorous security expertise.
Project Thoth is a native application on Red Hat OpenShift, hosted on the Operate First environment. Thoth can resolve software packages not to the “latest” but to the “greatest” library version. Thoth offers a variety of recommendation types to determine the "greatest" version based on different criteria, which guides the installation process. Security is now one of Thoth's recommendation types.
How Thoth makes security recommendations
Thoth uses three main sources for security-based advisories that affect Python packages:
- The Python Packaging Advisory Database
- Results computed in Thoth's "security indicators" workflow
- Security scorecards for open source projects by the Open Source Security Foundation (OpenSSF)
Recently, the Python Packaging Authority (PyPA) announced a publicly available database of known vulnerabilities in open source Python libraries, mostly those hosted on the Python Package Index (PyPI). Thoth’s background data aggregation logic periodically fetches the database of known vulnerabilities and automatically blocks the resolution of software package versions that are prone to security vulnerabilities.
Note: Watch this video for a demonstration of how the resolver flags security problems.
Thoth engineers created the second source of data for security-based advisories. This source uses Thoth’s data aggregation architecture, in which each package imported by the application is statically scanned for possible issues using the open source Bandit tool. The information derived by Bandit is used to score packages during resolution. See Thoth’s security advisories documentation for more about how Thoth uses Bandit.
Note: This video demonstrates how the Bandit-based recommendations are computed.
The third source of security-related advisories consists of security scorecards that provide health metrics for open-source software. See the OpenSSF blog or the OpenSSF GitHub repository for more about security scorecards.
Thoth’s resolution engine can easily be extended to include more sources of information. We are constantly looking for new sources and mechanisms for scoring packages. If you have domain knowledge in this area, the Thoth team welcomes your contributions and suggestions.
Get a Thoth security recommendation
To enable security-based recommendations that include the sources stated earlier, simply configure the recommendation type to be "security" in any of Thoth's three types of client tooling:
- The Thamos command-line interface (CLI)
- Thoth’s OpenShift Source-to-Image (S2I) container images
- jupyterlab-requirements
The easiest way to test security-based recommendations is to install the Thamos CLI, configure it, and ask Thoth for an advisory:
$ pip install thamos
$ thamos config
$ thamos add flask
$ thamos advise --recommendation-type security
In this example, the cloud-based resolver evaluates versions of Flask as well as its dependencies. It will look for known security issues and suggest the most appropriate versions based on the most recent security knowledge for those dependencies. (See the end of this article for a video demonstration of the resolution process.)
Using Thoth in the S2I build process
The Thoth team provides container images that have Thoth tooling to consume recommendations during the OpenShift S2I build process. These container images are hosted on Quay.io in the thoth-station organization. For the documentation, see the GitHub repository hosting Thoth's S2I container images.
To benefit from Thoth’s recommendations in an OpenShift S2I process, use Thoth’s S2I base images during the build process. Because these images are compatible with vanilla S2I container images (in fact, the Thoth images are based on the vanilla ones), all you need to do is change the builder container image used in the S2I build process. The thoth-s2i tool can facilitate an automated transition to a Thoth-guided S2I build process.
Note: This video shows the adjusted OpenShift S2I build process with Thoth advisories.
Watch a video demonstration
The following video gives an in-depth demonstration of how to resolve Python software packages without security vulnerabilities.
Helping the Python community create healthy applications
As part of Project Thoth, we are accumulating knowledge about Python packages to help Python developers create healthy and secure applications. If you would like to follow updates in the project, please subscribe to the Thoth Station YouTube channel or follow us on the @ThothStation Twitter handle.
Last updated: September 19, 2022