Python offers a wealth of programming libraries, which often invoke functions from other libraries in complex hierarchies. While these libraries make it possible to develop powerful applications quickly, the ever-changing library versions often introduce conflicts that cause runtime or build-time issues. Thoth, an open source project developed within the Artificial Intelligence Center of Excellence (AICoE), is dedicated to alleviating this problem in Python programs. This article looks at Thoth prescriptions, a mechanism that you can use to avoid clashing library versions in your Python applications.
Note: For more about resolving Python project dependencies with Thoth, see our recent introduction to Thoth Dependency Monkey.
Curated knowledge about Python libraries
One of Thoth's major offerings is a cloud-based resolver, which examines the packages and libraries requested by an application and determines the best way to resolve them so that they work together in the target environment. Thoth's prescription repository is comparable to the more familiar security project from the Python Packaging Authority (PyPA), a curated database of known vulnerabilities in the Python ecosystem. The Python Advisory DB resulting from that project is now available on GitHub. The repository contains YAML files describing known vulnerabilities in machine-readable form.
Whereas the Python Advisory DB focuses only on security flaws, Thoth prescriptions are more generic and are directly consumed by the resolver. The database curates a broad range of knowledge about Python libraries and packages: Their communities, known build-time issues, runtime issues, compatibility with native dependencies, suggestions for which runtime to use, or other suggestions of interest to Python package consumers.
Prescription information is stored as YAML files and used automatically by the Thoth resolver to guide Python application developers. Anyone can contribute to this database. The Thoth resolver automatically loads prescriptions and consults them during the resolution process, so that resolved dependencies are in a healthy state. As a result, developers can focus on application development rather than on fixing library issues. Unlike other resolvers, such as pip, Pipenv, or Poetry, which tend to resolve the latest libraries, Thoth’s resolver chooses the libraries that best fit the application's needs, and takes the prescriptions into consideration.
Examples of Thoth prescriptions at work
In a previous article, the Project Thoth team showed an issue in the TensorFlow software stack that occurred when urrlib3 was installed with package six. The problem is recorded in a prescription that helps the Thoth resolver avoid trying to combine problematic versions of these packages. Applications using the Thoth resolver do not suffer from this recognized runtime problem.
Another example is a Pillow issue that will not work with NumPy. A prescription is provided to protect Python application stacks from this issue.
Yet another example warns users about the archived argparse project. Using archived projects on GitHub that suffer from the problem might drive users away, so this prescription notifies users about the project state.
Browse the Thoth prescriptions repository or documentation for more examples.
Watch a video about Thoth prescriptions
Are you curious about how "greatest" stacks are resolved? This video introduces a community-curated database that Thoth users can benefit from:
Note: see Healing Python applications with prescriptions for a video overview of the prescriptions concept. Visit the Thoth Station YouTube channel for more videos like these.
Calling all Python developers and package maintainers!
If you are a Python developer or Python package maintainer, we encourage you to get involved in building the prescriptions database. You can report issues to be turned into prescriptions, which are used to help create healthy Python applications.
If you would like to report a library issue, reach out to us at the Thoth Station support repository. You can also write prescriptions directly by following the online documentation. If you would like to be notified when a new prescription is created for a library, add yourself to the repository's CODEOWNERS file to follow per-project prescription updates.
Helping the Python community create healthy applications
As part of Project Thoth, we are accumulating knowledge to help Python developers create healthy applications. If you would like to follow updates in project Thoth, subscribe to our YouTube channel or follow us on the @ThothStation Twitter handle.
Last updated: September 19, 2022