Skip to main content
Redhat Developers  Logo
  • Products

    Platforms

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat AI
      Red Hat AI
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • View All Red Hat Products

    Featured

    • Red Hat build of OpenJDK
    • Red Hat Developer Hub
    • Red Hat JBoss Enterprise Application Platform
    • Red Hat OpenShift Dev Spaces
    • Red Hat OpenShift Local
    • Red Hat Developer Sandbox

      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Secure Development & Architectures

      • Security
      • Secure coding
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • Product Documentation
    • API Catalog
    • Legacy Documentation
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Thousands of PyPI and RubyGems RPMs now available for RHEL 9

June 7, 2022
Miroslav Suchý Jakub Kadlčík, Karolina Surma
Related topics:
LinuxOpen sourcePython
Related products:
Red Hat Enterprise Linux

Share:

    Red Hat Enterprise Linux 9 now offers convenient (but unsupported) access to RPMs from two of the largest and most popular code repositories: The Python Package Index (PyPI) for Python and the RubyGems collection for Ruby. This new offering makes it easier to use thousands of community libraries in your projects. We'll look at the repositories in this article.

    The Red Hat repository ecosystem and COPR

    Red Hat supports about 2,500 packages. They are of very high quality, created and maintained by experts. Outside our supported offerings for RHEL 9, we also have Extra Packages for Enterprise Linux (EPEL 9) with an additional 3,000 packages. But there are a lot of other important libraries and utilities on the internet.

    We certainly cannot package everything. But we have decided to provide you with an additional 151,000 RPM packages by packaging everything we can (with an exception we'll discuss in a moment) in PyPI and RubyGems. The packages have been added to another set of unsupported packages called COPR, which stands for community projects. Both EPEL and COPR were formed by the Fedora project.

    Benefits of using the signed Red Hat repositories

    As noted, Red Hat does not offer support for COPR packages. Each is provided as-is, without warranty. If you have an issue with some package, please contact an upstream author.

    However, Red Hat does provide some security through signatures. Packages on PyPI are not signed, but Red Hat signs the RPM packages in COPR repositories. This means that you can audit the packages: You know which files belong to which package and vice versa, and you will be able to check whether a file was altered by a user.

    To enable the PyPI COPR on your computer, enter:

    # dnf copr enable @copr/PyPI epel-9-x86_64

    Now, if pip install foo works for you, installing python3-foo from this repository will work too.

     To enable the RubyGems COPR, enter:

    # dnf copr enable @rubygems/rubygems epel-9-x86_64 

    Having enabled the COPRs on your system, you can get the signed keys by entering:

    # dnf install distribution-gpg-keys
    # rpm --import /usr/share/distribution-gpg-keys/copr/copr-@copr-PyPI.gpg
    # rpm --import /usr/share/distribution-gpg-keys/copr/copr-@rubygems-rubygems.gpg

    Details and limitations

    Not all packages from PyPI and RubyGems could be included in our COPRs. Some packages suffered from build problems, which I'll describe later. Others were excluded due to licensing conflicts. If the maintainers of a package fail to explicitly assign a free or open source license, they leave the gem by default under conventional "All rights reserved" copyright. Because these packages are not open source, we cannot distribute them.

    We provide the latest version of each Python package and RubyGem. When a new version is published, we rebuild it and update the repository. The previous version is deleted after 14 days. If you need an older version, upload it to Red Hat Satellite or keep a local copy.

    What we built from PyPI

    Many improvements to Python packaging were made in recent years, both upstream and downstream in RPM.

    With new RPM macros created by Python maintainers in Red Hat, it is now possible to create a deterministic converter from Python package metadata to RPM specfiles. We used a new tool, pyp2spec, making use of those new RPM macros, to rebuild PyPI packages as RPMs in COPR.

    Starting in December 2021, we tried to rebuild all packages on PyPI (more than 330,000 at the time) in Fedora Rawhide. When it became possible to build packages for EPEL 9 in COPR, we went ahead and reran the build. We completed this project, building 79,842 packages for Red Hat Enterprise Linux 9 and making them available in the PyPI COPR.

    Only the packages that were built successfully for Fedora Rawhide were submitted for the Red Hat Enterprise Linux 9 rebuild. Trying to rebuild the whole PyPI repository would require months, and there was little chance that any given EPEL build would succeed if the Rawhide one didn't.

    Based on the build logs of the failed packages, we found the following problems:

    • 32% had missing build dependencies in our COPR repository, often because a package required a specific version of a dependency.
    • 20% didn't have a license or specified a license that wasn't detected as open source.
    • About 13% of packages had an error somewhere in the upstream configuration leading to the build failure. Typically, the problem was due either to missing files in the source archive or to failures to import modules that weren't declared as build dependencies.
    • 12% of packages didn't have the source archive uploaded to PyPI, which prevented us from building the RPM. If a package was built successfully for Rawhide but not for EPEL, the typical culprit was missing dependencies.

    If you encounter a package that's installable from pip, but is not available in our COPR, you can try contacting the upstream maintainers to discuss loosening version requirements for dependencies, fixing the license declaration, uploading the source archive to PyPI, or resolving other problems as the case may be.

    For instructions on how to reproduce the build, see our PyPI COPR repository.

    What we built from RubyGems

    Support for building RPM packages in COPR directly from RubyGems.org was introduced back in 2016. The procedure uses a tool called gem2rpm to convert gem metadata into an RPM specfile and produce an SRPM package based on it.

    Utilizing this feature, we have rebuilt all of RubyGems.org for Fedora Rawhide. In this detailed blog post, you can find more information about the success rate, the size of the packages and their metadata, COPR internals, and takeaways for the RPM toolchain.

    Countless performance improvements, and months of building later, we are now announcing that all of RubyGems.org was rebuilt for Red Hat Enterprise Linux 9. The RubyGems project in COPR provides 71,952 packages, which is almost half of the RubyGems.org service. We got 19,635 failures because of unmet dependencies, and around 77,000 gems were skipped and not even attempted to be built because of missing licenses. A full 37% of gems in RubyGems do not specify a license.

    If your gem isn't available from the RubyGems COPR, it is most likely due to a missing license. Please resolve such issues with the respective gem owners. The same applies to missing dependencies. If you find a problem with the generated specfiles, please file a new issue for gem2rpm.

    Conclusion

    Our PyPI and RubyGems projects demonstrate Red Hat's desire to help programmers make the most of free and open source resources. During our months-long efforts, we turned up weaknesses in the source repositories that guide upstream developers to produce more robust packages.

    Last updated: October 8, 2024

    Related Posts

    • The Security Benefits of RPM Packaging

    • RPM packaging: A simplified guide to creating your first RPM

    Recent Posts

    • How to deploy the Offline Knowledge Portal on OpenShift

    • Autoscaling vLLM with OpenShift AI

    • Filtering packets from anywhere in the networking stack

    • PostGIS: A powerful geospatial extension for PostgreSQL

    • Kafka Monthly Digest: September 2025

    What’s up next?

    Linux server image

    The Intermediate Linux Cheat Sheet introduces developers and system administrators to more Linux commands they should know.

    Get the free cheat sheet
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Platforms

    • Red Hat AI
    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue