Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Bootstrapping POWER8 little endian and common pitfalls

December 19, 2014
Aldy Hernandez
Related topics:
Linux
Related products:
Red Hat Enterprise Linux

Share:

    gnu logoEarlier this year I was asked to bootstrap our core tools (compiler, assembler, linker, and libraries) from the ground up, to help the rest of the team in providing enough infrastructure for bootstrapping an entire OS to POWER8 little endian.  Since I spend most of my days working on the upstream development of the GNU Compiler Collection (GCC), prior to this project I hadn't actually worked much with either RHEL's development processes or RPM as a whole.  So leading our effort to bootstrap the tools onto a new architecture required a lot of coming up to speed.

    Not being one to shy away from learning an entire new infrastructure, I accepted, and so began a 6 month ordeal fighting with everything from the assembler to GNU Emacs. Having learned so much from this project, I thought it would be good to write some of it down, both for the curious, and to help in future bootstrapping efforts.

    Consequently, I'd like to give an overview of how an OS is bootstrapped, and what insights I've learned that can help developers in designing packages that are easy to bootstrap and bring up in new architectures.

    Everything begins with the compiler and associated tools. Without a compiler and assembler there can be no editor, no GNOME, and no kernel. And without an OS, you have to depend on a pre-existing OS to bootstrap your compiler.

    Bootstrapping GCC from zero is a little bit of black magic, mixed with a fair amount of frustration and swearing mixed in. My general approach was the following:

    1. Cross build GNU Binary Utilities (binutils) (assembler, linker, and an assortment of other low-level tools) so we could run the assembler/linker/etc from an x86 machine and generate ppc64le binaries. This involved porting upstream ppc64le support into our RHEL 7.x code base.
    2. Build a minimal cross GCC using the binutils in step 1. This is basically a cross GCC without target libraries.
    3. Then we used the minimal GCC to build minimal GNU C Library (glibc) headers.
    4. Use the the above to build a cross GCC again, but this time with libgcc and some minimal target libraries.
    5. Build glibc.
    6. Build a full GCC that uses the glibc recently built. Once this stage is completed, we have a compiler capable of building ppc64le binaries from another host (x86 in my case).

    Once we have a bare compiler and libraries working, we can use these tools to manually build the core supporting packages (make, sed, bash, tar, gawk, grep, etc etc). And by building these packages manually, I mean completely manually, because we can't depend on ./configure or bash or anything else.

    Eventually, we end up with a set of packages that can be run on a ppc64le system (or a simulated qemu system in my case), where we can build the rest of the packages ala ./configure and make (albeit with many manual hacks thrown in).

    Most of the above steps had been automated by the 64-bit ARM team. I heavily modified their scripts, with changes that will hopefully make it into their stage1/stage2 scripts (see Stage1 notes and Stage2 notes).

    Past manual building with ./configure and make, we eventually have enough packages to build `rpm', at which point things get progressively easier tools-wise, but progressively harder package-wise. From here on out, packages must be built with rpm, but care must be taken to build things in the appropriate order. The appropriate order is mostly unknown, so a lot of trial and error happens at this stage. Building autoconf may need emacs which may need ImageMagick, at which point you're pretty much building the entire OS.

    This stage is the most infuriating and takes the longest. You must build packages breaking circular dependencies by hand-- sometimes, by hacking .spec files, and sometimes by hacking source files. But eventually you end up with enough RPMs to be able to fire up yum and ultimately mock. Once you get to mock, if you can successfully rebuild all your packages within it, you are done.

    For the uninitiated, mock is a tool that works in unison with yum to build a given SRPM in a chroot.  It takes care of downloading and installing RPM dependencies and building your SRPM in isolation from your build system's setup.  It is useful in building and testing packages, as you can build a package for say RHEL 7.1, even though your host system is Fedora 20.  If you read through the mock build logs, you can see exactly which packages your package needs to build.  This is very useful in minimizing your package's dependencies and optimizing the bootstrap the process.

    Does all this sound hard? It is! Perhaps not hard, but time consuming and frustrating. It's at this point where you write down everything that could have been done differently... So here are a few hints and tips for package developers and maintainers to aid in the inevitable future architecture that some poor soul must port RHEL/Fedora to.

    • Do not stream ABI details to disk. ABI details should never be streamed to disk. This makes it extremely hard to bootstrap an OS, and guarantee a clean ABI.
    • Verify and re-verify that you have the minimum amount of dependencies. With every OS release, verify that your BuildRequires and Requires are correct in your .spec files. Unnecessary requirements complicate the bootstrap process.
    • Try to build your package from a minimal chroot. This exercise will help you determine what is actually needed for your package. You'll be surprise how often ImageMagick is unnecessarily required for some core package.
    • Avoid having simpler packages depend on more complex ones... for example, right now, bison depends on Perl, and autoconf depends on emacs.
    • libtool is evil. Do you really need it? Can you do without it? If you can't, make sure your package can auto reconfigure itself if a user touches an .m4 file. And make sure, it can be reconfigured with the auto tools in the OS version you are building for.
    • autoconf is also evil. See above notes on libtool.
    • Do not make assumptions about endianness of architecture. Glibc has macros for determining this (see <endian.h> and the __BYTE_ORDER macro). Heck, there are even autoconf tests for that if you must.
    • When possible use the lua extension language for rpm instead of external tool dependencies (say perl or php). Lua is a an embedded scripting language available for RPM .spec files that can minimize your package build dependencies.
    • RPM spec files have a blessed way of dealing with patches. Don't try to be clever. For instance, groff.spec uses `git' to apply patches. Really, don't be clever. If you dislike RPM's patch syntax/usage, propose a better solution.
    • Use .spec file options such as --without-docs or globals like %{bootstrap} to specify shortcuts around building the entire package.  90% of bootstrapping problems stem from missing dependencies for documentation.  Providing a %global or an option to avoid building documentation saves a lot of effort in the bootstrapping process.
    • Only place tests in the %check section so they can be easily overridden while bootstrapping.

    All in all, bootstrapping is a tedious process, but its pain can be alleviated by following simple guidelines and testing your adherence to them periodically (ideally with every release).

    And the end results can be great - Red Hat Enterprise Linux 7.1 Beta is available to all RHEL subscribers today with support for POWER8 Little Endian.

    Thanks for reading - as ever, we welcome your feedback via the comments.

    Last updated: March 15, 2023

    Recent Posts

    • How to build a Model-as-a-Service platform

    • How Quarkus works with OpenTelemetry on OpenShift

    • Our top 10 articles of 2025 (so far)

    • The benefits of auto-merging GitHub and GitLab repositories

    • Supercharging AI isolation: microVMs with RamaLama & libkrun

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue