< Summit 2020

Building your apps for Red Hat Enterprise Linux: Recommended practices for certification and supportability

Introductory | Take advantage of Red Hat Container Certification
Joel Destefano

Joel Destefano

Shubha Bavde head shot

Shuba Bavde

Learn about certification and supportability of Red Hat Enterprise Linux applications. We'll cover policy, best practices, and guidelines for certifying non-containerized applications on Red Hat Enterprise Linux, along with certifying using Red Hat Enterprise Linux Universal Base Image for containers.

Talk Text

Joel Destefano (00:03):

Welcome everyone to Red Hat Summit Virtual Experience 2020. Today's topic is building apps for Red Hat Enterprise Linux with recommended practices for certification and supportability. My name is Joel Destefano. I'm a certification product manager here at Red Hat. With me today is my colleague Shubha Badve, who's also a certification project manager.


Joel Destefano (00:29):

So today's agenda. We'll go over an overview of Red Hat Enterprise Linux build-and-certify options. They'll include, what is Red Hat technology certification at a high level? We'll also talk about how Red Hat Enterprise Linux 8 is foundational to today's application-development options.


Joel Destefano (00:47):

Then we'll have two different deep dives. One of them, I will cover, which will go over building and certifying non-containerized apps on Red Hat Enterprise Linux 8. I'll cover specific topics for best practices, supportability, and certification, and I'll also point you to some resources to help you with your build-and-certify journey.


Joel Destefano (01:07):

At that point, I'm going to hand it off to Shubha, and she's going to cover building and certifying containerized apps with Red Hat Enterprise Linux Universal Base Image. She'll cover, what is the Red Hat Enterprise Linux Universal Base Image? She'll also cover specific topics for best practices, supportability, and certification along with covered resources.


Joel Destefano (01:31):

So, what is Red Hat technology certification at a high level? Well, up on the screen here is our mission statement, which is an assurance that a commercial software product or solution interoperates with Red Hat platform in a secure manner with best practices and is jointly supported. What this means is, whether you're a developer of commercial software that will run on Red Hat platforms or an enterprise developer who might consume a commercial vendor's product, you should understand the value of certification and how it relates to interoperability, security, best practices, and supportability.


Joel Destefano (02:06):

Certification literally guides the developer when it comes to packages, libraries, and runtimes with a focus on security and life cycle. What this does is, it increases confidence in an application stack and ensures an ease of supportability. Here's a Venn diagram. Basically, we have on one side of the recommended practices that we've built up over the years at Red Hat. Then we have key aspects of supportability in another circle.


Joel Destefano (02:34):

When you put those together, that's how we derive and come up with a certification program. In this case, the certification program is for Red Hat Enterprise Linux, and there are two discreet but closely related build-and-certify paths that we'll discuss today.


Joel Destefano (02:53):

In the agenda, I talked about how Red Hat—RHEL is the foundation to what we do. And when you are going to pick a Linux technology platform, you want to make sure that your technology platform can support existing product development and also accelerate development of future application-modernization efforts.


Joel Destefano (03:13):

Now, existing development is covered with common runtimes such as PHP, Python, Ruby, and Node.js, and the number of other supportive packages and libraries that we present with predictable life cycles and a commitment to security, as always. But Red Hat Enterprise Linux also provides a path to modernization with some inherent core technologies and tooling that we'll discuss in the coming slides.


Joel Destefano (03:39):

So, how do you accelerate development of future modernization efforts on Red Hat Enterprise Linux 8? Well, RHEL 8 is both a development sandbox for containers and a great single-node container deployment platform. Core technologies—such as cgroups for resource management, namespace for process isolation, and SELinux for security—are integral to containers.


Joel Destefano (04:01):

And these are ingrained in Red Hat Enterprise Linux 8. It's important to note that change happens really fast, but if you've been in the industry long enough, you realize that the fundamentals tend to stay the same and it's nothing to be scared or daunted by. Red Hat Enterprise Linux is literally the foundation for the present and the future.


Joel Destefano (04:23):

Specifically, how can Red Hat Enterprise Linux 8 accelerate development of future modernization efforts? Well, RHEL 8 has adopted a container tooling that's compatible with the OCI specifications. This is a set of command line tools that can operate a container without a container engine. Podman, Buildah, and Skopeo are the three main tools.


Joel Destefano (04:43):

Podman is for managing container images; Buildah for building, pushing, and signing container images; and Skopeo for copying, inspecting, deleting, and also signing images. So, now let's dive a bit into building and certifying non-containerized applications on RHEL.


Joel Destefano (05:02):

Then my colleague Shubha will take the presentation home with the journey to containers with Red Hat Enterprise Linux Universal Base Image. So, I talked to you earlier about accumulating best practices and bringing in aspects of supportability into the build process, which will result in a certified application.


Joel Destefano (05:21):

Now, accounting for best practices and supportability when building translates to more easily developed, more easily packaged, deployed, maintained, certified, and supported applications, giving everyone—Red Hat, our partners, and our customers—more confidence in the deployed application stack.


Joel Destefano (05:41):

Now, please note that certification status is not a gate to Red Hat's world-class technical support organization working support case. Certification simply enables a better support experience through proactive technical checks of how a third-party software application affects the underlying Red Hat Linux platform. To learn more about how support operates in the references section or the resource section, I have a production scope of coverage for support, which you should refer to.


Joel Destefano (06:11):

So, let's talk about best practices, supportability, and your build-and-certify journey with some specifics. Now, certification is bound by certification policy, which is a collection of rules, statements, and guidelines, many of which are derived by feature supportability found in Red Hat Enterprise Linux.


Joel Destefano (06:29):

I'll go over some examples. With our internal testing of footprints and hypervisors that are supported, we've ensured portability between them, and certification can be conducted on any one of these footprints or hypervisors and pass through the others. It's important to know, however, that certification never replaces a vendor statement of support, and vendors' documents should always be referenced. The last block here is on supported architectures. There's a slight difference here.


Joel Destefano (06:56):

Certification is tied to a specific hardware architecture and doesn't pass through. If you certify an x86, that's not going to pass through to IBM power. You need to run another certification. So, some more-specific checks. Let's talk a little bit about the Red Hat kernel, the use of third-party kernel modules. We're going to also talk about Red Hat packages and third-party packages.


Joel Destefano (07:21):

So, certification aims to determine whether or not the kernel has been modified in a way that could cause issues with supportability. Now, the use of third-party kernel modules is totally acceptable, but what it will do is instigate something called an exception review.


Joel Destefano (07:33):

Red Hat will take an engineer and look a little bit closer at the kernel modules to see what they're doing and make sure that you're in a good position for support. It's important to remember that recompiling the kernel from memory mapping is a big difference than installing a kmod to do packet inspection for security application, and the exception review process aims to take a look at these types of things.


Joel Destefano (07:56):

It's the same principle when it comes up Red Hat packages that we ship and the use of third-party packages. We don't want to have Red Hat packages modified in a way where supportability becomes questionable. So, we'll have an exception review if packages are modified.


Joel Destefano (08:10):

There is a white list for acceptable modifications. For example, configuration-related changes to an RPM is not going to be a failure. That's going to be white-listed. Third-party packages required for the application are obviously acceptable, but we want to confirm that they don't interfere with Red Hat packages, so there'll also be an exception review here. This is all in an effort to increase ease of supportability when a ISV commercial software product is running on a Red Hat platform and a customer calls in with support.


Joel Destefano (08:45):

Want to know two things here. I want to talk about the API and ABI compatibility guidelines. I also want to talk about packaging with RPM. The API/ABI guidelines—so ISVs in customer applications—can reduce or avoid migration issues between major and minor versions that they could follow during application development.


Joel Destefano (09:07):

It's a tiered framework with levels one through four from the broadest range of compatibility to least range compatibility. In the non-containerized RHEL certification, we have actually added an RPM analysis tool, and it checks to see: If an application has packages and RPM, how well is it packaged?


Joel Destefano (09:26):

Versioning, provenance, and dependency tracking are checked with Red Hat's guiding principles for packaging RPMs well. It's important to note that a well-packaged RPM application is nicely suited for the journey to containers as a good separation of configuration code and data, along with awareness of versioning, provenance, and dependency tracking are key aspects of portability and life cycle, which are very important underlying principles to the principles of containers.


Joel Destefano (09:57):

So, here we've collected some resources for your build-and-certify journey that I think every developer, whether you're a developer producing a commercial vendor products or an enterprise-grade developer consuming such products. The RHEL 8 release notes. The kernel administration guide. Managing, monitoring, updating the kernel. Packaging and distributing software with RPM. The certification policy, of course: I highly recommend you take a look at that, as it'll go over all the checks in our certification for non-containerized RHEL applications.


Joel Destefano (10:29):

The API and the ABI compatibility guidelines are very important. And again, so there's no confusion with how support operates, the production support scope of coverage is also listed. A couple more resources for you to start your build-and-certify journey with Red Hat: There's the Red Hat ecosystem catalog, which will go over listed, certified applications. If you want to start a certification journey with us, you can use Partner Connect to get started. And as always, there's the developer program.


Joel Destefano (11:17):

And with that, I'm going to turn it over to Shubha, who's going to take this presentation home and talk about Red Hat Enterprise Linux Universal Base Image and building and certifying containerized applications.


Shubha Badve (11:34):

Thank you, Joel. What we have seen in recent years is that people are adopting hybrid cloud infrastructures more and more, and that has prompted the adoption for application modernization. Now, for a typical monolithic application, the application-modernization journey would look something similar to what I have shown here.


Shubha Badve (11:58):

That is going from monolithic architecture to a client-server type N-tier architecture to a microservices architecture. There are two distinct characteristics of a monolithic application, which make it desirable to modernize it. Now, it is very difficult to update a monolithic application. It is also very difficult as well as expensive to scale a monolithic application, because you're not able to scale just the components that need scaling but you have to scale the entire application. That makes it very expensive.


Shubha Badve (12:39):

Now, instead, if you adopted the microservices technology, where the components are smaller and are loosely coupled, you can update, deploy, and scale individual components independently. There are two technologies which are widely adopted for application modernization. One is Linux containers, and the other is Kubernetes.


Shubha Badve (13:07):

Red Hat offers two technology platforms which align very well with these two technologies. One is the Red Hat Enterprise Linux platform, and the other is Red Hat OpenShift Container Platform. Now, our focus for today's session is on the Red Hat Enterprise Linux platform.


Shubha Badve (13:31):

Now, as we have worked with our partners as well as customers—those who have gone through the application-modernization journey or the containerization journey—what we have seen is that their requirements haven't really changed from the traditional application development and now containerized application development perspective.


Shubha Badve (13:54):

They still want the flexibility to be able to build anywhere. By anywhere, what I mean is that they want to be able to develop or build in Red Hat environments as well as in non Red Hat environments. They also want to be able to select a suitable language runtime that fits their application. They also want to be able to add just what they need from content or package perspective to keep the size of their application as small as possible from memory footprint on this perspective.


Shubha Badve (14:31):

Now, from an operation-staff perspective, they don't want to compromise the security and performance. They don't want to see the security and performance of the application compromised. So, what Red Hat has done in order to make our partners' and customers' application-modernization journey smoother is that last year we released the Red Hat Universal Base Image.


Shubha Badve (15:00):

The purpose with which we released the Red Hat Universal Base Image was to make a highest-quality enterprise-grade but most flexible base container image available to our partners and to our customers as they start to containerize their application or develop right from scratch containerized applications. Now, even though the name suggests that it's a base image, the Red Hat Universal Base Image is really a content set. It offers three different base images with a minimal memory footprint. It offers prebuilt language runtime images, so they're already prebuilt for you. And it also offers a subset of the entire Red Hat Enterprise Linux package set.


Shubha Badve (15:52):

Now, the Red Hat Universal Base Image is made available at no charge to any developer, to any user. That is because we released it with its own end user license agreement. You don't need Red Hat subscription to use and distribute UBI. That means what you can do is, you can standardize on UBI as the base image for all your container needs.


Shubha Badve (16:21):

What that does is that you only need to develop using UBI, whether you're running in Red Hat environment or you intend to run your applications in Red Hat environment or in non Red Hat environment. You can still use UBI as your standard base image.


Shubha Badve (16:40):

What that does is that it even further simplifies your build engineering or CI/CD pipeline. Now, we offer three base images, so you can choose just the right one based on what application you're developing. In our experience, for 80% of the workload, the UBI base image should be suitable for you. Now, if you really want to minimize the memory footprint on this, then you should go with ubi-minimal. If you intend to run multiple services within a single container, then you want to choose ubi-init.


Shubha Badve (17:21):

We only have now one wishlist item left on this list, which is security and performance. As I alluded to it before, the Red Hat Universal Base Image is based on Red Hat Enterprise Linux. It offers the exact same performance, security, as well life cycle as Red Hat Enterprise Linux. We have a separate performance engineering team who tests these base images with all sorts of workloads to make sure the performance is not affected.


Shubha Badve (18:02):

We also have engineers who are part of different security communities who are actively looking for any new vulnerabilities that we can proactively fix. So, you can be—rest assured that when you are starting your containerization journey with UBI as a base image, you are in good hands. And there you have it. Whether you are developing a traditional application with Red Hat Enterprise Linux or you're starting your containerization journey with the Red Hat Universal Base Image, you have all the flexibility and the choices and a solid secure foundation. Now, let's see how the Red Hat Container Certification fits into all this.


Shubha Badve (18:52):

As Joel mentioned earlier, the certification is offered as part of the Red Hat Partner Connect program. Now, if you are an end user who is deploying third-party application in your environment or you are a Red Hat partner developing products based on Red Hat Enterprise Linux, you should benefit from the certification process.


Shubha Badve (19:17):

And that is because as part of the certification, we verify that the partner products are aligned with the best practices and the supportability criteria which are set forth by Red Hat. So, a win-win situation. Now, in order to certify, our partners have to meet certain requirements.


Shubha Badve (19:43):

There are certain requirements which have to be met for the product as a whole. Then there are certain requirements for the individual container images which are based on our best practices. So, product as a whole needs to be tested and supported on the targeted Red Hat platform.


Shubha Badve (20:02):

That can be Red Hat Enterprise Linux platform or the Red Hat OpenShift Container Platform. The partner must include supporting documentation. For the individual container images, they must be built on supported Red Hat container base images.


Shubha Badve (20:20):

And the base images we currently allow are RHEL 7 or UBI 7 or UBI 8 base images. The container images must include latest security updates, must include metadata for the ease of management, and avoid any modification to the Red Hat packages, which are included within those container images.


Shubha Badve (20:45):

We also recommend—and this is a big a recommendation from the perspective of security—we want to make sure that our partners are not running these container images as the root user. And also they're not requesting host-level privileges while they run these container images. That is to protect the security of the underlying host or platform.


Shubha Badve (21:15):

And that is very important. Of course, there are some exceptions, and we handle them as such. Now, unfortunately, the UBI content set just wasn't enough for all our partners. They just couldn't develop or containerize their application using just a subset offer at Red Hat Enterprise Linux.


Shubha Badve (21:38):

And for those partners, this year in March, we expanded the scope of containers of certification. Now, the Red Hat Partner Connect members can freely use and redistribute any Red Hat Enterprise Linux user-based packages when they build upon UBI base images and upon completing Red Hat Container Certification.


Shubha Badve (22:05):

Now, here I have included a Dockerfile, which you can use sort of as a template as you start your containerization journey. This will show you what different best practices you can accommodate while you're developing your container image. Now, here, as I had said before, we require that the container image must use Red Hat base image, and here via using UBI 8, UBI base image.


Shubha Badve (22:35):

The labels which are included here is the metadata that is going to make management easier. Now, if you are developing your application using just the UBI content set, then you want to make sure that you're pulling the necessary packages from the UBI repo. And in order to do that, you want to make sure that the subscription manager is disabled. So, you must do that inside your Dockerfile, because you don't want to accidentally pull any packages from the RHEL repo.


Shubha Badve (23:16):

Now, if you're developing your product using the entire RHEL user-based packages, then you want to make sure that you are going to build your images on a subscribed RHEL host. Red Hat does offer a no-cost RHEL subscription through the Red Hat Developer program and as well as for partners through the Red Hat Partner Connect program. You can certainly take advantage of that.


Shubha Badve (23:43):

Now, by default on a RHEL subscribed host, only the RHEL server repo is enabled. So, if you need to include packages from any other repos, what you want to do is that—specifically enable those repos inside your Dockerfile using the "enable rebuild" command.


Shubha Badve (24:04):

So, I hope that this will serve you as a template as you start your containerization journey and it will help you along. Just as a last point from supportability perspective, how everything fits together. If you have an application that is developed using the Red Hat Universal Base Image—what I'm trying to show here is that, if you deploy it on a non Red Hat platform, then you can still consume any updates that we provide for the packages or the base images or the language runtime as a result of a new security errata or as we have released a new version or something of that sort.


Shubha Badve (24:54):

So, the support gets better from right to left. Now, in the middle block, what I've shown is that if you have a UBI-based application that you have deployed a Red Hat platform, now you get enterprise support from Red Hat.


Shubha Badve (25:09):

So, now you can call us for any problems with the runtimes or the base image or any problems with the platform. Now, the leftmost block is where you get the highest level of support. And that is when your deployed is certified partner product as a workload and you're running, or you have deployed, those UBI base images on a Red Hat platform.


Shubha Badve (25:36):

Now, if you have problems with any of the components up and down the stack, you can call Red Hat, because Red Hat now for the certified third-party products provides collaborative support. That's the highest level of support. And I hope as a partner or as an end user, you can take advantage of all these benefits.


Shubha Badve (26:03):

In the end—hold on a minute, let me turn off that slide. I have also included some resources that you can refer to if you are just looking to use the Red Hat Universal Base Image or you want to get certified with us. I would strongly encourage you to go to Red Hat Partner Connect portal if you're starting certification journey. And if you're starting your development using UBI, then go to the Red Hat Developer site and take a look at the UBI FAQ page.


Shubha Badve (26:37):

I think you'll find those resources very helpful. And that's all I have for you today. Thank you for joining us. If you're a partner, go to Red Hat Partner Connect. Become a member. If you are a developer, adopt UBI and start your application-modernization journey. Thank you all for joining us again. Thank you.

Session Speakers

Joel Destefano


Joel Destefano

Red Hat - Principal Product Manager

Joel is a Product Manager for Red Hat Enterprise Linux application certification and Kubernetes since Kubernetes 0.13.

Shubha Bavde head shot

Shuba Bavde

Red Hat - Principle Product Manager

Product Manager of Red Hat Container Certification service.