Automate TLS certificate management using cert-manager Operator for OpenShift

Having successfully installed the cert-manager Operator for Red Hat OpenShift, let's now explore a practical scenario to leverage its capabilities.

In this use case, we aim to integrate cert-manager with the cluster ingress operator (Figure 1). Typically, each ingress controller employs a default certificate for secure routes, unless a custom certificate is explicitly specified. Here, our objective is to utilize cert-manager to generate a custom certificate issued by a self-signed certificate authority (CA). Subsequently, we will proceed to update the default certificate of the ingress controller to use this newly generated certificate.

Notably, you have the flexibility to employ other supported issuer types such as ACME, Vault, and Venafi if needed.

In this lesson, you will:

• Integrate cert-manager with the ingress operator to automatically secure your ingress routes with custom certificates.
• Set up a self-signed CA and manage the entire certificate lifecycle using cert-manager resources.

Create a self-signed CA issuer

The initial step involves configuring either an Issuer or ClusterIssuer. These entities function as resources that represent certificate authorities responsible for signing certificates when they receive signing requests. Issuer is specific to a namespace while ClusterIssuer operates at the cluster level.

Here we'll create a SelfSigned ClusterIssuer, issue a root certificate, then use that root as a CA issuer:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-cluster-issuer #----(1)
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-root-ca
  namespace: cert-manager
spec:
  isCA: true
  commonName: selfsigned-root-ca # Note this
  secretName: root-ca-key-pair #----(2)
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: selfsigned-cluster-issuer # match (1)
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: ca-issuer
spec:
  ca:
    secretName: root-ca-key-pair # match (2)

Feel free to modify the resource names, and take note of the commonName for verification purposes. The root CA will be stored in the root-ca-key-pair secret within the cert-manager namespace by the cert-manager operator.

Issue certificate

cert-manager's Certificate resource facilitates automatic issuance, renewal, and storage of signed certificates, contingent on configured Issuer or ClusterIssuer resources. Next, we will create a Certificate using the CA ClusterIssuer you created above, for the default ingress controller in openshift-ingress namespace.

You can use the oc whoami --show-server command to get the correct cluster-domain:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: ingress-wildcard-cert
  namespace: openshift-ingress
spec:
  isCA: false
  commonName: "*.apps.<cluster-domain>" # change me
  dnsNames:
    - "apps.<cluster-domain>"           # change me
    - "*.apps.<cluster-domain>"         # change me
  usages:
    - server auth
  issuerRef:
    kind: ClusterIssuer
    name: ca-issuer
  secretName: ingress-wildcard-tls

The relevant certificate will be created and stored in the ingress-wildcard-tls secret. This is the certificate which we all care about. 

Change the ingress controller's default certificate

Next, we'll update the defaultCertificate reference in the default IngressController CR by patching it to point to the new certificate secret:

$ oc patch ingresscontroller.operator default \
--type=merge -p \
'{"spec":{"defaultCertificate": {"name": "ingress-wildcard-tls"}}}' \
-n openshift-ingress-operator

Once the certificate is replaced, all applications, the web console, and CLI will be secured with the specified certificate for encryption.

Replace the CA bundle certificate

OpenShift's Proxy certificates enable custom CAs for secure egress connections, through the trustedCA field and a dedicated proxy validator in the openshift-config-managed namespace. You can refer to the documentation for understanding CA bundle certificates. This step is only necessary when using a custom CA to manage the trusted certs inside OpenShift. Internet CAs like the certs issued online can skip this step.

Now, let’s do the replacement.

1. Extract the ca.crt from the ingress-wildcard-tls secret:

   $ oc extract secret/ingress-wildcard-tls -n openshift-ingress

2. Generate a ConfigMap to encapsulate the root CA certificate utilized for signing the wildcard certificate:

   $ oc create configmap issued-ca-bundle \
   --from-file=ca-bundle.crt=./ca.crt \
   -n openshift-config

3. Use the created ConfigMap and update the proxy configuration across the entire cluster:

   $ oc patch proxy cluster \
   --type=merge \
   --patch='{"spec":{"trustedCA":{"name":"issued-ca-bundle"}}}'

   Wait a bit for the changes to propagate through your cluster, and ensure its health using fundamental commands like:

   $ oc get nodes
   $ oc get pods --all-namespaces
   $ oc get clusteroperators

Fantastic! Your cluster is now fully configured to fortify all connections with the bespoke custom certificate.

See it in action

Now it's time to verify if your connections are indeed being served by the certificates.

$ echo Q | openssl s_client -connect console-openshift-console.apps.<cluster-domain>:443 -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -enddate

Sample result:

subject=CN = *.apps.<cluster-domain>  # <- check this
issuer=CN = selfsigned-root-ca        # <- check this
notAfter=Apr  2 13:15:59 2024 GMT

$ curl -v --cacert ./ca.crt https://console-openshift-console.apps.<cluster-domain>

Sample result:

*   Trying 20.111.222.333:443...
* Connected to console-openshift-console.apps.<cluster-domain> (20.111.222.333) port 443 (#0)
...
*  CAfile: ./ca.crt
...
* Server certificate:
*  subject: CN=*.apps.<cluster-domain>
*  start date: Jan  3 13:15:59 2024 GMT
*  expire date: Apr  2 13:15:59 2024 GMT
...
*  issuer: CN=selfsigned-root-ca
*  SSL certificate verify ok.      # <- check this
...

To verify SSL authentication, cross-check both the CN (common name) and certificate start and expiration details. Ensure alignment with expectations, and rest assured, cert-manager will handle automatic certificate renewal.

Wrap up

In this learning path, we walked through the intricacies of managing certificates on OpenShift, emphasizing its compatibility across diverse architectures with the enhanced support of the cert-manager Operator in version 1.13.0.

From deploying an OpenShift cluster on IBM Power Virtual Server (VS) to installing the cert-manager Operator, we explored a seamless end-to-end process. The practical side covered creating a self-signed CA issuer, issuing a root certificate, and utilizing cert-manager for managing certificates in the default ingress controller for secure applications.

Awesome. I hope you found this guide both informative and actionable. Keep learning. And as promised, here are a few useful links for further reference:

• cert-manager Operator for Red Hat OpenShift overview
• cert-manager Documentation
• Preparing to install on IBM Z and IBM LinuxONE
• Preparing to install on IBM Power Virtual Server
• Installation overview | Installing | OpenShift Container Platform 4.14
• Understanding the Ingress Operator | Networking | OpenShift Container Platform 4.14