Enhance data security in OpenShift Data Foundation

In our previous article on Red Hat OpenShift Data Foundation (ODF), we demonstrated how to configure cluster-wide encryption at rest using HashiCorp Vault as the Key Management System (KMS). 

In this installment, we will take that foundation further. We’ll explore how to enable encryption on a per-PersistentVolume (PV) basis, allowing teams to isolate their encrypted storage volumes at the namespace level. This is especially valuable in multi-tenant environments where different projects or teams require distinct encryption boundaries.

Per-PV StorageClass encryption

While cluster-wide encryption protects all data at rest, it's often insufficient for environments with stricter security demands, such as:

• Multi-tenant Red Hat OpenShift clusters, where different teams or customers share infrastructure.
• Compliance requirements that mandate different encryption keys per project, tenant, or sensitivity level.

Prerequisites

Before diving into per-PV (StorageClass) encryption, you should already have the following:

• ODF installed and running on your OpenShift cluster.
• Vault deployed and integrated with ODF for cluster-wide encryption (see our previous article).
• A basic understanding of Kubernetes StorageClasses and PersistentVolumeClaims.

Steps to enable StorageClass encryption

We will walk through these four steps:

1. Create a new Vault role and Access Control List (ACL) for the tenant.

2. Create an overriding ConfigMap in the tenant namespace.

3. Create the tenant service account and assign permissions.

4. Create PersistentVolumeClaims (PVCs) in the tenant namespace using the new StorageClass.

The installation process remains largely the same as previously. However, when you reach the Security and network step in the ODF installation wizard, make sure to enable StorageClass encryption (Figure 1).

Enabling this option creates an additional StorageClass on the cluster, which we've set as the default StorageClass going forward:

$ oc get sc
NAME                                              PROVISIONER                             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
gp2-csi                                           ebs.csi.aws.com                         Delete          WaitForFirstConsumer   true                   4h22m
gp3-csi                                           ebs.csi.aws.com                         Delete          WaitForFirstConsumer   true                   4h22m
localblock                                        kubernetes.io/no-provisioner            Delete          WaitForFirstConsumer   false                  35m
ocs-storagecluster-ceph-rbd                       openshift-storage.rbd.csi.ceph.com      Delete          Immediate              true                   2m30s
ocs-storagecluster-ceph-rbd-encrypted (default)   openshift-storage.rbd.csi.ceph.com      Delete          Immediate              true                   2m30s
ocs-storagecluster-cephfs                         openshift-storage.cephfs.csi.ceph.com   Delete          Immediate              true                   2m49s

1. Create a new Vault role and ACL for the tenant

Each tenant (or namespace) can have its own Vault path (e.g., kv/tenant-a/). However, for simplicity, we've continued to use the odf backend path defined previously:

oc -n vault exec pods/vault-0 -- \
    vault write auth/kubernetes/role/csi-kubernetes \
    bound_service_account_names=ceph-csi-vault-sa \
    bound_service_account_namespaces='*' \
    policies=odf \
    ttl=5m

The previously defined Vault ACL grants full access (CRUDL) to all secrets under the odf/ path (which again, we're reusing here). If you haven't yet applied the Vault policy for ODF from the previous article, here it is again:

oc -n vault exec -i pods/vault-0 -- vault policy write odf - <<EOF
path "odf/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
path "sys/mounts" {
  capabilities = ["read"]
}
EOF

2. Create an overriding ConfigMap in the tenant namespace

We'll now create a new ConfigMap that mirrors some values from the original csi-kms-connection-details ConfigMap (in openshift-storage).

Let's create a test tenant namespace:

oc new-project tenant-a

Now apply the overriding ConfigMap:

oc apply -f - <<EOF 
kind: ConfigMap
apiVersion: v1
metadata:
  name: csi-kms-connection-details
  namespace: tenant-a
data:
  vault-tenant-sa: |-
    {
      "encryptionKMSType": "vaulttenantsa",
      "vaultAddress": "https://vault.vault.svc:8200",
      "vaultTLSServerName": "",
      "vaultAuthPath": "kubernetes",
      "vaultBackendPath": "odf",
      "vaultCAVerify": "true",
      "tenantSAName": "ceph-csi-vault-sa"
    }
EOF

3. Create the tenant service account and assign permissions

Create the service account:

oc create sa ceph-csi-vault-sa -n tenant-a

Grant it the necessary permissions via the auth-delegator role:

oc adm policy add-role-to-user system:auth-delegator system:serviceaccount:tenant-a:ceph-csi-vault-sa

Note that in ODF, the default service account name used for Vault integration in tenant namespaces is ceph-csi-vault-sa.

4. Create PVCs in the tenant namespace using the new StorageClass

Developers (or tenants) can now request encrypted volumes using the StorageClass configured for Vault:

oc apply -f - <<EOF 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: encrypted-pvc
  namespace: tenant-a
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: ocs-storagecluster-ceph-rbd-encrypted
  volumeMode: Block
EOF

The PVC expectedly goes into a Bound state:

$ oc get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                            VOLUMEATTRIBUTESCLASS   AGE
encrypted-pvc   Bound    pvc-b73d1006-5249-4091-9de5-3637c92b069e   5Gi        RWO            ocs-storagecluster-ceph-rbd-encrypted   <unset>                 3s

We can now see that an additional Vault key has been created under the odf backend path with the prefix openshift-storage:

$  oc -n vault exec vault-0 -- vault kv list odf
Keys
----
0001-0011-openshift-storage-0000000000000004-6d243b27-0eee-4f15-8a81-7219cb958923
noobaa-root-master-key-backend
rook-ceph-osd-encryption-key-ocs-deviceset-localblock-0-data-0whn2q
rook-ceph-osd-encryption-key-ocs-deviceset-localblock-1-data-0twdbr
rook-ceph-osd-encryption-key-ocs-deviceset-localblock-2-data-07ckr9

Wrap up

Per-namespace encryption with OpenShift Data Foundation and Vault unlocks powerful isolation and security controls for OpenShift clusters hosting sensitive or multi-tenant workloads.

While cluster-wide encryption provides a strong baseline, the double encryption capability of ODF—applying encryption at the PV level—enables greater control and adherence to more rigorous compliance standards if that is relevant to your organization.