Page
Verify with Multus and deploy the storage cluster
Before creating the storage cluster, run the multus validation tool to verify the network configuration. This interactive tool provided by Red Hat will run a series of validation tests that determine whether the current Network Attachment Definitions (NAD) and system configurations will support ODF using multus. The tool can be downloaded from the Red Hat Knowledgebase, which also contains installation instructions.
Prerequisites:
- NNCP added
- NAD applied
- DHCP configured for the cluster
In this lesson, you will:
- Run the Multus tool to verify the network configuration.
- Deploy the storage cluster.
Run a validation test
To run the validation test, pass the name of the cluster and public storage networks. Ensure you are logged into the OpenShift cluster with administrator privileges prior to running the command.
./rook multus validation run --cluster-network=odf-cluster-network --public-network=odf-public-network
2025/05/05 16:51:36 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
2025-05-05 16:51:36.042904 I | rookcmd: created kube client interface from default CLI parameters
2025-05-05 16:51:36.043103 I | multus-validation: starting multus validation test with the following config:
namespace: openshift-storage
serviceAccountName: rook-ceph-system
publicNetwork: odf-public-network
clusterNetwork: odf-cluster-network
resourceTimeout: 3m0s
flakyThreshold: 30s
hostCheckOnly: false
nginxImage: quay.io/nginx/nginx-unprivileged:stable-alpine
nodeTypes:
shared-storage-and-worker-nodes:
osdsPerNode: 19
otherDaemonsPerNode: 0
placement:
nodeSelector: {}
tolerations: []
2025-05-05 16:51:36.085529 I | multus-validation: continuing: expected number of image pull pods not yet ready: image pull daemonset for node type "shared-storage-and-worker-nodes" expects zero scheduled pods
…
2025-05-05 16:54:25.192015 I | multus-validation: all 57 clients are 'Ready'
RESULT: multus validation test succeeded!
cleaning up multus validation test resources in namespace "openshift-storage"
multus validation test resources were successfully cleaned upThe tool will verify the connectivity across the nodes, but does not complete any performance or load testing. If the tool indicates a failure, reference the KCS article for troubleshooting recommendations. View the pod logs from the tool to view potential issues. Once debugging is complete, clean up the running instances by executing the tool with the cleanup option.
./rook multus validation cleanup --namespace openshift-storageStorageCluster Custom Resource Definition
The storage cluster Custom Resource Definition (CRD) defines and manages the Ceph storage cluster. The storage cluster will need to be deployed through the command-line interface (CLI) so that it can include the IP address ranges managed by the DHCP server for the public and cluster networks. If the .spec.network.addressRanges are not included in the CRD, the Ceph pods will not be able to start correctly. Make sure that all ranges from all data centers/zones for the public and cluster networks that are managed by the DHCP servers are included.
apiVersion: ocs.openshift.io/v1
kind: StorageCluster
metadata:
annotations:
cluster.ocs.openshift.io/local-devices: 'true'
name: ocs-storagecluster
namespace: openshift-storage
spec:
storageDeviceSets:
- count: 3
dataPVCTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: '1'
storageClassName: odf-osd-localvolumeset
volumeMode: Block
name: ocs-deviceset-odf-osd-localvolumeset
replica: 3
monDataDirHostPath: /var/lib/rook
network:
addressRanges:
cluster:
- 172.10.11.0/24
- 172.10.75.0/24
- 172.10.139.0/24
public:
- 172.10.10.0/24
- 172.10.74.0/24
- 172.10.138.0/24
connections:
encryption:
enabled: true
provider: multus
selectors:
cluster: openshift-storage/odf-cluster-network
public: openshift-storage/odf-public-network
resourceProfile: performance
flexibleScaling: falseApply the StorageCluster configuration using the oc command.
oc apply -f storage-cluster.yamlGet the status of the storage cluster.
oc get storagecluster -n openshift-storage
NAME AGE PHASE EXTERNAL CREATED AT VERSION
ocs-storagecluster 10m Ready 2025-05-08T15:18:59Z 4.18.2Verify the isolated storage networks.
oc get storagecluster ocs-storagecluster -n openshift-storage -o=jsonpath='{.spec.network}{"\n"}' | jq
{
"addressRanges": {
"cluster": [
"172.21.136.0/24",
"172.21.200.0/24",
"172.21.72.0/24"
],
"public": [
"172.21.135.0/24",
"172.21.199.0/24",
"172.21.71.0/24"
]
},
"connections": {
"encryption": {
"enabled": true
}
},
"multiClusterService": {},
"provider": "multus",
"selectors": {
"cluster": "openshift-storage/odf-cluster-network",
"public": "openshift-storage/odf-public-network"
}
}Summary
We have walked through deploying an ODF Storage Cluster using Multus for dedicated networks across availability zones, which improves performance, availability, and security. With careful planning, this approach can be scaled to your organization's infrastructure to build highly available and performant solutions that can withstand infrastructure failures and meet demanding application requirements.
Ready to learn more about Kubernetes and OpenShift? Try these learning paths:
