Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

A developer’s guide to Kubernetes storage concepts

October 6, 2022
Fernando Lozano
Related topics:
CI/CDKubernetes
Related products:
Red Hat OpenShift Container Platform

Share:

    This second article continues a series discussing Kubernetes storage concepts. I will define the concepts of volumes, persistent volume claims, and storage classes, and why they should matter to a developer. I will also explain how persistent volumes and storage provisioners enable system administrators to manage storage for a Kubernetes cluster while offering developers self-service to storage. You will also discover the special abilities of stateful sets.

    Follow the series:

    • Part 1:  How Kubernetes improves developer agility

    The first part of this series explains why storage is essential, even for developers developing "stateless" applications based on microservices architectures. I also explain the role of Kubernetes in managing and providing storage volumes for applications running as pods.

    • Part 3:  Why CSI drivers are essential in Kubernetes storage

    I describe the unique needs of containerized applications. You will discover how CSI drivers enable advanced storage features necessary for production environments and are useful for CI/CD pipelines.

    3 concepts of Kubernetes storage for developers:

    There are three main concepts of Kubernetes storage that developers must know to facilitate collaboration with system admins and to improve connections between applications and Kubernetes storage services:

    1. Which volume types to use and which to avoid

    Storage in Kubernetes originally started with just a volume attribute inside a pod resource. Multiple containers from a pod can mount the same volume, possibly in different paths, because each container has its own isolated file system.

    Kubernetes originally defined many volume types that connect to different storage services and protocols, such as AWS EBS or FibreChannel LUNs (Figure 1). It would be best if you avoided most volume types because they embed environmental data, such as an iSCSI initiator address, which is not very flexible for system administrators. You do not want to change your manifests because one environment uses OpenStack cinder disks, another uses NFS, or the NFS server address differs between data centers.

     

    Direct references to various types of persistent storage from a pod are obsolete.
    Figure 1: Direct references to various types of persistent storage from a pod are obsolete.

     

    Some volume types, such as emptyDir and downwardAPI, have valid uses. You can also use volumes for run-time access to configuration data from ConfigMaps and secrets. But for persistent data stored as disk files, you should focus on the persistent volume claim (PVC) volume type as the general-purpose mechanism to configure storage for an application.

    2. PVC: Best volume type for persistent data storage

    A persistent volume claim (PVC) resource represents a request for storage by an application. A persistent volume claim specifies desired storage characteristics, such as capacity and access mode. It does not specify a storage technology such as iSCSI or NFS. With the persistent volume claim volume type, the only configuration setting is the name of a persistent volume claim resource in the same namespace as your pod.

    The lifecycle of a PVC is tied not to a pod but a namespace. Multiple pods from the same namespace but potentially different workload controllers can connect to the same PVC. You can also sequentially connect and detach storage to different application pods as you need to initialize, convert, migrate, or back up data.

    Kubernetes matches each persistent volume claim to a persistent volume (PV) resource that can satisfy the requirements of the claim (Figure 2). It is not an exact match. A PVC could be bound to a PV with a larger disk size than requested, and a PVC that specifies single access could be bound to a PV that is shareable for multiple concurrent accesses. PVCs do not enforce policy but declare what an application needs, which Kubernetes provides on a best-effort basis.

     

    A PVC requests a persistent volume, which can be configured to use various types of physical storage.
    Figure 2: A PVC requests a persistent volume, which can be configured to use various types of physical storage.

     

    Even if your PVC specifies a size, storage space might not be guaranteed or reserved for your application. The result depends on the underlying storage technology. For example, the same shared volume in a file server backs multiple volumes, or thin volumes could back them from a Ceph RBD server.

    In the old days of Kubernetes, a cluster administrator created a set of PV resources in advance as a pool of volumes available for application requests. From time to time, the administrator must create more space and find and clean up unused volumes. That process was called manually provisioned storage.

    Each manually provisioned PV specifies all the parameters required for access to a storage service or server: AWS IAM credentials for EBS volumes or an NFS server address and share path. This practice is good for separating concerns because the cluster administrator decides which types of storage are available and isolates developers from all connection parameters required for access to each storage service. However, manual provisioning places a large workload on system administrators.

    3. Storage classes

    PVs and PVCs improved over the initial feature of volume types from Kubernetes but still required a lot of labor from IT. So Kubernetes incorporated dynamic volume provisioners, which create persistent volumes on demand to match persistent volume claims.

    Under this system, PVCs do not specify provisioned storage but instead refer to a storage class (Figure 3). Cluster administrators create storage classes as required to manage different storage services or different storage tiers of the same service. Each storage class specifies a storage provisioner and the connection parameters the provisioner accepts.

     

    The storage provisioner matches a PVC to appropriate physical storage through a storage class.
    Figure 3: The storage provisioner matches a PVC to appropriate physical storage through a storage class.

     

    For example, one storage class might connect to an NFS server and another to a Fibre Channel Storage Area Network (SAN). By specifying the corresponding class, PVCs connect to the desired storage service.

    In another example, one storage class might specify the Azure Standard HDD disk type, and another might specify the Azure Premium SSD disk type. By choosing between these classes, a developer picks either cheaper or higher-performance storage.

    A dynamic storage provisioner connects to the storage service to perform any task that might be required to make storage available. The task could require creating a new LUN that matches the size of a PVC or just mounting a shared folder.

    Dynamic storage provisioners make storage a self-service in Kubernetes, alleviating the workload on system administrators and enabling agility for developers.

    As a developer, you might not require access to any network storage service during development. Your local Kubernetes cluster might use a minimal storage provisioner that uses host folders or disk partitions. So you can test your application using the same PVC resources and pod manifests as the production environment. Or your local Kubernetes cluster might come preconfigured with a set of manually provisioned PVs.

    Persistent volumes vs. stateful set workload controllers

    A common misconception among developers is that you must use stateful set workload controllers to manage pods that require persistent storage. Other workload controllers, such as a job or a deployment, can manage pods that mount volumes from PVCs.

    For example, when running a single database pod, it is perfectly fine to use a deployment resource to manage a relational database pod. If you need to manage a relational database running as multiple pods, you probably require more than the capabilities of a Kubernetes stateful set alone. Do not underestimate the complexity of such configurations.

    Kubernetes stateful sets alone do not make any application magically capable of using high-availability and scalability features such as replication, sharding, and distributed transactions. The application has to be designed and configured for such features.

    A few application needs might match stateful set features, such as the Cassandra example from Kubernetes documentation. Unfortunately, most databases require fine-tuned settings for each instance or pod, and stateful sets do not provide a clean way of feeding these per-instance settings.

    You can craft startup scripts that configure databases to work as part of a stateful set, such as in the replicated MySQL example from Kubernetes documentation; it isn't trivial. Instead of creating those snowflakes, you might consider using a specialized controller for your database workload, such as the CrunchyData PostgreSQL operator.

    So what is the difference between a stateful set and other Kubernetes workload controllers regarding persistent storage? The difference is that a stateful set allows each pod to use separate storage.

    Most workload controllers include only a pod template, which refers to one or more PVCs. The workload controller does not manage these PVCs. If you create multiple pods from the same template, as in a deployment with more than one replica, these pods refer to the same set of PVCs. They are all sharing the same volumes and their files.

    Unlike other workload resources, stateful sets include an array of PVC templates in addition to a pod template. Scaling a stateful set creates a set of PVCs for each pod (Figure 4). Each pod gets its dedicated set of volumes and doesn't share files with other pods from the same stateful set—at least not from the volumes managed by the stateful set. You can still refer to PVCs not managed by a stateful set.

     

    A stateful set creates a PVC template for each pod, unlike other workload controllers.
    Figure 4: A stateful set creates a PVC template for each pod, unlike other workload controllers.

    Kubernetes facilitates developer and system administrator collaboration

    The main concepts that a developer needs to know about storage in Kubernetes are volumes in pods, persistent volume claims, and storage classes. Persistent volumes and storage provisioners are the domain of system administrators. But knowledgeable developers can collaborate more effectively with administrators to ensure that applications work as designed under load. Developers can also plan how to handle disaster recovery and data sharing between applications.

    The final article of this series, Why CSI drivers are essential in Kubernetes storage, completes the presentation of essential Kubernetes storage concepts by introducing CSI drivers. The article emphasizes the need for storage products designed for Kubernetes versus storage designed for traditional physical and virtual data centers or Infrastructure-as-a-Service (IaaS) clouds.

    Thanks a lot to Andy Arnold and Greg Deffenbau for their reviews of this article.

    Last updated: October 31, 2023

    Related Posts

    • Persistent storage in action: Understanding Red Hat OpenShift's persistent volume framework

    • How to maximize data storage for microservices and Kubernetes, Part 1: An introduction

    • Managing persistent volume access in Kubernetes

    Recent Posts

    • Storage considerations for OpenShift Virtualization

    • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

    • EE Builder with Ansible Automation Platform on OpenShift

    • How to debug confidential containers securely

    • Announcing self-service access to Red Hat Enterprise Linux for Business Developers

    What’s up next?

    Learn and experiment with Kubernetes using the no-cost Developer Sandbox for Red Hat OpenShift! We created a tutorial that walks developers through how to create an application using Kubernetes step-by-step, making the lives of developers easier by making Kubernetes development simple, fast, and fun.

    Learn by doing
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue