How to configure persistent storage with OpenShift or Kubernetes for development environment
- We know that containers in Openshift or Kubernetes don’t persist data. Every time we start an application, it is started in a new container with an immutable Docker image.
Hence, any persisted data in the file systems is lost when the container stops. Hence if an application or container is rebuilt or restarted than we can’t view previous logs or if we are using containers with mysql or any other database then schema, tables, and all data will be lost, if using any messaging broker than if there is journal file than it will also not persist.
Hence, these ephemeral containers cannot be used in production environment. In a production environment, we must configure a shared storage.
- But what about the development environment, because we might not always have enough labs and VM’s available. To rescue we have volume type hostPath, which can be easily set up with Minishift and Minikube.
- This article will provide details how to setup hostPath volume type.
1. PersistentVolumes: We have to configure PersistentVolume. We can create a pv.yaml file like:
apiVersion: v1 kind: PersistentVolume metadata: name: pv0002 spec: accessModes: - ReadWriteOnce capacity: storage: 500Mi hostPath: path: /data/pv0002/
- Above we have created pv0002 Persistent Volume. With a specification like accessModes, capacity, and hostPath.
- In OpenShift we can easily create PersistentVolume with the command,
oc create -f pv.yaml
- To view all configured persistent volume, run the command below,
oc get pv
2. Check hostpath privileges: It needs elevated privileges to read and write from hostpath /data/pv0002. So we would have to first ssh to the host machine (which is a minishift virtual machine) then set the privileges so that all users can read/write.
minishift ssh docker@minishift:~$ id uid=1000(docker) gid=50(staff) groups=50(staff),100(docker) docker@minishift:~$ pwd /home/docker docker@minishift:~$ cd /data docker@minishift:~$ sudo chmod -R 777 pv0002 docker@minishift:/mnt/sda1/data$ ls -ltr|grep pv0002 drwxrwxrwx 5 root root 4096 Oct 18 07:41 pv0002/ docker@minishift:/mnt/sda1/data$ pwd /data
3. PersistentVolumeClaims: To access this storage from Projects, PersistentVolumeClaims must be created that can access the PersistentVolume. PersistentVolumeClaims are created for each Project with customized claims for a certain amount of storage with certain access modes. We can create another yaml (mysql-deployment.yaml) file with below configuration. Using below configuration, we can have a mysql container, which would also persist the data. This is a reference from this link.
apiVersion: v1 kind: Service metadata: name: mysql spec: ports: - port: 3306 selector: app: mysql clusterIP: None --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: mysql-pv-claim spec: accessModes: - ReadWriteOnce resources: requests: storage: 300Mi --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: mysql spec: selector: matchLabels: app: mysql strategy: type: Recreate template: metadata: labels: app: mysql spec: containers: - image: mysql:5.6 name: mysql env: # Use secret in real usage - name: MYSQL_ROOT_PASSWORD value: password ports: - containerPort: 3306 name: mysql volumeMounts: - name: mysql-persistent-storage mountPath: /var/lib/mysql volumes: - name: mysql-persistent-storage persistentVolumeClaim: claimName: mysql-pv-claim
Points to note above:
- We are creating a service mysql.
- We are creating PersistentVolumeClaim mysql-pv-claim, which should bind to volume pv0002.
- We are creating a Deployment mysql, with the environment variable MYSQL_ROOT_PASSWORD and mountPath /var/lib/mysql which mounts to persistentVolumeClaim mysql-pv-claim. We know persistentVolumeClaim is referring to /data/pv0002/, we have defined this path in persistent volume pv0002.
4. Deploy the yaml content: We can run this yaml as
oc create -f mysql-deployment.yaml
5. Check if pv and pvc are bound: We can get details of persistentvolume and persistentvolumeclaim as
[cpandey@cpandey minishit_persistence_host]$ oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES AGE mysql-pv-claim Bound pv0002 500Mi RWO 1d [cpandey@cpandey minishit_persistence_host]$ oc get pv NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE pv0002 500Mi RWO Retain Bound myproject/mysql-pv-claim 1d [cpandey@cpandey minishit_persistence_host]$ oc get service NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql None none 3306/TCP 1d
- Above we find that pv0002 is bound and claimed by mysql-pv-claim.
- Also, we see that mysql service is up and running on port 3306 port.
6. Now if we again ssh to minishift. We will find files created within /data/pv0002 hostpath.
docker@minishift:~$ cd /data/pv0002 docker@minishift:/mnt/sda1/data/pv0002$ ls -ltr total 110608 -rw-rw---- 1 10000400 root 50331648 Oct 17 03:51 ib_logfile1 drwx------ 2 10000400 root 4096 Oct 17 03:51 performance_schema/ -rw-rw---- 1 10000400 root 56 Oct 17 03:51 auto.cnf drwx------ 2 10000400 root 4096 Oct 17 03:51 mysql/ drwx------ 2 10000400 root 4096 Oct 17 03:55 csp/ -rw-rw---- 1 10000400 root 12582912 Oct 18 07:41 ibdata1 -rw-rw---- 1 10000400 root 50331648 Oct 18 07:42 ib_logfile0
7. Access mysql pod: Below we can access mysql pod. Here note that database csp and table testtable was created manually on the first run.
[cpandey@cpandey minishit_persistence_host]$ oc get pods NAME READY STATUS RESTARTS AGE mysql-63082529-66jyn 1/1 Running 1 1d [cpandey@cpandey minishit_persistence_host]$ oc rsh mysql-63082529-66jyn $ mysql -u root -ppassword mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | csp | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.01 sec) mysql> use csp; Database changed mysql> show tables; +---------------+ | Tables_in_csp | +---------------+ | testtable | +---------------+ 1 row in set (0.00 sec) mysql> select * from testtable; Empty set (0.00 sec)
8. Verify if data persists: Now let us delete the current running pod and check if mysql db and tables still exist.
[cpandey@cpandey minishit_persistence_host]$ oc delete pod mysql-63082529-66jyn pod "mysql-63082529-66jyn" deleted [cpandey@cpandey minishit_persistence_host]$ oc get pods NAME READY STATUS RESTARTS AGE mysql-63082529-66jyn 1/1 Terminating 1 1d mysql-63082529-q7e5n 1/1 Running 0 4s [cpandey@cpandey minishit_persistence_host]$ oc get pods NAME READY STATUS RESTARTS AGE mysql-63082529-q7e5n 1/1 Running 0 14s [cpandey@cpandey minishit_persistence_host]$ oc rsh mysql-63082529-q7e5n $ mysql -u root -ppassword mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | csp | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.00 sec) mysql> use csp; mysql> show tables; +---------------+ | Tables_in_csp | +---------------+ | testtable | +---------------+ 1 row in set (0.00 sec)
This way we can easily configure persistence even in our development environment, which is mostly minishift or minikube.
Join the Red Hat Developer Program (it’s free) and get access to related cheat sheets, books, and product downloads.
To learn more, visit our Join the Red Hat Developer Program (it’s free) and get access to related cheat sheets (e.g. containers), books (e.g. microservices), and product downloads that can help you with your microservices and/or container application development.
Download and learn more about Red Hat JBoss Fuse, an innovative modular, cloud-ready architecture, powerful management and automation, and world class developer productivity. It is Java™ EE 7 certified and features powerful, enterprise-grade features such as high availability clustering, distributed caching, messaging, transactions, and a full web services stack.