Use Persistent Volumes and Persistent Volume Claims with Oracle Cloud Native Environment

Introduction

Because container-based applications are stateless by default, any changes to files during the container lifetime are lost. However, if your application is stateful, the ablility to persist any changes to the filesystem becomes relevant. It is this requirement that has led to Kubernetes supporting many types of volumes. Many volume types that you may have encountered already are ephemeral (they only exist during the pod lifetime). Kubernetes also supports persistent volumes, which persist data stored on them beyond the pod lifetime. There are several Persistent Volume types supported by Kubernetes, and this Lab will demonstrate one of the simplest - the local type, which stores data on devices stored locally on one of the cluster nodes.

This tutorial shows how to create and use Persistent Volumes and Persistent Volume Claims with Oracle Cloud Native Environment. Both Persistent Volumes and Persistent Volume Claims work together to provide persistence to any container-based applications deployed onto Oracle Cloud Native Environment. You will start by covering how to use Persistent Volumes initially, and then cover the use of Persistent Volume Claims.

Objectives

In this lab, you will learn:

The difference between a Persistent Volume (PV) and a Persistent Volume Claim (PVC)
How to use Persistent Volumes and Persistent Volume Claims with Oracle Cloud Native Environment.

Prerequisites

Minimum of a 3-node Oracle Cloud Native Environment cluster:
- Operator node
- Kubernetes control plane node
- Kubernetes worker node
Each system should have Oracle Linux installed and configured with:
- An Oracle user account (used during the installation) with sudo access
- Key-based SSH, also known as password-less SSH, between the hosts
- Installation of Oracle Cloud Native Environment and Oracle Cloud Infrastructure Cloud Controller Manager (oci-ccm) module

Deploy Oracle Cloud Native Environment

Note: If running in your own tenancy, read the linux-virt-labs GitHub project README.md and complete the prerequisites before deploying the lab environment.

Open a terminal on the Luna Desktop.

Clone the linux-virt-labs GitHub project.

git clone https://github.com/oracle-devrel/linux-virt-labs.git

Change into the working directory.
```
cd linux-virt-labs/ocne
```

Install the required collections.

ansible-galaxy collection install -r requirements.yml

Deploy the lab environment.
```
ansible-playbook create_instance.yml -e localhost_python_interpreter="/usr/bin/python3.6" -e use_oci_ccm=true
```
The free lab environment requires the extra variable local_python_interpreter, which sets ansible_python_interpreter for plays running on localhost. This variable is needed because the environment installs the RPM package for the Oracle Cloud Infrastructure SDK for Python, located under the python3.6 modules.
Important: Wait for the playbook to run successfully and reach the pause task. At this stage of the playbook, the installation of Oracle Cloud Native Environment is complete, and the instances are ready. Take note of the previous play, which prints the public and private IP addresses of the nodes it deploys and any other deployment information needed while running the lab.

Confirm the Storage Class Used on your Cluster

There are different storage classes available in different environments. The free lab environment is using Oracle Cloud Storage and has been pre-configured to work with oci-ccm (The Oracle Cloud Infrastructure Cloud Controller Manager ).

Open a terminal and connect via ssh to the ocne-control-01 node.
```
ssh oracle@<ip_address_of_node>
```

Confirm which StorageClass is available in the free lab environment.

kubectl get storageclass

Example Output:

NAME               PROVISIONER                       RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
oci                oracle.com/oci                    Delete          Immediate              false                  6m
oci-bv (default)   blockvolume.csi.oraclecloud.com   Delete          WaitForFirstConsumer   true                   6m
oci-bv-encrypted   blockvolume.csi.oraclecloud.com   Delete          WaitForFirstConsumer   true                   6m

Note: If you are using your own Kubernetes cluster, it is essential to confirm what Storage Classes are available and then modify the value in your Persistent Volume/Persistent Volume Claim for them to work. The free lab uses the default value of oci-bv.

Create a Persistent Volume

Enabling a container to persist its data between sessions is a two-stage process. Firstly the disk resource has to be defined (the Persistent Volume), then it needs to be assigned to the container (Persistent Volume Claim). This section will complete the first of these.

Create the Persistent Volume Claim YAML file.
```
cat << 'EOF' > ~/pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-pv
spec:
  storageClassName: oci-bv
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/tmp/test-pv"
EOF
```
The principal fields to note are:
- accessModes: the value used here (ReadWriteOnce) means the volume can be mounted as ReadWrite by only one Pod, whilst being mounted by any other Pods on the same node in Read mode only (for more information look here ).
- storage: 1Gi defines the size of the Persistent Volume. This example is 1Gb
- The storageClassName variable is set to oci-bv
- hostPath defines the 'type' of Persistent Volume. The hostPath value used here will ONLY work on a single node, making it only suitable for testing (more information is available here ). The hostPath value allows the Persistent Volume to emulate networked storage. In this example, the hostPath variable configures the storage class to store the Persistent Volume data at /tmp/test-pv. (Note: If you are using a different storage driver you may need to use a different provisioner from hostPath.)
Deploy the Persistent Volume.
```
kubectl apply -f pv.yaml
```

Confirm the Persistent Volume deployed.

kubectl get pv

Example Output:

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
test-pv   1Gi        RWO            Retain           Available           oci-bv                  2m2s

(Optional) You can also look at more detail about the Persistent Volume.

kubectl describe pv/test-pv

Example Output:

Name:            test-pv
Labels:          <none>
Annotations:     <none>
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    oci-bv
Status:          Available
Claim:           
Reclaim Policy:  Retain
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        1Gi
Node Affinity:   <none>
Message:         
Source:
    Type:          HostPath (bare host directory volume)
   Path:          /tmp/test-pv
    HostPathType:  
Events:            <none>

Interesting Information: This illustrates how the fields defined in the YAML file map to the Persistent Volume. Did you spot some unused fields, such as Node Affinity ? They are used to test functionality if your install has multiple Worker nodes, which is out of scope for this lab.

Create a Persistent Volume Claim

A Persistent Volume cannot be used in isolation. Instead, a Persistent Volume Claim must be defined and created. Next, you will complete the second of the two steps.

Create the Persistent Volume Claim YAML file.

The Persistent Volume Claim enables the deployed application to request physical storage.

cat << 'EOF' > ~/pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
spec:
  storageClassName: oci-bv
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
EOF

Create the Persistent Volume Claim.
```
kubectl apply -f pvc.yaml
```
This command instructs Kubernetes to look for a Persistent Volume that matches the PVC claims requirements. Kubernetes looks to match the storageClass type requested and, if located, binds this claim to the identified volume.

Confirm the Persistent Volume Claim created.

kubectl get pvc/test-pvc

Example Output:

NAME       STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-pvc   Pending                                      oci-bv         1m43s

(Optional) Retrieve more detail related to the Persistent Volume Claim.

kubectl describe pvc/test-pvc

Example Output:

Name:          test-pvc
Namespace:     default
StorageClass:  oci-bv
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                Age                   From                         Message
  ----    ------                ----                  ----                         -------
 Normal  WaitForFirstConsumer  11s (x22 over 5m22s)  persistentvolume-controller  waiting for first consumer to be created before binding

All that is left now is to test that this delivers persistence by creating a new deployment that uses this Persistent Volume Claim.

Create a Stateful Application Deployment

Next, you will create a deployment descriptor YAML file to deploy a stateful application using the Persistent Volume Claim.

Create a Kubernetes new deployment YAML file to deploy Nginx on Oracle Cloud Native Environment.

cat << 'EOF' > ~/pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pvc-pod
spec:
  containers:
    - name: pvc-pod-container
      image: nginx:latest
      ports:
        - containerPort: 80
      volumeMounts:
        - mountPath: "/tmp/test-pv"
          name: data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: test-pvc
EOF

Where:

The volumeMounts variable defines the path mounted inside the deployed Pod. This example is mounted as tmp/test-pv
The volumes variable defines which Persistent Volume Claim the Pod deployment should use. This example uses test-pvc

Deploy the Pod.
```
kubectl apply -f pod.yaml
```
Confirm the Pod deployed and is running.
```
kubectl get pod
```
Example Output:
```
NAME      READY   STATUS    RESTARTS   AGE
pvc-pod   1/1     Running   0          1m
```
Notice that the STATUS column confirms that the Pod deployed and should be running. The next stage is to test that the Persistent Volume is mounted and accessible.

(Optional) If the Pod deploys successfully, the Persistent Volume Claim should have claimed the Persistent Volume.

kubectl get pv

Example Output:

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM              STORAGECLASS   REASON   AGE
test-pv   1Gi        RWO            Retain           Bound    default/test-pvc   oci-bv                  20m

kubectl get pvc

Example Output:

NAME       STATUS   VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-pvc   Bound    test-pv   1Gi        RWO            oci-bv         20m

Notice that the STATUS column in the Persistent Volume and the Persistent Volume Claim confirm they are Bound. Additionally, the CLAIM column for the Persistent Volume shows the Persistent Volume Claim (test-pvc) you created earlier has Bound to the Persistent Volume.

Log into the deployed Pod and test the Persistent Volume

These steps show how to get a session shell into the running container and test that writing data stores that data outside of the Pod. Then you will delete the Pod and recreate it. If it works, the data will remain accessible to the freshly deployed Pod.

Get a shell inside the Pod.
```
kubectl exec -it pod/pvc-pod -- /bin/bash
```
The command sends you to a shell prompt inside the Pod you just deployed.
Check whether there are any files in the mounted volume yet.
```
ls /tmp/test-pv
```
As expected, nothing returns.
Write a file to the volume.
```
echo "bar" > /tmp/test-pv/foo
```
Confirm the file shows in the volume.
```
ls /tmp/test-pv
```
The output shows the file foo exists.
Exit the Pod shell environment.
```
exit
```
This command exits the Pods shell and returns to the host Oracle Linux node.
The file you created in the volume is now stored independently of the Pod. Next, you will confirm this by deleting and recreating the Pod.
Delete the Pod.
```
kubectl delete pod/pvc-pod
```
Deploy the Pod again.
```
kubectl apply -f pod.yaml
```

Shell into the Pod again.

kubectl exec -it pod/pvc-pod -- /bin/bash

Confirm the foo file you created earlier is still accessible.
```
cat /tmp/test-pv/foo
```
The output displays the contents of the foo file.

Where is the Data Written to?

A primary reason to store files outside of any container is that the information they contain is valuable to your business or you personally. Therefore, you need to know where the data is located to manage that information, for example, to ensure backups work. The following steps will show how to find the data on the free lab deployment.

In the free lab environment, the Pod deployment executes on the ocne-worker-01 node. Any file(s) are saved to the local disk on that node. The next steps show how to confirm that they are stored there.

Check the /tmp/test-pv directory.
```
ssh ocne-worker-01 "ls /tmp/test-pv"
```
The output shows the file foo exists.
Confirm the foo file holds the same content you entered from inside the Pod.
```
ssh ocne-worker-01 "cat /tmp/test-pv/foo"
```
The output of bar confirms that the content exists and that the `foo' file you created from within the Pod was saved externally, and the Persistent Volume and Persistent Volume Claim provided the mechanism for the Pod to do it.

Summary

This concludes the walkthrough by introducing Persistent Volumes and Persistent Volume Claims and how to use them to provide some flexibility over how you manage your data.

For More Information