Deploying Percona Kubernetes Operators with OpenEBS Local Storage

Deploying Percona Kubernetes Operators with OpenEBS Local StorageNetwork volumes in
Kubernetes provide great flexibility, but still, nothing beats
local volumes from direct-attached storage in the sense of database
performance.

I want to explore ways to deploy both Percona
Kubernetes Operators
(Percona Kubernetes Operator for
Percona XtraDB Cluster
and Percona Kubernetes
Operator for Percona Server for MongoDB
) using local
volumes, both on the bare-metal deployments or in cloud
deployments.

Simple Ways

There are two ways available out of the box to deploy using
local storage, which you can use immediately.

You can specify in cluster deployment yaml volume specification,
using either hostPath:

volumeSpec:
      hostPath:
        path: /data
        type: Directory

Or

emptyDir

(which will be equal somewhat to ephemeral storage in EC2):

volumeSpec:
      emptyDir: {}

While this will work, it is a very rigid way to force local
storage, and it is not really the “Kubernetes way”, as we will
lose the capability to manage data volumes. We want to see a more
uniform way with Persistent Volumes and Persistent Volumes
Claims.

Persistent Volumes

Recognizing limitations with hostPath and emptyDir, Kubernetes
introduced
Local Persistent Volumes
.

Unfortunately, while this will work for the simple deployment of
single pods, it does not work with dynamically created volumes
which we need for Operators. We need the support of
Dynamic Volume Provisioning
.

There are several projects to combine Dynamic Volume
Provisioning with Local Persistent Volumes, but I did not have much
success with them, and the only project which worked for me is
OpenEBS.

OpenEBS provides much more than just Local Persistent Volumes,
but in this blog, I want to touch only OpenEBS Local PV
Hostpath.

OpenEBS Local PV Hostpath

This is actually quite simple, and this is what I like about
OpenEBS. First, we will define storage classes:

For the local nvme storage:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: localnvme
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: hostpath
      - name: BasePath
        value: /data/openebs
provisioner: openebs.io/local
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

And for the local ssd:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: localssd
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: hostpath
      - name: BasePath
        value: /mnt/data/ebs
provisioner: openebs.io/local
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

And after that we can deploy Operators using StorageClass, for
example  in cluster deployment yaml:

volumeSpec:
      persistentVolumeClaim:
        storageClassName: localnvme
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 200Gi

The cluster will be deployed using localnvme StorageClass, and
using space on /data/openebs.

Now we can observe used volumes with kubectl get pv:

NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                        STORAGECLASS       REASON   AGE
pvc-325e11ff-9082-432c-b484-c0c7a3d1c949   200Gi      RWO            Delete           Bound    pxc/datadir-cluster2-pxc-2   localssd                    2d18h
pvc-3f940d14-8363-450e-a238-a9ff09bee5bc   200Gi      RWO            Delete           Bound    pxc/datadir-cluster3-pxc-1   localnvme                   43h
pvc-421cbf46-73a6-45f1-811e-1fca279fe22d   200Gi      RWO            Delete           Bound    pxc/datadir-cluster2-pxc-1   localssd                    2d18h
pvc-53ee7345-bc53-4808-97d6-2acf728a57d7   200Gi      RWO            Delete           Bound    pxc/datadir-cluster3-pxc-0   localnvme                   43h
pvc-b98eca4a-03b0-4b5f-a152-9b45a35767c6   200Gi      RWO            Delete           Bound    pxc/datadir-cluster2-pxc-0   localssd                    2d18h
pvc-fbe499a9-ecae-4196-b29e-c4d35d69b381   200Gi      RWO            Delete           Bound    pxc/datadir-cluster3-pxc-2   localnvme                   43h

There we can see data volumes from two deployed clusters.
OpenEBS Local PV Hostpath is now my go-to method to deploy clusters
with the local storage.