A newer version of this documentation is available.

View Latest

Persistent Volumes Zoning Guide

    +
    When deploying on a cloud that spans multiple availability zones, certain steps are required in order to use persistent volumes. These steps will vary depending on the version of Kubernetes being provided by your cloud.
    Kubernetes 1.12+
    • Step 1: Create a storage class with multi-zone binding

    • Step 2: Add the storage class to the cluster configuration

    If your cloud provider is running Kubernetes 1.11.x, refer to the legacy zoning guide.

    There is no upgrade path from Kubernetes 1.11 to 1.12 configurations since the underlying StorageClass used by a pod cannot be changed. Therefore, consider using Kubernetes 1.12+ for easier long-term maintenance of your cluster since the configuration is less complex for deployments that use server groups.

    Create a Multi-Zoned Storage Class

    When creating volumes across multiple availability zones, we need to ensure that the volumes are bound in the same zone as the pod. This can be accomplished by creating a StorageClass with volumeBindingMode set to WaitForFirstConsumer. When volumes are created under this binding mode, it means that the pod will determine which node the volumes should bind to since the pod is considered its first consumer. Another reason why this is necessary is because the default binding mode of Immediate often results in scheduling conflicts when the pod is created in a different zone than its volumes.

    IMPORTANT

    Even if you are not using server groups you will still need to set the binding mode to WaitForFirstConsumer when your cloud has multiple zones. Some cloud providers may not require this, but for performance, it’s recommended that pods and volumes are always placed in the same zone.

    The following snippet is an example of a storage class that uses the binding mode WaitForFirstConsumer to ensure volumes are bound to the proper zone:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      labels:
        k8s-addon: storage-aws.addons.k8s.io
      name: gp2-multi
    parameters:
      type: gp2
    provisioner: kubernetes.io/aws-ebs
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer

    Notice here that volumeBindingMode is set to WaitForFirstConsumer. Everything else can be the exact same as your other storage classes.

    Add the Storage Class to the Configuration

    Storage classes are used by the persistent volume claim templates to dynamically create volumes for Pods. Therefore, the final step of configuration is to specify the storage class which should be used for creating volumes in the volumeClaimTemplates.spec.storageClassName section of the spec. In the example below the gp2-multi storage class we just created is specified as the storage class for provisioning volumes:

    spec:
      servers:
        - name: data-nodes
          size: 3
          services:
            - data
            - index
          pod:
            volumeMounts:
              default: gp2-multi
              data: gp2-multi
      volumeClaimTemplates:
        - metadata:
            name: gp2-multi
          spec:
            storageClassName: gp2-multi
            resources:
              requests:
                storage: 1Gi

    Multi-Dimensional Scaling Across Zones

    The most practical use-case for deploying Kubernetes across multiple availability zones is to prevent data loss in the event a outage. Therefore, Couchbase server groups are often used in these environments, and when combined with persistent volumes, an even higher level of resiliance to disaster can be achieved.

    The following example shows the use of server groups with persistent volumes on a cloud that’s configured with multiple availability groups:

    spec:
      servers:
        - name: zone-a
          size: 3
          services:
            - data
            - index
          serverGroups:
            - us-east-1a
          pod:
            volumeMounts:
              default: gp2-multi
              data: gp2-multi
        - name: zone-b
          size: 3
          services:
            - data
            - index
          serverGroups:
            - us-east-1b
          pod:
            volumeMounts:
              default: gp2-multi
              data: gp2-multi
        - name: zone-c
          size: 3
          services:
            - data
            - index
          serverGroups:
            - us-east-1c
          pod:
            volumeMounts:
              default: gp2-multi
              data: gp2-multi
      volumeClaimTemplates:
        - metadata:
            name: gp2-multi
          spec:
            storageClassName: gp2-multi
            resources:
              requests:
                storage: 1Gi

    Here we have three zones, each able to consume volumes provided by a single storage class named gp2-multi.

    See the CouchbaseCluster configuration documentation for more detailed information about defining a cluster with persistent volumes.