Bucket Backend Migration Documentation

In Couchbase Server 7.6.0 the backend of a storage bucket can be migrated between Magma and Couchstore.

Migrating Couchbase Bucket storage Backends

The migration of a bucket requires each node the bucket is using to be swap-rebalanced out of a cluster. Once each node has been replaced then the bucket will be completely using the new storage backend. The Couchbase Operator can perform this migration. The migration for buckets can be triggered in a number of ways.

Using managed CouchbaseBuckets

To migrate a CouchbaseBucket resource that is managed by the operator the spec.storageBackend for a CouchbaseBucket resource can be modified to the new storage backend. This feature is only available in Couchbase Operator versions 2.6.1+.

Note that the Magma backend has additional requirements on the bucket: * Bucket memory quota being 1024MB or more * Bucket must use the "fullEviction" eviction policy.

Example bucket spec

apiVersion: couchbase.com/v2
kind: CouchbaseBucket
metadata:
  name: bucket1
spec:
  memoryQuota: 1024Mi
  storageBackend: magma
  evictionPolicy: fullEviction

Using Managed CouchbaseBuckets with defaultStorageBackend cluster annotation

A large number of buckets can be migrated at once by changing the default storage backend using the cao.couchbase.com/buckets.defaultStorageBackend annotation. This will only effect buckets that don’t explicity set spec.storageBackend. The default storage backend is couchstore, but this annotation can be used to override the default to magma

Example of a bucket that doesn’t set spec.storageBackend:

apiVersion: couchbase.com/v2
kind: CouchbaseBucket
metadata:
  name: bucket1
spec:
  memoryQuota: 1024Mi
  evictionPolicy: fullEviction

The cao.couchbase.com/buckets.defaultStorageBackend annotation can then be applied to a Couchbase cluster spec to migrate all the buckets from one backend to another.

Example of a cluster spec to change the default storage backend to magma.

apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
  annotations:
    cao.couchbase.com/buckets.defaultStorageBackend: magma
spec:
  image: couchbase/server:7.6.0
  ...

If this annotation is removed from a cluster the buckets will then attempt to be migrated back to the couchstore backend so this annotation should stay on the cluster if the buckets should continue using the magma backend. Alternatively the bucket specs can be updated to explicitly set the backend to magma after which the annotation can then be removed.

Migrating unmanaged buckets with targetUnmanagedBucketStorageBackend cluster annotation

The Couchbase Operator can also migrate unmanaged buckets. However this will target all bucket in the cluster. It requires the cluster to have bucket management disabled. It also requires that the buckets meet the neccesary requirements to be migrated to magma: * Bucket memory quota must be 1024MB or more * Bucket must use the "fullEviction" eviction policy.

To use the annotation apply the targetUnmanagedBucketStorageBackend annotation to your cluster.

apiVersion: couchbase.com/v2
kind: CouchbaseCluster
metadata:
  name: cb-example
  annotations:
    cao.couchbase.com/buckets.targetUnmanagedBucketStorageBackend: magma
buckets:
  managed: false
spec:
  image: couchbase/server:7.6.0
  ...

Whilst this annotation is on the cluster, the Couchbase Operator will try to migrate all unmanaged buckets to the specified backend. When the annotation is removed then the Operator will stop attempting the migrations. If any buckets fail migration due to not meeting the requirements then the annotation can be removed until the buckets have been modified to meet the requirements, then reapplied at a later time to attempt migration again.