Release Notes for Couchbase Autonomous Operator 2.2

If you are not utilizing TLS, you can skip this section.

The TLS requirements have been modified as of Autonomous Operator 2.1. In order to ease the migration from legacy client bootstrap (CCCP) to the newest version (GCCCP), the Autonomous Operator requires Couchbase cluster subject alternative names (SANs) to be updated. Consult the TLS tutorial for a full list of all the required SANs, and the TLS rotation guide in order to prepare for upgrade. Failure to perform this step will result in errors from the dynamic admission controller (DAC) once upgraded.

When upgrading from version 2.0.x, Couchbase clusters will undergo a mandatory upgrade cycle.

Pod readiness checks were previously driven by an exec based readiness probe. This was a security concern because it granted the Autonomous Operator pods/exec privileges, which may not be acceptable in highly regulated environments. As of Autonomous Operator 2.1, readiness checks are performed using readiness gates that use the Kubernetes API exclusively.

You can use the couchbaseclusters.spec.rollingUpgrade configuration parameter to speed up this upgrade. To enable this feature while upgrading the Autonomous Operator, stop the old Operator, replace the CRDs, edit the couchbaseclusters.spec.rollingUpgrade field to enable bulk upgrades, then start the new Operator. Refer to Upgrade the Operator for further details.

When using public networking with external-DNS or generic networking, the Operator polls new cluster members to ensure addressability and connectivity before allowing the member to be added to the cluster. Failure to do so would result in client errors and partial availability.

This has a downside in that polling DNS immediately would poison caching DNS servers with negative lookup results (negative caching).

In order to mitigate this problem, a new field couchbaseclusters.spec.networking.waitForAddressReachableDelay, is introduced. This allows a time period to be configured between pod creation and polling, allowing time for load balancers to be provisioned and DNS entries to be propagated. This defaults to 2m, changing the old behavior of immediate polling.

The new default images are couchbase/exporter:1.0.5 and couchbase/fluent-bit:1.0.4. The latest backup image is couchbase/operator-backup:1.1.1. These images feature functional and security updates.

Existing clusters will continue to use the component versions they were provisioned with. Should you wish to update existing Couchbase clusters to make use of these new images, then they will undergo a rolling upgrade to facilitate the update. You should plan a maintenance window accordingly.

This release adds support for the following platforms:

This release also adds support for the following utilities:

This release drops support for the following platforms:

The Autonomous Operator now supports Couchbase cluster auto-scaling for all Couchbase services. This means that Couchbase cluster server class configurations containing stateful services like Data and Index can now be configured to automatically scale in response to observed metrics. (Previously, only stateless deployments of the Query Service were supported.)

To help users get started in production, an auto-scaling best practices guide has been provided. This guide discusses relevant metrics for scaling individual Couchbase Services, and includes recommended settings based on internal benchmark testing performed by Couchbase.

The couchbaseclusters.spec.enableOnlineVolumeExpansion has been added to allow for the expansion of persistent volumes that are already in use by Couchbase clusters, thus removing the need for a rolling upgrade of cluster pods. The Autonomous Operator achieves this by working in conjunction with Kubernetes Persistent Volume Expansion to claim additional storage for running pods without any downtime.

For additional information and requirements, refer to Online Volume Expansion.

The Autonomous Operator now supports log forwarding through the optional deployment of a third party log processor. The log processor runs in a sidecar container on each Couchbase pod, which then reads the log files and forwards them to standard console output. For this purpose, Couchbase supplies a default log processor image based on Fluent Bit.

In addition, audit logging can now be configured via the CouchbaseCluster resource specification, thus allowing for automated audit logging via the Autonomous Operator.

The Couchbase Server default minimum TLS version is 1.0. This is insecure and easily compromised if a client downgrades to a version less than 1.2. Operator 2.2 makes TLS 1.2 the minimum by default, and will automatically update the TLS minimum version unless it is explicitly specified with the couchbaseclusters.spec.networking.tls.tlsMinimumVersion parameter.

You can prevent this change from occurring when upgrading the Operator on existing clusters by setting couchbaseclusters.spec.networking.tls.tlsMinimumVersion to TLS1.0 after upgrading the CRDs, and before restarting the Operator.

Couchbase Server requires that the TLS resources be called pkey.key and chain.pem, however the majority of 3rd-party certificate managers generate tls.key and tls.crt. This format is governed by the Kubernetes kubernetes.io/tls secret type. To provide integration with 3rd-party providers, and to maintain backward compatibility with existing clusters, the Operator now provides secret shadowing.

A new TLS source — couchbaseclusters.spec.networking.tls.secretSource — allows you to use kubernetes.io/tls type secrets to configure Couchbase Server. When using this new TLS source, a shadow copy of the TLS secret is created, with the key names changed to those hard coded in Couchbase Server. The shadow secret can then be mounted and used inside Couchbase Server pods. As the secret is shadowed, the Operator can reformat private keys, and therefore now supports PKCS#8 formatted private keys.

3rd-party certificate managers can be used with clusters that are configured with this new TLS source. Refer to the tutorial Using a Certificate Manager for an example.

For existing clusters using the legacy couchbaseclusters.spec.networking.tls.static source, the Operator works as before — directly mounting and consuming the TLS secret without a shadow secret.

Prior to this release, server groups had to be enabled on cluster creation, and were immutable for the lifetime of the cluster. These restrictions have now been lifted.

It is now possible to enable Couchbase Server groups, and Operator pod scheduling, while a cluster is running.

It is also now possible to modify the list of server groups used for scheduling. Previously if an availability zone were to suffer an outage, the Operator would continually attempt to recreate failed pods in the same availability zone to maintain even balance across the requested set. While the availability zone continues to suffer an outage, the Couchbase cluster would be undersized, potentially impacting the ability to service client requests. This release allows the availability zones to be modified, excluding the failed one, so that the Operator is able to scale the cluster back up to the correct size.

All server group migration operations use a shortest-path algorithm in order to minimize disruption.

Prior to this release, any updates to XDCR remote clusters would be silently ignored. In order to modify any setting, the remote cluster and all replications would have to be deleted and recreated from scratch.

This release adds the ability to modify remote cluster identification and authentication settings. This provides the ability to rotate passwords and certificates on the remote cluster, or even replace the remote cluster entirely.

Kubernetes allows pods to reserve and limit compute resources. Resource reservation provides Kubernetes with the ability to fairly schedule pods so that they don’t compete for CPU and memory.

The Operator now provides the ability to automatically manage pod CPU and memory resource requests with the couchbaseclusters.spec.autoResourceAllocation field. When in use, pods will have their resource requests automatically populated depending on the services enabled on that pod, and the individual Couchbase service quotas.

Updating quotas will cause the cluster to upgrade as pod resource requests are updated. A new dummy memory quota for the query service is introduced to allow management of memory resource requests, even though Couchbase itself does not provide the ability to constrain this service.

Prior to this release, the Operator allowed two types of upgrade — a one pod at a time rolling upgrade, and a whole cluster upgrade.

This release allows rolling upgrades to be extended to upgrade either a fixed number of pods, or a percentage of the cluster size. Both values can be set, and the Autonomous Operator will select the one that results in the fewest number of pods being upgraded at a time.

Rolling upgrades can be configured with the couchbaseclusters.spec.rollingUpgrade parameter.

With the release of Autonomous Operator 2.2, the operator-backup container image has been enhanced to support a wider range of Couchbase Server and Autonomous Operator versions. As part of this update, the operator-backup image has switched to semantic versioning which is no longer based on Couchbase Server version.

At the time of this writing, the new image — operator-backup:1.1.0 — is compatible with all versions of the Autonomous Operator that support managed backup and restore (version 2.0 and later).

The new image is still based on the cbbackupmgr utility, therefore its ability to backup and restore data between different versions of Couchbase Server is still dependent on the compatibility of the underlying cbbackupmgr version. However, the operator-backup:1.1.0 image is based on cbbackupmgr 6.6.2, which is capable of backing up and restoring data from all versions of Couchbase Server that have ever been supported by the Autonomous Operator up to this point. When support for new Couchbase Server versions is added to the Autonomous Operator, support for backing up those versions will be added to the latest operator-backup image.

The latest version of the Couchbase Prometheus Exporter allows for certain customizations to exported metrics. The following customizations are currently supported:

For more information, refer to Customizing Metrics.