Kubernetes users often need to check the status of various objects (such as pods, deployments, and StatefulSets) that have been deployed inside their Kubernetes clusters. Status checking is done by describing one of the objects in the cluster using
kubectl describe command.
kubectl describe command outputs the configuration for the specified object, a status section, and an event section. The status section varies depending on the object type and shows various health metrics related to the object described. The events sections is printed last and shows major events that have happened during the life of the object.
Since the Couchbase Operator registers a CouchbaseCluster Custom Resource Definition (CRD) with Kubernetes, it allows Kubernetes to know about Couchbase clusters natively. This means that the CouchbaseCluster becomes another object that can be described to get the configuration, status, and events specific to a particular Couchbase cluster.
To describe a Couchbase cluster named
cb-example, run the following command:
kubectl describe couchbasecluster cb-example
Note that the object name in the above command is
couchbasecluster. You can also use the shorthand name
cbc as the object name to save some typing.
kubectl describe cbc cb-example
oc describe cbc cb-example
Below is an example of the output of the
kubectl describe command. The various parts of the output and their significance will be discussed in detail in the following sections.
Name: cb-example Kind: CouchbaseCluster Spec: ... Status: Admin Console Port: 30239 Admin Console Port SSL: 31628 Buckets: Default: Conflict Resolution: seqno Enable Flush: true Enable Index Replica: false Eviction Policy: fullEviction Io Priority: high Memory Quota: 128 Name: default Replicas: 1 Type: couchbase Cluster Id: 9ae7fe4634e0360cf9c9245cb4ebb27b Conditions: Balanced: Message: Data is equally distributed across all nodes in the cluster Reason: Cluster is balanced Status: True ... Control Paused: false Current Version: enterprise-5.5.2 Members: Ready: cb-example-0000 cb-example-0001 cb-example-0002 Phase: Running Reason: Size: 3 Events: FirstSeen LastSeen Count From Type Reason Message --------- -------- ----- ---- -------- ------ ------- 21m 21m 1 couchbase-operator Normal NewMemberAdded New member cb-example-0000 added to cluster
Often when the status spec of the schema is updated, an associated event is generated which details the reason why the change occurred. For instance, increasing the size of the cluster causes an add member event to appear in the Events section of the status result. The following section explains the information provided in the status response, along with any events associated with value updates.
Admin Console Port: 30239 Admin Console Port SSL: 31628
These ports are used for exposing the Couchbase cluster’s web console. This section of the status is only visible when the
exposeAdminConsole setting of the cluster spec is enabled.
See admin-console-access.adoc for information on how to expose and access the Couchbase Server Web Console.
Enabling and disabling the
exposeAdminConsole setting produces the following events, respectively:
Events: ... ServiceCreated Service for admin console `cb-example-ui` was created ... ServiceDeleted Service for admin console `cb-example-ui` was deleted
Buckets: Default: Memory Quota: 128 Name: default ...
A list of buckets currently active within the Couchbase cluster. The values of the bucket status are the same as the values provided in the bucket specification. See CouchbaseCluster Configuration for details about the bucket specification. When buckets from the specification are added, updated, or removed the following events are generated, respectively:
Events: ... BucketCreated A new bucket `default` was created ... BucketEdited Bucket `default` was edited ... BucketDeleted Bucket `default` was deleted
The bucket section of the status only reflects valid changes that have been made to the cluster. In the case of attempts to update the bucket section with an invalid specification, the following message will be seen in the
Conditions section of the Status:
Conditions: ManageBuckets: Message: Bucket: default cannot change (default) bucket param='conflictResolution' from 'seqno' to 'timestamp' Reason: Bucket edit failed Status: False
To resolve errors resulting from invalid updates, edit the cluster specification according to the message shown under Conditions and apply the updated specification.
Cluster Id: 9ae7fe4634e0360cf9c9245cb4ebb27b
The cluster identifier as provided by the Couchbase cluster. This value is provided directly by the Couchbase cluster and is therefore never updated by any changes to the cluster specification.
Members: Ready: cb-example-0000 cb-example-0001 cb-example-0002
Members represent pods that are managed by the Couchbase Operator. All members in the Ready section represent pods that make up the Couchbase cluster. The status of cluster members is directly affected by the value of
Size in the cluster spec. When
Size is increased, a member add event is generated, followed by rebalance:
Events: ... NewMemberAdded New member cb-example-0003 added to cluster ... RebalanceStarted A rebalance has been started to balance data across the cluster
Removing a member generates the expected removal event, followed by a rebalance:
Events: ... RebalanceStarted A rebalance has been started to balance data across the cluster ... MemberRemoved Existing member cb-example-0003 removed from the cluster
|It is also possible for a MemberRemoved event to be generated when an auto-failover occurs and a member is replaced by a new member.|
Conditions section of the status is also updated as the operator works to resolve changes to cluster spec size or failures. Important conditions to check when scaling are:
Balanced: Denoting whether data is equally distributed across all nodes in the cluster.
Available: Denoting whether all members are up and all vBuckets are available.
Scaling: Denoting whether the cluster is currently scaling.
See Conditions and Events for more information about these conditions and their statuses.
The current phase of the cluster displayed as one of the following phases:
Creating: When a cluster is first deployed.
Running: After the
couchbaseclusterresource object is available and the first orchestrator pod is running.
Failed: When a failure occurs within the creation or running phase of the cluster.
None: Initial state denoted by an empty string.
The size of the Couchbase cluster. When you change this value in the cluster specification, this value is also updated as members are added to the Couchbase cluster. If an error occurs while the Operator is attempting to conform the cluster to the desired size, check the
Conditions section for information about the current state of the cluster. See admin-console-access.adoc for information on how to manually access the cluster and collect logs when additional troubleshooting is needed.
Conditions: Balanced: Message: Data is equally distributed across all nodes in the cluster Reason: Cluster is balanced Status: True ...
A list of conditions reflecting the current state of the Couchbase cluster. Each Condition item is denoted by a
Type along with an associated
Status. The various conditions and their statuses are documented in Conditions and Events.
Events: 29m 29m 1 couchbase-operator-1917615544-j1mg8 Normal NewMemberAdded New member cb-example-0003 added to cluster 29m 29m 1 couchbase-operator-1917615544-j1mg8 Normal RebalanceStarted A rebalance has been started to balance data across the cluster 51m 51m 1 couchbase-operator-1917615544-j1mg8 Normal BucketEdited Bucket `default` was edited
Events generated during cluster reconciliation. The last 10 events are recorded and timestamped as the Operator works to reconcile the cluster to its desired cluster state. The types of events that can occur throughout the lifecycle of a CouchbaseCluster are documented in Conditions and Events.