A newer version of this documentation is available.

View Latest

Logs and Troubleshooting

This section provides information about how to diagnose and troubleshoot problems with the Couchbase Operator or your deployment.

When troubleshooting the Couchbase Operator, it is important to rule out Kubernetes itself as the root cause of the problem you are experiencing. See the Kubernetes Troubleshooting Guide for information about debugging applications within a Kubernetes cluster.

The following topics are also helpful to understand when troubleshooting the Operator:

Full Deployment Logs

The Operator is distributed with a support tool which can automatically collect resources, logs and events from the Kubernetes cluster for use in support cases. It is also capable of collecting logs from Couchbase server instances via cbcollect_info. Please see the documentation.

Operator Logs

The Couchbase Operator generates logs that can help troubleshoot your deployment. Using kubectl or oc, you can choose to print the Operator logs to stdout.

On Kubernetes:

# Get name of operator pod
$ kubectl get po -lapp=couchbase-operator
NAME                                  READY     STATUS    RESTARTS   AGE
couchbase-operator-1917615544-h20bm   1/1       Running   0          20h

# Get logs
$ kubectl logs couchbase-operator-1917615544-h20bm
time="2018-01-23T22:56:34Z" level=info msg="Obtaining resource lock" module=main
time="2018-01-23T22:56:34Z" level=info msg="Starting event recorder" module=main
time="2018-01-23T22:56:34Z" level=info msg="Attempting to be elected the couchbase-operator leader" module=main
time="2018-01-23T22:56:51Z" level=info msg="I'm the leader, attempt to start the operator" module=main
time="2018-01-23T22:56:51Z" level=info msg="Creating the couchbase-operator controller" module=main

On OpenShift:

# Get name of operator pod
$ oc get po -lapp=couchbase-operator

Watch for the following messages which indicate that the Operator is unable to reconcile your cluster into a desired state:

  • Logs with level=error

  • Operator is unable to get cluster state after N retries

Profiling the operator

The Couchbase operator serves profiling data on it’s default listenAddress localhost:8080. You can access this endpoint by performing a kubectl exec into the operator pod, for example:

$ kubectl exec -it couchbase-operator-599bcf47f-8wswh sh
$ wget -O- "localhost:8080/debug/pprof/trace" | less

Another option is to create a custom service or forward the service directly to your localhost.

$ kubectl port-forward couchbase-operator-599bcf47f-8wswh  8080:8080
$ go tool pprof localhost:8080/debug/pprof/heap
$ (pprof) traces

See Also

Refer to the Couchbase Server Troubleshooting guide for additional information about reporting issues.