Logs and Troubleshooting
This section provides information about how to diagnose and troubleshoot problems with the Couchbase Operator or your deployment.
When troubleshooting the Couchbase Operator, it is important to rule out Kubernetes itself as the root cause of the problem you are experiencing. See the Kubernetes Troubleshooting Guide for information about debugging applications within a Kubernetes cluster.
The following topics are also helpful to understand when troubleshooting the Operator:
The Operator is distributed with a support tool which can automatically collect resources, logs and events from the Kubernetes cluster for use in support cases. It is also capable of collecting logs from Couchbase server instances via cbcollect_info. Please see the documentation.
The Couchbase Operator generates logs that can help troubleshoot your deployment. Using
oc, you can choose to print the Operator logs to
# Get name of operator pod $ kubectl get po -lapp=couchbase-operator NAME READY STATUS RESTARTS AGE couchbase-operator-1917615544-h20bm 1/1 Running 0 20h # Get logs $ kubectl logs couchbase-operator-1917615544-h20bm time="2018-01-23T22:56:34Z" level=info msg="Obtaining resource lock" module=main time="2018-01-23T22:56:34Z" level=info msg="Starting event recorder" module=main time="2018-01-23T22:56:34Z" level=info msg="Attempting to be elected the couchbase-operator leader" module=main time="2018-01-23T22:56:51Z" level=info msg="I'm the leader, attempt to start the operator" module=main time="2018-01-23T22:56:51Z" level=info msg="Creating the couchbase-operator controller" module=main
# Get name of operator pod $ oc get po -lapp=couchbase-operator
Watch for the following messages which indicate that the Operator is unable to reconcile your cluster into a desired state:
Operator is unable to get cluster state after N retries
The Couchbase operator serves profiling data on it’s default listenAddress
localhost:8080. You can access this endpoint by performing a
kubectl exec into the operator pod, for example:
$ kubectl exec -it couchbase-operator-599bcf47f-8wswh sh $ wget -O- "localhost:8080/debug/pprof/trace" | less
Another option is to create a custom service or forward the service directly to your localhost.
$ kubectl port-forward couchbase-operator-599bcf47f-8wswh 8080:8080 $ go tool pprof localhost:8080/debug/pprof/heap $ (pprof) traces
Refer to the Couchbase Server Troubleshooting guide for additional information about reporting issues.