Monitor with the UI
The Statistics screen allows statistics for Couchbase, Ephemeral, and Memcached buckets to be displayed; either for a specific server in the cluster, or for the cluster as a whole.
Access Bucket Statistics
To access bucket statistics, first, access the Buckets screen, by left-clicking on the Buckets tab, in the left-hand, vertical navigation bar:
When the Buckets screen is displayed, if one or more buckets have been defined, information on each is displayed, on a row that features columns for name, items, RAM used, and more. To the right of each row appear tabs for Documents, and Statistics.
Left-click on the Statistics tab for the bucket whose information you wish to display:
The Statistics screen is now displayed. Each of the principal areas of the screen is described below, in order of vertical presentation.
General Controls
The upper panel of the Statistics screen provides controls whereby display-parameters can be configured.
-
Bucket Selection — Located at the upper left, this pull-down menu allows selection of a bucket, statistics for which will be duly displayed. This is initially set to the choice the user made on the Buckets screen, when they accessed the current, Statistics screen.
-
Server Selection — Located at the upper right, this pull-down menu allows selection either of an individual server, or of the whole cluster, as the basis for the information displayed by all components on the Statistics screen.
-
Interval Selection — Located at the right, these vertically arranged tabs support selection of the time-interval according to which information will be displayed. For example, selecting Minute causes information to be shown for the most recent minute, with the chart continuously being updated.
-
Ops Per Sec. — The main chart for the Statistics screen. This initially shows operations per second, either for a specific server, or for the whole cluster (depending on the Server Selection that has been made), displayed according to the specified time-interval. Note that left-clicking on any of the smaller charts, displayed lower on the screen, redisplays that chart as the main chart for the page, thereby substituting it for this default.
Statistics Groups
Below the general controls, information is aggregated and displayed according to the following statistics groups. All groups provide information that relates either to a specific server, or to the whole cluster, depending on the Server Selection that has been made.
-
Server Resources — Consumption of server resources.
-
vBucket Resources — vBucket activities, including active, replica and pending operations.
-
Disk Queues — Active, replica and pending disk-related activities.
-
DCP Queues — DCP-related activities.
-
View Statistics — Statistics for each production view configured within the bucket or system.
-
Index Statistics — Index statistics, provided per index.
-
Query Statistics — Query Service activities. Note that these statistics are aggregated across the entire cluster, rather than per bucket or per server.
-
XDCR Destination — Statistics on Cross Data Center Replication (XDCR).
-
Top Keys — The top 10 most actively used keys, within the selected bucket.
Note that within these statistics groups, some charts are displayed only when their corresponding service or process is hosted. For example, the chart for outgoing XDCR operations is displayed only when XDCR is replicating data from the selected bucket.
Note also that certain charts displayed to indicate Couchbase-bucket disk activity are not displayed for Ephemeral buckets, since Ephemeral buckets do not support persistence.
Viewing Statistics Per Server
Each chart features a per server option, which allows the data to be displayed in detail for each server in the cluster.
For example, left-click on the per server option for the Management Port Reqs/Sec chart, located in the Server Resources statistics group:
The display changes as follows:
The display now provides separate information for each of the two servers in the current cluster, which are 101.143.192.101 and 101.143.192.103.
Bucket Monitoring — Summary Statistics
The summary section provides an overview of bucket-related activity. Each chart shows information based on the currently selected bucket.
The following statistics are included:
| Statistic | Description |
|---|---|
ops per second |
The total number of operations per second on this bucket. |
cache miss ratio |
Ratio of reads per second to this bucket which required a read from disk rather than RAM. |
gets per sec. |
Number of get operations per second from this bucket. |
total gets per sec. |
Total number of get operations per second from this bucket. |
sets per sec. |
Number of set operations per second to this bucket. |
deletes per sec. |
Number of delete operations per second for this bucket. |
CAS ops per sec. |
Number of operations with a CAS identification per second for this bucket. |
active docs resident % |
The percentage of active items cached in RAM in this bucket. |
items |
Number of unique items (documents) stored in the bucket. |
temp OOM per sec. |
Number of temporary out of memory conditions per second. |
low water mark |
Low water mark for this bucket (based on the configured bucket RAM quota). |
high water mark |
High water mark for this bucket (based on the configured bucket RAM quota). |
memory used |
Amount of memory used for storing the information in this bucket. |
disk creates per sec. |
Number of new items created on disk per second for this bucket. This chart is not displayed when an Ephemeral bucket has been selected. |
disk updates per sec. |
Number of items updated on disk per second for this bucket. This chart is not displayed when an Ephemeral bucket has been selected. |
disk reads per sec. |
Number of reads per second from disk for this bucket. This chart is not displayed when an Ephemeral bucket has been selected. |
disk write queue |
Number of items waiting to be written on disk in this bucket. This chart is not displayed when an Ephemeral bucket has been selected. |
disk read failures |
Number of disk read failures. This chart is not displayed when an Ephemeral bucket has been selected. |
disk write failures |
Number of disk write failures. This chart is not displayed when an Ephemeral bucket has been selected. |
docs data size |
Size of the stored document data. This chart is not displayed when an Ephemeral bucket has been selected. |
docs total disk size |
Size of the persisted stored document data on disk. This chart is not displayed when an Ephemeral bucket has been selected. |
doc fragmentation % |
Document fragmentation of persisted data as stored on disk. This chart is not displayed when an Ephemeral bucket has been selected. |
total disk size |
Total size of the information for this bucket as stored on disk, including persisted and view index data. This chart is not displayed when an Ephemeral bucket has been selected. |
view data size |
Size of the view data information. This chart is not displayed when an Ephemeral bucket has been selected. |
views total disk size |
Size of the view index information as stored on disk. This chart is not displayed when an Ephemeral bucket has been selected. |
views fragmentation % |
Percentage of fragmentation for a given view index. This chart is not displayed when an Ephemeral bucket has been selected. |
view reads per sec. |
Number of view reads per second. This chart is not displayed when an Ephemeral bucket has been selected. |
disk update time |
The time required to update data on disk. This chart is not displayed when an Ephemeral bucket has been selected. |
disk commit time |
The time required to commit data on disk. This chart is not displayed when an Ephemeral bucket has been selected. |
bg wait time |
The average background fetch time in microseconds. |
Incoming XDCR ops/sec. |
The incoming XDCR operations per second for this bucket. |
Intra-replication queue |
Number of items remaining to be sent to consumer in this bucket. |
outbound XDCR mutations |
Number of mutations to be replicated to other clusters. This chart is displayed only if the selected bucket is a source for ongoing replication. |
N1QL queries/sec |
Number of N1QL requests processed per second. |
index data size |
The data size consumed by the index. |
index disk size |
The total disk sized consumed by the index. |
index fragmentation % |
The percentage fragmentation of the index. |
index scanned/sec |
Number of index items scanned by the indexer per second. |
fts bytes indexed/sec |
Number of full text search bytes indexed per second. |
fts queries/sec |
Number of full text search queries per second. |
fts disk size |
The total full text search disk size for this bucket. |
avg active drift/mutation |
Average drift (in seconds) per mutation on active vBuckets. |
avg replica drift/mutation |
Average drift (in seconds) per mutation on replica vBuckets. |
active ahead exceptions/sec |
Total number of ahead exceptions all active vBuckets. |
replica ahead exceptions/sec |
Total number of ahead exceptions all replica vBuckets. |
Monitoring Server Resources
The Server Resources statistics group displays general resource information for either the selected server, or for the whole cluster; including swap usage, free RAM, CPU utilization percentage, connections, port requests, streaming requests, index RAM used, remaining index RAM, and FTS RAM used.
The displayed statistics are:
| Statistic | Description |
|---|---|
swap usage |
The amount of swap space in use. |
free RAM |
The amount of RAM available. |
Max CPU utilization % |
The percentage of CPU in use across all available cores. |
connections |
The number of connections. |
Management port requests/sec |
The rate of HTTP requests on this port. |
idle streaming requests |
The number of streaming requests. |
streaming wakeups/sec |
The number of streaming request wakeups. |
Max index RAM used % |
The percentage of index RAM in use across all indexes. |
remaining index RAM |
The amount of index RAM available. |
fts RAM used |
The amount of RAM used by FTS. |
analytics heap used |
Amount of JVM heap used by Analytics Service. |
analytics system load |
System load for Analytics Service. |
analytics thread count |
Number of threads for Analytics Service. |
analytics gc count/sec |
Number of JVM garbage collections per seconde for Analytics Service. |
analytics gc time (ms.)/sec |
The amount of time in milliseconds per second spent performing JVM garbage collections for Analytics Service. |
analytics bytes read/sec |
Number of disk bytes read per second for Analytics Service. |
analytics bytes written/sec |
Number of disk bytes written per second for Analytics Service. |
analytics total disk size |
The total disk size used by the Analytics Service. |
Monitoring vBucket Resources
The vBucket Resources statistics group provides information for all vBucket types within the cluster, across three different states; which are Active, Replica, and Pending. The charts are displayed as a table. Each row supports a particular statistic. The first three columns respectively correspond to the Active, Replica and Pending states. The fourth column provides the total for each statistic.
The individual statistics are as follows:
| Statistic | Description |
|---|---|
vBuckets |
The number of vBuckets within the specified state. |
items |
Number of items within the vBucket of the specified state. |
resident % |
Percentage of items within the vBuckets of the specified state that are resident (in RAM). |
new items per sec. |
Number of new items created in vBuckets within the specified state. Note that this statistic is not valid for the Pending state. |
ejections per sec. |
Number of items ejected per second within the vBuckets of the specified state. |
user data in RAM |
Size of user data within vBuckets of the specified state that are resident in RAM. |
metadata in RAM |
Size of item metadata within the vBuckets of the specified state that are resident in RAM. |
Monitoring Disk Queues
The Disk Queues statistics group displays information for data being placed into disk queues. Information is displayed for each of the disk-queue states, which are Active, Replica, and Pending.
Note that this statistics group is not displayed when the selected bucket is an Ephemeral bucket.
The charts are displayed as a table. Each row supports a particular statistic. The first three columns respectively correspond to the Active, Replica and Pending states.
The displayed statistics are:
| Statistic | Description |
|---|---|
items |
The number of items waiting to be written to disk for this bucket for this state. |
fill rate |
The number of items per second being added to the disk queue for the corresponding state. |
drain rate |
The number of items actually written to disk from the disk queue for the corresponding state. |
average age |
The average age of items (in seconds) within the disk queue for the specified state. |
Monitoring DCP Queues
The DCP Queues statistics group shows information about DCP connections for the selected bucket. The charts are displayed as a table; with each row corresponding to a particular statistic; and the five columns corresponding to DCP queues respectively for Replication, XDCR, Views/Indexes, Analytics, and Other.
The statistics are as follows:
| Statistic | Description |
|---|---|
DCP connections |
Number of DCP connections from this bucket. |
DCP senders |
Number of senders for this bucket. |
items remaining |
Number of items remaining to be sent. |
drain rate items/sec |
Number of items per second leaving the queue. |
drain rate bytes/sec |
Number of bytes per second leaving the queue. |
backoffs/sec |
Number of backoffs received on the queue. |
Monitoring View Statistics
The View Stats statistics group shows information on individual design documents within the selected bucket. One block of stats is shown for each production-level design document.
The statistics are as follows:
| Statistics | Description |
|---|---|
data size |
Size of the data required for this design document. |
disk size |
Size of the stored index as stored on disk. |
view reads per sec. |
Number of read operations per second for this view. |
Monitoring Index Statistics
The Index Statistics statistics group provides per index information on GSI Indexes.
The statistics are as follows:
| Statistic | Description |
|---|---|
items scanned/sec |
Number of index items scanned by the indexer per second. |
disk size |
Total disk file size consumed by the index. |
data size |
Actual data size consumed by the index. |
memory used |
Total memory consumed by the index storage. |
total mutations remaining |
Number of documents pending to be indexed. |
drain rate items/sec |
Number of documents indexed by the indexer per second. |
total indexed items |
The total number of documents indexed. |
average item size |
The average size of each index item. |
% fragmentation |
Percentage fragmentation of the index. This indicates the % of disk space consumed by the index, but not utilized for items stored in the index. Note that at small index sizes of less than a hundred kb, the statis overhead of the index disk file will inflate the index fragmentation percentage. |
requests/sec |
Number of requests served by the indexer per second. |
bytes returned/sec |
Number of bytes per second read by a scan. |
avg scan latency(ns) |
The average time to serve a scan request in nanoseconds. |
cache resident percent |
Percentage of index data resident in memory. |
index cache miss ratio |
Percentage of accesses made to this data from disk, rather than from RAM. |
Monitoring Analytics Statistics
The Analytics Stats statistics group shows information on the Analytics Service. Note that if the Analytics Service is not running, the charts are blank, and the statistic is given as N/A (_Not Applicable).
The statistics are as follows:
| Statistic | Description |
|---|---|
ops/sec |
Total number of operations (gets, sets, and deletes) processed for the Analytics Service on this server, per second. |
sync failed records |
Number of records not successfully parsed during bucket resynchronization. |
total ops since bucket connect |
Total number of operations processed by the Analytics Service for this server since the bucket was last connected. |
Monitoring Outgoing XDCR
The Outgoing XDCR statistics group provides information on XDCR operations that are supporting cross datacenter replication, from the current cluster to a destination cluster.
The statistics are as follows:
| Statistic | Description |
|---|---|
mutations |
Number of mutations to be replicated. |
percent completed |
Percentage of mutations replicated. |
mutations replicated |
Number of mutations replicated. |
mutations filtered per sec. |
Number of mutations filtered out, and therefore not replicated. |
mutations skipped by resolution |
Number of mutations that have failed conflict resolution on the source-side, and therefore not replicated. |
mutation replication rate |
Number of mutations replicated per second. |
data replication rate |
Number of bytes replicated per second. |
opt. replication rate |
Rate of optimistic replications in terms of number of replicated mutations per second. |
doc checks rate |
Rate of doc checks per second. |
ms meta batch latency |
Weighted average latency in milliseconds of sending getMeta and waiting for conflict resolution result from remote server. |
ms doc batch latency |
Weighted average latency in milliseconds of sending replicated mutations to remote cluster. |
doc reception rate |
Rate of mutations received from DCP in terms of number of mutations per second. |
Monitoring Query Statistics
The Query statistics group provides information on the Query Service. Note that these statistics are aggregated across the entire cluster, rather than per bucket or per server.
The statistics are as follows:
| Statistic | Description |
|---|---|
requests/sec |
Total number of N1QL requests processed per second. |
selects/sec |
Total number of SELECT requests processed per second. |
request time(sec) |
The average end-to-end time to process a query in seconds. |
service time(sec) |
The average time to execute a query in seconds. |
result size |
The average size in bytes of the data returned by the query. |
errors |
The total number of N1QL errors returned so far. |
warnings |
The total number of N1QL warnings returned so far. |
result count |
The average number of results (documents) returned by a query. |
queries > 250ms |
Number of queries that take longer than 250ms. |
queries > 500ms |
Number of queries that take longer than 500ms. |
queries > 1000ms |
Number of queries that take longer than 1000ms. |
queries > 5000ms |
Number of queries that take longer than 5000ms. |
invalid requests/sec |
Number of requests for unsupported endpoints per second. |
Monitoring Incoming XDCR
The Incoming XDCR Operations statistics group provides information on the XDCR operations that are coming into to the current cluster from a remote cluster.
The statistics are as follows:
| Statistic | Description |
|---|---|
metadata reads per sec. |
Number of documents XDCR scans for metadata per second. XDCR uses this information for conflict resolution. |
sets per sec. |
Set operations per second for incoming XDCR data. |
deletes per sec. |
Delete operations per second as a result of the incoming XDCR data stream. |
total ops per sec. |
Total of all the operations per second. |
Memcached Statistics
If a Memcached bucket is defined, selecting that bucket displays Memcached statistics group for the bucket.
The statistics are as follows:
| Statistic | Description |
|---|---|
ops per sec. |
Total operations per second serviced by this bucket. |
hit ratio |
Percentage of get requests served with data by this bucket. |
RAM used |
Total amount of RAM used by this bucket. |
items |
Number of items stored in this bucket. |
evictions per sec. |
Number of items evicted per second from this bucket. |
sets per sec. |
Number of set operations serviced per second by this bucket. |
gets per sec. |
Number of get operations serviced per second by this bucket. |
bytes TX per sec. |
Number of bytes per second sent from this bucket. |
bytes RX per sec. |
Number of bytes per second sent into this bucket. |
get hits per sec. |
Number of get operations per second for data that this bucket contains. |
delete hits per sec. |
Number of delete operations per second for data that this bucket contains. |
incr hits per sec. |
Number of increment operations per second for data that this bucket contains. |
decr hits per sec. |
Number of decrement operations per second for data that this bucket contains. |
delete misses per sec. |
Number of delete operations per second for data that this bucket does not contain. |
decr misses per sec. |
Number of decrement operations per second for data that this bucket does not contain. |
get misses per sec. |
Number of get operations per second for data that this bucket does not contain. |
inc misses per sec. |
Number of increment operations per second for data that this bucket does not contain. |
CAS hits per sec. |
Number of CAS operations per second for data that this bucket contains. |
CAS badval per sec. |
Number of CAS operations per second using an incorrect CAS ID for data that this bucket contains. |
CAS misses per sec. |
Number of CAS operations per second for data that this bucket does not contain. |