Getting XDCR Stats

Requests for XDCR statistics about a destination cluster are performed on the source cluster.

Description

All XDCR statistical requests use the UUID, a unique identifier for destination cluster. The UUID is retrieved with the GET /pools/default/remoteClusters HTTP method and URI. Many of these statistics are exposed in the Couchbase Web Console.

You need to provide a properly URL-encoded URI string for the destination endpoint when requesting XDCR statistics.

HTTP method and URI

The destination endpoint follows the /pools/default/buckets/[source_bucket]/stats/ URI endpoint:

GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

Where the destination endpoint is:

replications/[remote_UUID]/[source_bucket]/[destination_bucket]/[stat_name]

Where the HTTP endpoint string with full URI is:

http://[node address]:[port]/pools/[source_bucket]/buckets/[bucket_name]/stats/replications/ \
  [remote_UUID]/[source_bucket]/[destination_bucket]/[stat_name]

Where the HTTP string with a properly URL-encoded URI is:

http://[node address]:[port]/pools/[source_bucket]/buckets/[bucket_name]/stats/ \
    replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2F[stat_name]
Stat name Description

bandwidth_usage

Bandwidth used during replication, measured in bytes per second.

changes_left

Number of updates still pending replication.

data_replicated

Size of data replicated in bytes.

docs_checked

Number of documents checked for changes.

docs_failed_cr_source

Number of documents that have failed conflict resolution on the source cluster and not replicated to target cluster.

docs_filtered

Number of documents that have been filtered out and not replicated to target cluster.

docs_opt_repd

Number of docs sent optimistically.

docs_received_from_dcp

Number of documents received from DCP.

docs_rep_queue

Number of documents in replication queue.

docs_processed

Number of documents processed by pipeline.

docs_written

Number of documents written to the destination cluster via XDCR.

meta_latency_wt

Weighted average time for requesting document metadata. XDCR uses this for conflict resolution prior to sending the document into the replication queue.

num_checkpoints

Number of checkpoints issued in replication queue.

num_failedckpts

Number of checkpoints failed during replication.

rate_received_from_dcp

Number of documents received from DCP per second.

rate_replicated

Rate of documents being replicated, measured in documents per second.

size_rep_queue

Size of replication queue in bytes.

time_committing

Seconds elapsed during replication.

wtavg_docs_latency

Weighted average latency for sending replicated changes to destination cluster.

wtavg_meta_latency

Weighted average time for requesting document metadata. XDCR uses this for conflict resolution prior to sending the document into the replication queue.

Getting destination cluster info

For example, the following code example displays the destination remote name, URI and UUID (among other data):

// curl request example for retrieving the destination cluster UUID
curl -u Administrator:password http://10.5.2.54:8091/pools/default/remoteClusters


// example results
[
    {
        "deleted": false,
        "hostname": "10.5.2.117:8091",
        "name": "Remote117",
        "uri": "/pools/default/remoteClusters/Remote117",
        "username": "Administrator",
        "uuid": "995618a6a6cc9ac79731bd13240e19b5",
        "validateURI": "/pools/default/remoteClusters/Remote117?just_validate=1"
    }

Retrieving docs_written stats

HTTP method and URI

GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

Where the [destination_endpoint] is:

replications/[remote_UUID]/[source_bucket]/[destination_bucket]/docs_written

Syntax

Curl request syntax for number of documents written:

curl -u [admin]:[password]
    http://[node address]:8091/pools/default/buckets/[source_bucket]/stats/
    replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2Fdocs_written

Example

To get the number of documents written:

curl -u Administrator:password \
    http://10.5.2.54:8091/pools/default/buckets/default/stats/ \
    replications%2F8ba6870d88cd72b3f1db113fc8aee675%2Fdefault%2Fdefault%2Fdocs_written

Response

The above command produces the following output that shows that XDCR transferred 1 million documents at each of the timestamps.

{"samplesCount":60,"isPersistent":true,"lastTStamp":1371685106753,"interval":1000,
"timestamp":[1371685048753,1371685049754,1371685050753,1371685051753,1371685052753,1371685053753,1371685054753,
1371685055753,1371685056753,1371685057753,1371685058752,1371685059753,1371685060753,1371685061753,1371685062753,
1371685063753,1371685064753,1371685065753,1371685066753,1371685067753,1371685068753,1371685069753,1371685070753,
1371685071753,1371685072753,1371685073753,1371685074753,1371685075753,1371685076753,1371685077753,1371685078753,
1371685079753,1371685080753,1371685081753,1371685082753,1371685083753,1371685084753,1371685085753,1371685086753,
1371685087753,1371685088753,1371685089753,1371685090753,1371685091754,1371685092753,1371685093753,1371685094753,
1371685095753,1371685096753,1371685097753,1371685098753,1371685099753,1371685100753,1371685101753,1371685102753,
1371685103753,1371685104753,1371685105753,1371685106753],
"nodeStats":{"127.0.0.1:8091":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
1000000,1000000,1000000,1000000,1000000,1000000,1000000]}}

Retrieving rate_replicated stats

HTTP method and URI

GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

Where the [destination_endpoint] is:

replications/[remote_UUID]/[source_bucket]/[destination_bucket]/rate_replicated

Syntax

Curl request syntax:

curl -u [admin]:[password]
    http://[node address]:8091/pools/default/buckets/[source_bucket]/stats/
    replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2Frate_replicated

Example

Curl request example to get the rate of replication:

curl -u Administrator:password \
    http://10.5.2.54:8091/pools/default/buckets/default/stats/ \
    replications%2F8ba6870d88cd72b3f1db113fc8aee675%2Fdefault%2Fdefault%2Frate_replicated

Response

This produces the following output:

{"samplesCount":60,"isPersistent":true,"lastTStamp":1371685006753,"interval":1000,
"timestamp":[1371684948753,1371684949753,1371684950753,1371684951753,1371684952753,1371684953753,1371684954753,
1371684955754,1371684956753,1371684957753,1371684958753,1371684959753,1371684960753,1371684961753,1371684962753,
1371684963753,1371684964753,1371684965753,1371684966753,1371684967753,1371684968752,1371684969753,1371684970753,
1371684971753,1371684972753,1371684973753,1371684974753,1371684975753,1371684976753,1371684977753,1371684978753,
1371684979753,1371684980753,1371684981753,1371684982753,1371684983753,1371684984753,1371684985754,1371684986753,
1371684987754,1371684988753,1371684989753,1371684990753,1371684991753,1371684992753,1371684993753,1371684994753,
1371684995753,1371684996753,1371684997753,1371684998776,1371684999753,1371685000753,1371685001753,1371685002753,
1371685003753,1371685004753,1371685005753,1371685006753],
"nodeStats":{"127.0.0.1:8091":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}}

Retrieving docs_opt_repd stats

HTTP method and URI

GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

Where the [destination_endpoint] is:

replications/[remote_UUID]/[source_bucket]/[destination_bucket]/docs_opt_repd

Syntax: get replication id

curl -s -u admin:password \
    http://[node address]:8091/pools/default/tasks

Example: get replication id

To get docs_opt_repd, get the replication id for a source and destination bucket via a list of the active tasks for a cluster:

curl -s -u Administrator:password \
    http://10.5.2.54:8091/pools/default/tasks

Response

This results in output as follows:

....
    "id": "def03dbf5e968a47309194ebe052ed21\/bucket_source\/bucket_destination",
    "source": "bucket_source",
    "target":"\/remoteClusters\/def03dbf5e968a47309194ebe052ed21\/buckets\/bucket_name",
    "continuous": true,
    "type": "xdcr",
    ....

Example: get docs_opt_repd stats

With this replication id, retrieve a sampling of stats for docs_opt_repd:

curl -s -u Administrator:password \
http://10.3.121.119:8091/pools/default/buckets/default/stats/ \
replications%2fdef03dbf5e968a47309194ebe052ed21%2fdefault%2fdefault%2fdocs_opt_repd

Response

This results in output similar to the following:

{
       "samplesCount":60,
       "isPersistent":true,
       "lastTStamp":1378398438975,
       "interval":1000,
       "timestamp":[
          1378398380976,
          1378398381976,
          ....

Retrieving incoming write operations

HTTP method and URI

GET /pools/default/buckets/[bucket_name]/stats

Syntax

To retrieve the incoming write operations that occur on a destination cluster due to replication, make the request on your destination cluster.

Curl request syntax:

curl -u [admin]:[password] -X GET
    http://[Destination_IP]:8091/pools/default/buckets/[destination bucket]/stats

Example

Curl request example:

curl -u Administrator:password -X GET \
    http://10.5.2.117:8091/pools/default/buckets/testbucket2/stats

Response

This returns results for all stats. Within the JSON response, find the array xdc_ops. The value for this attribute is the last sampling of write operations on an XDCR destination cluster.

{
.................
"xdc_ops":[0.0,0.0,0.0,0.0,633.3666333666333,1687.6876876876877, \
2610.3896103896104,3254.254254254254,3861.138861138861,4420.420420420421, \
................
}