A newer version of this documentation is available.

View Latest

Getting XDCR Stats

  • reference
    +
    Requests for XDCR statistics about a destination cluster are performed on the source cluster.

    Description

    All XDCR statistical requests use the UUID, a unique identifier for destination cluster. The UUID is retrieved with the GET /pools/default/remoteClusters HTTP method and URI. Many of these statistics are exposed in the Couchbase Web Console.

    You need to provide a properly URL-encoded URI string for the destination endpoint when requesting XDCR statistics.

    HTTP method and URI

    The destination endpoint follows the /pools/default/buckets/[source_bucket]/stats/ URI endpoint:

    GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

    Where the destination endpoint is:

    replications/[remote_UUID]/[source_bucket]/[destination_bucket]/[stat_name]

    Where the HTTP endpoint string with full URI is:

    http://[node address]:[port]/pools/[source_bucket]/buckets/[bucket_name]/stats/replications/ \
      [remote_UUID]/[source_bucket]/[destination_bucket]/[stat_name]

    Where the HTTP string with a properly URL-encoded URI is:

    http://[node address]:[port]/pools/[source_bucket]/buckets/[bucket_name]/stats/ \
        replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2F[stat_name]
    Stat name Description

    bandwidth_usage

    Bandwidth used during replication, measured in bytes per second.

    changes_left

    Number of updates still pending replication.

    data_replicated

    Size of data replicated in bytes.

    docs_checked

    Number of documents checked for changes.

    docs_failed_cr_source

    Number of documents that have failed conflict resolution on the source cluster and not replicated to target cluster.

    docs_filtered

    Number of documents that have been filtered out and not replicated to target cluster.

    docs_opt_repd

    Number of docs sent optimistically.

    docs_received_from_dcp

    Number of documents received from DCP.

    docs_rep_queue

    Number of documents in replication queue.

    docs_processed

    Number of documents processed by pipeline.

    docs_written

    Number of documents written to the destination cluster via XDCR.

    meta_latency_wt

    Weighted average time for requesting document metadata. XDCR uses this for conflict resolution prior to sending the document into the replication queue.

    num_checkpoints

    Number of checkpoints issued in replication queue.

    num_failedckpts

    Number of checkpoints failed during replication.

    rate_received_from_dcp

    Number of documents received from DCP per second.

    rate_replicated

    Rate of documents being replicated, measured in documents per second.

    size_rep_queue

    Size of replication queue in bytes.

    time_committing

    Seconds elapsed during replication.

    wtavg_docs_latency

    Weighted average latency for sending replicated changes to destination cluster.

    wtavg_meta_latency

    Weighted average time for requesting document metadata. XDCR uses this for conflict resolution prior to sending the document into the replication queue.

    Getting destination cluster info

    For example, the following code example displays the destination remote name, URI and UUID (among other data):

    // curl request example for retrieving the destination cluster UUID
    curl -u Administrator:password http://10.5.2.54:8091/pools/default/remoteClusters
    
    
    // example results
    [
        {
            "deleted": false,
            "hostname": "10.5.2.117:8091",
            "name": "Remote117",
            "uri": "/pools/default/remoteClusters/Remote117",
            "username": "Administrator",
            "uuid": "995618a6a6cc9ac79731bd13240e19b5",
            "validateURI": "/pools/default/remoteClusters/Remote117?just_validate=1"
        }

    Retrieving docs_written stats

    HTTP method and URI

    GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

    Where the [destination_endpoint] is:

    replications/[remote_UUID]/[source_bucket]/[destination_bucket]/docs_written

    Syntax

    Curl request syntax for number of documents written:

    curl -u [admin]:[password]
        http://[node address]:8091/pools/default/buckets/[source_bucket]/stats/
        replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2Fdocs_written

    Example

    To get the number of documents written:

    curl -u Administrator:password \
        http://10.5.2.54:8091/pools/default/buckets/default/stats/ \
        replications%2F8ba6870d88cd72b3f1db113fc8aee675%2Fdefault%2Fdefault%2Fdocs_written

    Response

    The above command produces the following output:

    {"samplesCount":60,"isPersistent":true,"lastTStamp":1371685106753,"interval":1000,
    "timestamp":[1371685048753,1371685049754,1371685050753,1371685051753,1371685052753,1371685053753,1371685054753,
    1371685055753,1371685056753,1371685057753,1371685058752,1371685059753,1371685060753,1371685061753,1371685062753,
    1371685063753,1371685064753,1371685065753,1371685066753,1371685067753,1371685068753,1371685069753,1371685070753,
    1371685071753,1371685072753,1371685073753,1371685074753,1371685075753,1371685076753,1371685077753,1371685078753,
    1371685079753,1371685080753,1371685081753,1371685082753,1371685083753,1371685084753,1371685085753,1371685086753,
    1371685087753,1371685088753,1371685089753,1371685090753,1371685091754,1371685092753,1371685093753,1371685094753,
    1371685095753,1371685096753,1371685097753,1371685098753,1371685099753,1371685100753,1371685101753,1371685102753,
    1371685103753,1371685104753,1371685105753,1371685106753],
    "nodeStats":{"127.0.0.1:8091":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
    1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
    1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
    1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000,
    1000000,1000000,1000000,1000000,1000000,1000000,1000000]}}

    The output shows that 60 samples were taken. A sample was taken every 1000 milliseconds, and each sample is represented by its timestamp. 1 million documents are shown already to have been written when the first sample was taken; and this number is shown to have remained consistent at the time of each successive sample.

    Retrieving rate_replicated stats

    HTTP method and URI

    GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

    Where the [destination_endpoint] is:

    replications/[remote_UUID]/[source_bucket]/[destination_bucket]/rate_replicated

    Syntax

    Curl request syntax:

    curl -u [admin]:[password]
        http://[node address]:8091/pools/default/buckets/[source_bucket]/stats/
        replications%2F[remote_UUID]%2F[source_bucket]%2F[destination_bucket]%2Frate_replicated

    Example

    Curl request example to get the rate of replication:

    curl -u Administrator:password \
        http://10.5.2.54:8091/pools/default/buckets/default/stats/ \
        replications%2F8ba6870d88cd72b3f1db113fc8aee675%2Fdefault%2Fdefault%2Frate_replicated

    Response

    This produces the following output:

    {"samplesCount":60,"isPersistent":true,"lastTStamp":1371685006753,"interval":1000,
    "timestamp":[1371684948753,1371684949753,1371684950753,1371684951753,1371684952753,1371684953753,1371684954753,
    1371684955754,1371684956753,1371684957753,1371684958753,1371684959753,1371684960753,1371684961753,1371684962753,
    1371684963753,1371684964753,1371684965753,1371684966753,1371684967753,1371684968752,1371684969753,1371684970753,
    1371684971753,1371684972753,1371684973753,1371684974753,1371684975753,1371684976753,1371684977753,1371684978753,
    1371684979753,1371684980753,1371684981753,1371684982753,1371684983753,1371684984753,1371684985754,1371684986753,
    1371684987754,1371684988753,1371684989753,1371684990753,1371684991753,1371684992753,1371684993753,1371684994753,
    1371684995753,1371684996753,1371684997753,1371684998776,1371684999753,1371685000753,1371685001753,1371685002753,
    1371685003753,1371685004753,1371685005753,1371685006753],
    "nodeStats":{"127.0.0.1:8091":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}}

    Retrieving docs_opt_repd stats

    HTTP method and URI

    GET /pools/default/buckets/[source_bucket]/stats/[destination_endpoint]

    Where the [destination_endpoint] is:

    replications/[remote_UUID]/[source_bucket]/[destination_bucket]/docs_opt_repd

    Syntax: get replication id

    curl -s -u admin:password \
        http://[node address]:8091/pools/default/tasks

    Example: get replication id

    To get docs_opt_repd, get the replication id for a source and destination bucket via a list of the active tasks for a cluster:

    curl -s -u Administrator:password \
        http://10.5.2.54:8091/pools/default/tasks

    Response

    This results in output as follows:

    ....
        "id": "def03dbf5e968a47309194ebe052ed21\/bucket_source\/bucket_destination",
        "source": "bucket_source",
        "target":"\/remoteClusters\/def03dbf5e968a47309194ebe052ed21\/buckets\/bucket_name",
        "continuous": true,
        "type": "xdcr",
        ....

    Example: get docs_opt_repd stats

    With this replication id, retrieve a sampling of stats for docs_opt_repd:

    curl -s -u Administrator:password \
    http://10.3.121.119:8091/pools/default/buckets/default/stats/ \
    replications%2fdef03dbf5e968a47309194ebe052ed21%2fdefault%2fdefault%2fdocs_opt_repd

    Response

    This results in output similar to the following:

    {
           "samplesCount":60,
           "isPersistent":true,
           "lastTStamp":1378398438975,
           "interval":1000,
           "timestamp":[
              1378398380976,
              1378398381976,
              ....

    Retrieving incoming write operations

    HTTP method and URI

    GET /pools/default/buckets/[bucket_name]/stats

    Syntax

    To retrieve the incoming write operations that occur on a destination cluster due to replication, make the request on your destination cluster.

    Curl request syntax:

    curl -u [admin]:[password] -X GET
        http://[Destination_IP]:8091/pools/default/buckets/[destination bucket]/stats

    Example

    Curl request example:

    curl -u Administrator:password -X GET \
        http://10.5.2.117:8091/pools/default/buckets/testbucket2/stats

    Response

    This returns results for all stats. Within the JSON response, find the array xdc_ops. The value for this attribute is the last sampling of write operations on an XDCR destination cluster.

    {
    .................
    "xdc_ops":[0.0,0.0,0.0,0.0,633.3666333666333,1687.6876876876877, \
    2610.3896103896104,3254.254254254254,3861.138861138861,4420.420420420421, \
    ................
    }