Managing Advanced XDCR Settings

reference

XDCR advanced settings affect replications' behavior, performance, and timing.

HTTP method and URI

POST /settings/replications/

POST /settings/replications/[settings_URI]

GET /settings/replications/

GET /settings/replications/[settings_URI]

Description

Used with the POST method, the URIs respectively change global settings for all replications; and for a specific replication, which is referenced by its settings_URI. The settings_URI comprises the id for the replication.

The global settings are the default values used if settings are not specified during the creation of a particular replication. If settings are specified for a particular replication, the specified settings overwrite the global settings.

If the global settings are themselves changed, existing replications are not affected: only replications created after the change made to the global settings receive the updated global settings.

Used with the GET method, the URIs respectively retrieve global settings for all replications; and for a specific replication, which is referenced by its settings_URI.

Curl Syntax

curl -u [admin]:[password] -X POST \
http://[localhost]:8091/settings/replications

curl -u [admin]:[password] -X POST \
http://[localhost]:8091/settings/replications/[settings_URI]

curl -u [admin]:[password] -X GET \
http://[localhost]:8091/settings/replications

curl -u [admin]:[password] -X GET \
http://[localhost]:8091/settings/replications/[settings_URI]

Responses

If successful, GET and POST commands return 200 OK. Commands for establishing or retrieving either global settings or the settings for a specific replication or return an object containing the settings.

Failure to authenticate returns 401 Unauthorized. An URI featuring an incorrect hostname or port returns a failure to connect message. An incorrect settings_URI returns a 500 Internal Server Error.

An attempt to set a global-only setting for a specific replication returns 400 Bad Request, and an object indicating which submitted parameters were incorrect. For example, the attempted use of POST /settings/replications/[settings_URI] to establish values for goGC and goMaxProcs returns the following:

{
  "gogc": "Invalid settings key",
  "gomaxprocs": "Invalid settings key"
}

A replication-specific parameter (such as pauseRequested) is ignored, if incorrectly specified in an attempt to modify global settings.

Examples

The following examples show how settings can be established and retrieved.

Get Settings for All Replications

The following example retrieves settings for all replications:

curl -u Administrator:password 10.5.2.54:8091/settings/replications

If successful, the call returns an object similar to the following:

{
  "checkpointInterval": 600,
  "compressionType": "Auto",
  "desiredLatency": 50,
  "docBatchSizeKb": 2048,
  "failureRestartInterval": 10,
  "filterBypassExpiry": false,
  "filterDeletion": false,
  "filterExpiration": false,
  "goGC": 100,
  "goMaxProcs": 5,
  "logLevel": "Info",
  "networkUsageLimit": 0,
  "optimisticReplicationThreshold": 256,
  "priority": "High",
  "sourceNozzlePerNode": 2,
  "statsInterval": 1000,
  "targetNozzlePerNode": 2,
  "workerBatchSize": 500
}

Get Settings for a Specific Replication

To reference a specific replication, the replication’s settings_URI must first be retrieved. Use the following call, which is explained in detail in Getting Cluster Tasks. Note that the call pipes output to the jq program, to facilitate the readability of output.

curl -u Administrator:password -v -X GET \
http://10.143.192.101:8091/pools/default/tasks | jq '.'

If successful, the call returns an array of ongoing cluster tasks, similar to the following:

[
  {
    "statusId": "9dd69ac5b236b7d176540a7d797058a6",
    "type": "rebalance",
    "status": "notRunning",
    "statusIsStale": false,
    "masterRequestTimedOut": false
  },
  {
    "cancelURI": "/controller/cancelXDCR/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts",
    "settingsURI": "/settings/replications/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts",
    "status": "running",
    "replicationType": "xmem",
    "continuous": true,
    "filterExpression": "",
    "id": "9c1eff7bd61abea11adacc447d34edcc/travel-sample/ts",
    "pauseRequested": false,
    "source": "travel-sample",
    "target": "/remoteClusters/9c1eff7bd61abea11adacc447d34edcc/buckets/ts",
    "type": "xdcr",
    "recommendedRefreshPeriod": 10,
    "changesLeft": 0,
    "docsChecked": 0,
    "docsWritten": 21130,
    "maxVBReps": null,
    "errors": []
  },
  {
    "cancelURI": "/controller/cancelXDCR/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts2",
    "settingsURI": "/settings/replications/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts2",
    "status": "running",
    "replicationType": "xmem",
    "continuous": true,
    "filterExpression": "",
    "id": "9c1eff7bd61abea11adacc447d34edcc/travel-sample/ts2",
    "pauseRequested": false,
    "source": "travel-sample",
    "target": "/remoteClusters/9c1eff7bd61abea11adacc447d34edcc/buckets/ts2",
    "type": "xdcr",
    "recommendedRefreshPeriod": 10,
    "changesLeft": 0,
    "docsChecked": 0,
    "docsWritten": 21130,
    "maxVBReps": null,
    "errors": []
  }
]

A settings_URI if provided for each of two ongoing replications, within the second and third members of the array. Each settings_URI can be used to retrieve information on its corresponding replication.

For example, enter the following:

curl -X GET -u Administrator:password \ http://10.143.192.101:8091/settings/replications/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts | jq '.'

If the call is successful, an object containing the settings for the specified replication is returned:

{
  "checkpointInterval": 600,
  "compressionType": "Auto",
  "desiredLatency": 50,
  "docBatchSizeKb": 2048,
  "failureRestartInterval": 10,
  "filterBypassExpiry": false,
  "filterDeletion": false,
  "filterExpiration": false,
  "filterExpression": "",
  "logLevel": "Info",
  "networkUsageLimit": 0,
  "optimisticReplicationThreshold": 256,
  "pauseRequested": false,
  "priority": "High",
  "sourceNozzlePerNode": 2,
  "statsInterval": 1000,
  "targetNozzlePerNode": 2,
  "type": "xmem",
  "workerBatchSize": 500
}

Change a Setting for a Specific Replication

The following changes the checkpointInterval for a specific replication:

curl -v -X POST -u Administrator:password http://10.143.192.101:8091/settings/replications/9c1eff7bd61abea11adacc447d34edcc%2Ftravel-sample%2Fts -d checkpointInterval=700 | jq '.'

If successful, the call returns an object containing all current settings for the replication, including what has been changed:

{
  "checkpointInterval": 700,
  "compressionType": "Auto",
  "desiredLatency": 50,
  "docBatchSizeKb": 2048,
  "failureRestartInterval": 10,
  "filterBypassExpiry": false,
  "filterDeletion": false,
  "filterExpiration": false,
  "filterExpression": "",
  "logLevel": "Info",
  "networkUsageLimit": 0,
  "optimisticReplicationThreshold": 256,
  "pauseRequested": false,
  "priority": "High",
  "sourceNozzlePerNode": 2,
  "statsInterval": 1000,
  "targetNozzlePerNode": 2,
  "type": "xmem",
  "workerBatchSize": 500
}

XDCR Advanced Settings

The advanced settings for XDCR are as follows. Note that the subset of advanced settings that can be configured by means of Couchbase Web Console is provided at XDCR Advanced Settings.

Table 1. XDCR Advanced Settings
Parameter	Value	Description
`checkpointInterval`	Integer (60 to 14400).	Default: 600. The interval for checkpointing in seconds.
`compressionType`	String	Default: `Auto`. Specifies whether documents are to be compressed for XDCR, and if so, what compression type is to be used. For information, see XDCR Advanced Settings.
`desiredLatency`	Integer	Specifies the amount of time in milliseconds within which a high-priority replication’s currently remaining changes should be reduced in number to zero, by ongoing transmission of the changes from source to target bucket. If Couchbase Server estimates that the time required is greater than that specified by `desiredLatency`, the replication is considered to be backlogged: XDCR therefore attempts to communicate with the Data Service, to establish a higher DCP priority for the replication, and thereby ensure swifter transmission of data. This setting applies only to high-priority replications, or to medium-priority replications that achieve high-priority status (see XDCR Priority). The default value is 50. The lower the value, the higher (potentially) the transmission rate, and the greater the load on the target cluster.
`docBatchSizeKb`	Integer (10 to 10000)	Default: 2048. The size of a batch in kilobytes.
`failureRestartInterval`	Integer (1 to 300)	Default: 10. The number of seconds to wait after a failure before restarting replication.
`filterBypassExpiry`	Boolean (true or false)	Default: false. Whether a document’s TTL should be replicated with the document or not. A value of `true` means that the TTL is removed from the document.
`filterDeletion`	Boolean (true or false)	Default: false. Whether mutations corresponding to the deletion of documents on the source cluster should be either filtered out of the replication to the target cluster, or allowed to remain in. A value of `true` means that the mutation is filtered out, ensuring that it is not replicated to the target cluster; while a value of `false` means that the mutation is not filtered out, ensuring that it is replicated to the target cluster.
`filterExpiration`	Boolean (true or false)	Default: false. Whether mutations corresponding to the expiration of documents on the source cluster should be either filtered out of the replication to the target cluster, or allowed to remain in. A value of `true` means that the mutation is filtered out, ensuring that it is not replicated to the target cluster; while a value of `false` (the default) means that the mutation is not filtered out, ensuring that it is replicated to the target cluster.
`filterExpression`	String (a filter expression)	Default: null. A filter expression to be matched against the ids, field-names, values, and extended attributes of documents in the source bucket. Each document that produces a successful match is replicated; other documents are not replicated. For supported expressions, see XDCR Advanced Filtering Reference. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings.
`goGC`	Integer (0 to 100) or String (`off`)	Default: 100. Rhe initial garbage collection target percentage for the replication. A garbage collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage. A value of `off` disables the garbage collector entirely. This setting is only returned when global settings are retrieved; and can only be set when global settings are set. It cannot be set when an individual replication is created, or when its settings are modified.
`goMaxProcs`	Integer	Default: 4. The maximum number of threads used per node, to support XDCR. A greater number of threads increases parallelism, and may thereby produce enhanced XDCR performance. This setting is only returned when global settings are retrieved; and can only be set when global settings are set. It cannot be set when an individual replication is created, or when its settings are modified.
`logLevel`	String	Default: `Info`. The level of logging, such as `Error`/`Info`/`Debug`/`Trace`.
`networkUsageLimit`	Integer	Default: 0. Specifies the upper limit for network usage during replication, for the entire cluster, in MB per second. The default is 0, meaning no limit is applied. For information, see XDCR Advanced Settings.
`optimisticReplicationThreshold`	Integer (0 to (2010241024))	Default: 256. Documents with sizes less than this threshold (in bytes) will be replicated optimistically.
`pauseRequested`	Boolean (true or false)	Default: false. Indicates whether the replication has been issued a pause request. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings.
`priority`	`High`, `Medium`, or `Low`	Default: `High`. Resource-allocation for the replication. For information, see XDCR Priority.
`sourceNozzlePerNode`	Integer (1 to 100)	The number of nozzles that can be used for this replication per source cluster node. This together with `target_nozzle_per_node` controls the parallelism of the replication.
`statsInterval`	Integer (200 to 600000)	Default: 1000. The interval (in milliseconds) for statistics updates.
`targetNozzlePerNode`	Integer (1 to 100)	The number of outgoing nozzles per target node. This together with `source_nozzle_per_node` controls the parallelism of the replication.
`type`	String (`capi` or `xmem`)	Default: `xmem`. Note that `capi` and `xmem`, are respectively represented by Version 1 and Version 2, in Couchbase Web Console. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings.
`workerBatchSize`	Integer (500 to 10000)	Default: 500. The number of mutations in a batch.