Manage Replications

Capella Operational

Configure XDCR to replicate data between source and destination buckets. An XDCR replication allows bucket data to be replicated continuously from a specified bucket on a source cluster to a specified bucket on a target cluster.

Use the procedures on this page to create and manage XDCR (Cross Data Center Replication) between Capella clusters. For general information on XDCR in Capella and how it works, see Cross Data Center Replication (XDCR).

Accessing Replications

Replications, which can be viewed and managed by users with the Project Owner role, can be accessed on the source cluster’s Settings screen, by clicking on the Replication tab, which is located in the navigation panel, at the left. This brings up the Replication screen:

If no replication has yet been set up, a button for starting setup is displayed, in the upper, Replication area. Note that the contents of the lower, Self-Managed Targets area are described below, in Create a Replication from Capella to a Self-Managed Cluster.

If replications have already been defined for cluster, a summary of these is displayed in table format. For an example, see Observe an Ongoing Replication, below.

To create a replication, click on the Set Up Replication button:

Replication Setup

After the Setup Replication button has been clicked, the Setup screen appears.

Create a Replication

An XDCR replication is created on the cluster that will be the source of the data replication. This cluster is known as the source cluster, and it contains the data you want to replicate. Each replication targets a bucket on a different cluster, known as the destination cluster or target cluster, in which the data will be replicated.

To create a replication, you must have the Project Owner role for the projects that contain both the source and destination clusters.

Private Link connections do not support XDCR.

Clusters hosted by Couchbase Capella support the following replications:

Own Cloud (AWS) → Capella Hosted (AWS)
Capella Hosted (AWS) ←→ Capella Hosted (AWS)
Own Cloud (GCP) → Capella Hosted (GCP)
Capella Hosted (GCP) ←→ Capella Hosted (GCP)
Own Cloud (Azure) → Capella Hosted (Azure)
Capella Hosted (Azure) ←→ Capella Hosted (Azure)

Note that source and destination buckets must have the same conflict resolution method configured. If the buckets use different conflict resolution methods, then the replication will fail to be created.

A replication can be created from one Capella cluster to any other. This is described immediately below, in Create a Replication Between Capella Clusters.

A replication can also be created from an on-premises (or self-managed) cluster to a Capella cluster: this procedure is described below, in Create a Replication to Capella from a Self-Managed Cluster.

Create a Replication Between Capella Clusters

To create a replication using Capella clusters, proceed as follows:

Go to Select Replication Direction. Choose to create a One-way or a Two-way replication, by selecting the appropriate option. A one-way replication replicates data only from the source to the target. A two-way replication replicates data both from the source to the target, and from the target to the source. For example, to define a one-way replication, select One-way:
Go to Select Clusters.

The source cluster is already determined by the cluster on which you’re currently configuring the replication. The target cluster must be selected from the Target Cluster menu.
Go to Select Buckets. Each replication proceeds from a source bucket to a target bucket. Select these buckets from the Source Bucket and Target Bucket menus.

For example, if the source cluster and bucket are respectively named db-1 and travel-sample, and the target cluster and bucket are named db-2 and travel-sample-backup, the UI should be configured as follows:
Go to Filter Replication. To apply a filter to the replication, select Enable.

This displays an interactive field into which the filter expression can be entered.

To ensure that only documents that contain the value France for the key country are replicated, enter the expression REGEXP_CONTAINS(country, "France"). For example:

Note that the expression must contain at least two keys. An error-notification appears on the console, if the expression is in any way invalid.

Check the syntax of your expression, by clicking on the Check Syntax button. If the syntax is correct, this is indicated by a notification on the UI. If the syntax is incorrect, an error is flagged.

Next, optionally, test the expression against a specific document in the source bucket, by clicking on the control at the right of the Test Document panel. This expands as follows;

Using the controls at the right of each of the fields, select an appropriate scope, collection, and document ID from the pulldown menus provided. When completed, the panel appears as follows;

Click on Test Document to run the test. If the filter provides a match with the specified document, a success message is displayed. Otherwise, a failure message is displayed.
Choose your Deletion Filters.

For more information about deletion filters, see Deletion Filters in the Server documentation.
If your Capella cluster is on Couchbase Server version 7.2.2 or later, choose whether to Filter Binary Documents.

For more information about filtering binary documents from XDCR, see Filtering Binary Documents in the Server documentation.
Go to Select Replicaton Priority.

Select High, Medium, or Low. The meaning of each option is described in the UI: for more detailed information, see XDCR Priority. Note that a setting only takes effect if there are multiple replications with different priorities.
Go to Set Network Usage Limit. This option allows an upper limit for network usage to be established. To establish an upper limit, select Enable. This displays an interactive field, into which the limit, in MiB per second, should be entered:

Note that an established limit applies to all replications for the entire cluster. If no limit need be established, select Disable.
Go to Replicate All Scopes and Collections. If the replication is intended to replicate to the target all scopes and collections that have been defined on the source, allow the default Yes setting to remain. Note that for scopes and collections to be replicated in this way, each scope and collection must already exist with the same name on source and target.

If the replication is intended to replicate documents to a differently named combination of scope and collection on the target, the target scope and collection can be individually specified. To do this, select the No setting. The UI now presents an interactive Add Scope and Collection tab. When this is clicked on, the UI expands as follows:

Two interactive fields are now displayed, allowing a scope and collection to be specified for both source and target. When clicked with the mouse, each field presents a menu, from which selections can be made. The first selection is for a scope. For example, the scope on the source can be selected as follows:

After a scope has been selected, a further menu allows selection of a collection within the selected scope. For example:

Once appropriate selections have been made for source and target, the Replication All Scopes and Collections UI might appear as follows:
Click on the Setup Replication button, at the bottom of the screen, to start the replication:

Notifications now appear, explaining the some time must now elapse, to allow setup to be performed. Once setup is complete, details of the new, ongoing replication are displayed in the UI.

Observe an Ongoing Replication

Once one or more replications are underway, these are displayed on the Replications screen. For example:

The 'Replications' screen, showing an ongoing replication.

The process for defining an additional replication can be started by clicking the Setup Replication button.

Pause and Resume a Replication

Pausing an XDCR replication temporarily suspends the replication of data from the source to the target. Pausing a replication always occurs on the source cluster. Subsequent to being paused, a replication can be resumed.

To pause a replication, you must have the Project Owner role for the project that contains the source cluster. Note that when pausing a bidirectional replication, only the replication from the current cluster will be paused. To pause both directions of a bidirectional replication, you will need to pause the replication on both clusters individually.

Proceed as follows:

On the Replications screen, identify the ongoing replication that you wish to pause.
Click on the name of the source bucket for the replication. A summary of the replication is now displayed. For example:
Locate the Pause button at the upper right. To pause the replication, click on the Pause button.

The replication is now paused. This is reflected in the upper area of the screen, which appears as follows:

Note that the button previously labeled Pause is now labeled Resume. To resume the replication, click the Resume button.

Modify a Replication

Replications cannot be modified after creation. Desired modifications must be configured for an entirely new replication: with the old replication being deleted, if appropriate.

Delete a Replication

Deleting an XDCR replication stops the replication of data, and removes the defined replication from the cluster. Any replicated data will not be deleted.

To delete a replication, you must have the Project Owner role for the projects that contain both the source and destination clusters. Note that if the replication is bidirectional, when the replication is deleted on the source, both directions of the replication are thereby deleted, and are removed from the Replications tab of both clusters.

Proceed as follows:

On the Replications screen, identify the ongoing replication that you wish to delete.
Click on the name of the source bucket for the replication. A summary of the replication is now displayed. For example:
Locate the Delete button at the upper right. To delete the replication, click on the Delete button.

A dialog appears, requesting confirmation:

To confirm, enter the word delete into the interactive field. Then click on Delete Replication.

The replication is now deleted, and is removed from the list of ongoing replications.

Create a Replication to Capella from a Self-Managed Cluster

You can create an XDCR replication to a Capella cluster from a self-managed cluster that’s in an on-premises data center or a non-Capella cloud.

If the replication goes across the public Internet, you must add the IP address of the self-managed cluster Data Service nodes to the Allowed IP list of the Capella cluster, as shown in Step 2 of this procedure. For more information about adding IPs to the Allowed IP list, see Configure Allowed IP Addresses.

You do not need to do this if the replication travels a private network. For example, if you peered the self-managed cluster’s VPC with the Capella cluster’s VPC. In such cases, skip Step 2 of this procedure.

To set up XDCR from an on-premises cluster to a Capella cluster:

Create a cluster access credential in your Capella cluster.

For more information about creating cluster access credentials, see Configure Cluster Access Credentials.
If the replication crosses the public Internet, allowlist the public IP of the VM:
1. Get the public IP from your VM.
  
  ssh into the VM where Couchbase Server is running.
  # dig +short myip.opendns.com @resolver1.opendns.com
  67.212.150.204
2. Allowlist this IP in the Capella cluster:
  
  With your cluster open in Capella, go to Settings Allowed IP Addresses Add Allowed IP.
  
  For more information about adding IPs to the Allowed IP list, see Configure Allowed IP Addresses.

Get the hostname to use in the XDCR remote cluster reference.

.In SDKs, copy the Public Connection String without the couchbases:// prefix.

The public connection string is also the DNS SRV of the cluster.

Although the connection string is sometimes referred to as the "public connection string," if you have VPC peering set up the connection string resolves to private addresses. For more information, see Configure a VPC Peering Connection.

Copy the security certificate from the Capella cluster:

With your cluster open in Capella, go to Settings Security Certificate Copy

Set up XDCR Remote Reference.

XDCR compatibility can vary between different versions of Couchbase Enterprise Server. To view and confirm compatibility, see XDCR Compatibility.

In your on-premises cluster Couchbase Server Web Console:

Couchbase Server 7.X or later
Couchbase Server 6.6.X

If your self-managed cluster uses Couchbase Server 7.X or later, fill out the Add Remote Cluster dialog as follows:

Cluster Name

Any name

IP/Hostname

The connection string from Step 3 without the couchbases:// prefix.

As the hostname saves as a DNS SRV string in Couchbase Server 7.X and later, you do not need to specify a port.

For example: cb.q2pi8ahm18gn4w2p.cloud.couchbase.com

Username for Remote Cluster

Cluster credentials from Step 1.

Password

Cluster credentials from Step 1.

Certificate

Paste the security certificate copied in Step 4.

If your self-managed cluster uses Couchbase Server 6.6.X, fill out the Add Remote Cluster dialog as follows:


Cluster Name	Any name
IP/Hostname	The connection string from Step 3 without the `couchbases://` prefix. Couchbase Server versions previous to Couchbase Server 7.x cannot use DNS SRV for the XDCR remote reference. Specify the port number `18091` with the Capella connection string so that XDCR resolves it to one of the Capella cluster Data Service node hostnames. For example: `cb.q2pi8ahm18gn4w2p.cloud.couchbase.com:18091`
Username for Remote Cluster	Cluster credentials from Step 1.
Password	Cluster credentials from Step 1.
Certificate	Paste the security certificate copied in Step 4.

Cluster Name

Any name

IP/Hostname

The connection string from Step 3 without the couchbases:// prefix.

Couchbase Server versions previous to Couchbase Server 7.x cannot use DNS SRV for the XDCR remote reference. Specify the port number 18091 with the Capella connection string so that XDCR resolves it to one of the Capella cluster Data Service node hostnames.

For example: cb.q2pi8ahm18gn4w2p.cloud.couchbase.com:18091

Username for Remote Cluster

Cluster credentials from Step 1.

Password

Cluster credentials from Step 1.

Certificate

Paste the security certificate copied in Step 4.

Set up XDCR Replication.

Using the Couchbase Server Web Console for your on-premises cluster:

Then:

Create a Replication from Capella to a Self-Managed Cluster

A self-managed cluster is a cluster that is outside Capella. Data can be replicated from Capella to a self-managed cluster. To configure the replication, the Capella administrator must have the following:

The CA of the self-managed cluster.
Either of the following:
- The username and password for the self-managed cluster.
- A client certificate and private key provided for client access to the self-managed cluster.

Additionally, the self-managed cluster must be network-accessible to Capella. Ensure that:

The target cluster is accessible via the SSL ports 18091, 18092, and 11207.
Each Data-Service node on the Capella cluster can connect to each Data-Service node on the self-managed target.
All target-cluster firewalls allow access to the Capella Data-Service nodes.

To replicate data from Capella to a self-managed cluster; first, add the self-managed cluster as a target; then, select the self-managed cluster to be the target for a specific replication. These steps are described below. Note, however, that if the self-managed target is to be accessed over a private network, VPC Peering should be established for the connection, before the self-managed target is added. For information, see Configure a Private Network.

Add a Self-Managed Target

To add a self-managed target, proceed as follows:

Access the Replication screen. Under the main area provided for configuring replications, the Self-Managed Targets area is visible. Here, if no self-managed target has yet been added, the No Self-Managed Targets Found notification is displayed; and under this, the Add Self-Managed Target button.

Click on the Add Self-Managed Target button. This brings up the Add Self-Managed Target screen.
Take note of the information provided at the top of the screen: this contains essential requirements for establishing a replication from Capella to a self-managed target.
Enter a name for the self-managed target into the Target Name field.
Enter the IP address or Fully Qualified Domain Name of the self-managed target into the IP/Hostname field.
Optionally, add the appropriate administrative username and password for the self-managed target into the Username for Self Managed Target and Password fields. A username and password must be added unless you intend to authenticate by means of a client certificate and private key (described below).
Specify a Network Type, by selecting the appropriate radio button. This determines whether the internal or alternate network address of a target is used, in cases where the target can be contacted by means of either. When selected, the buttons Force using alternate address and Force using internal address each ensure that only its corresponding option can be used for establishing this replication. When selected, the button Auto ensures that Capella itself will make the decision.

For information on specifying internal and alternate addresses, see Using Alternate Addresses and Specifying Addresses.
Paste the CA of the self-managed target into the TLS Certificate field.
If you wish Capella to authenticate by means of a client certificate and private key, check the User Client Certificate Authentication checkbox. This displays the interactive fields Client Certificate and RSA Private Key, into which the certificate and key should be pasted. Note that checking this checkbox deactivates the Username and Password fields: anything previously entered into those fields will consequently be ignored.

If you wish to authenticate with a username and password, leave the User Client Certificate Authentication checkbox unchecked.
Click on the Add Self-Managed Target button. The self-managed target is now added to Capella.
The Replication screen is now displayed. The newly added self-managed target is now listed in the Self-Managed Targets area.

Replicate to a Self-Managed Target

After a self-managed cluster has been established as a target for Capella, a replication from Capella to the self-managed target can be configured. Proceed as follows:

On the Replication screen, click on the Setup Replication button. This brings up the Set Up Cross Data Center Replication screen.
Use the Select Replication Direction selector to establish One-Way as the replication direction to the self-managed targeted. Note that two-way replication cannot be used, when the target is self-managed.
Select the target cluster by accessing the pull-down menu in the Target Cluster field. Note that clusters in the menu are divided into two groups, which are Self-Managed and Capella Managed. The self-managed cluster that will be the target of the replication appears in the menu under the Self-Managed heading.
In the Select Buckets fields, add the names of the source and target buckets.
Complete the setup by appropriately configuring the sections Filter Documents, Select Replication Priority, Set Network Usage Limit, and Replicate All Scopes and Collections (as described previously, on this page).
Click on the Set Up Replication button. Capella now sets up the specified replication.

Error and Other Notifications

If a connectivity issue arises when replicating to a self-managed target, the issue is indicated in the Self-Managed Targets area of the Replication page: the notification One or more nodes has connectivity issues appears under the Connectivity Status column for the self-managed target:

Issues with connecting to self-managed target.

The replication itself is also represented in the list of all ongoing replications. A yellow circle, with an integer inside, appears in the Errors column, to indicate that a number of errors have occurred:

Click on the circle to see the errors: this displays the Replication Errors panel. Each error is listed, with its accompanying text.

Note that not all of the listings represent errors: some listings may be warnings or informational.