Manage Replications

Configure XDCR to replicate data between source and destination buckets. An XDCR replication allows bucket data to be replicated continuously from a specified bucket on a source database to a specified bucket on a target database.

Use the procedures on this page to create and manage XDCR (Cross Data Center Replication) between Capella databases. For general information on XDCR in Capella and how it works, see Cross Data Center Replication (XDCR).

Accessing Replications

Replications, which can be viewed and managed by users with the Project Owner role, can be accessed on the source database’s Settings screen, by clicking on the Replication tab, which is located in the navigation panel, at the left. This brings up the Replication screen:

If no replication has yet been set up, a button for starting setup is displayed, in the upper, Replication area. Note that the contents of the lower, Self-Managed Targets area are described below, in Create a Replication from Capella to a Self-Managed Cluster.

If replications have already been defined for database, a summary of these is displayed in table format. For an example, see Observe an Ongoing Replication, below.

To create a replication, click on the Set Up Replication button:

Replication Setup

After the Setup Replication button has been clicked, the Setup screen appears.

Create a Replication

An XDCR replication is created on the cluster that is to be the source of the data replication. This cluster is known as the source cluster, and it contains the data you want to replicate. Each replication targets a bucket on a different cluster, known as the destination cluster or target cluster, in which the data will be replicated.

To create a replication, you must have the Project Owner role for the projects that contain both the source and destination clusters.

Private Link connections do not support XDCR.

Clusters hosted by Couchbase Capella support the following replications:

Own Cloud (AWS) → Capella Hosted (AWS)
Capella Hosted (AWS) ←→ Capella Hosted (AWS)
Own Cloud (GCP) → Capella Hosted (GCP)
Capella Hosted (GCP) ←→ Capella Hosted (GCP)
Own Cloud (Azure) → Capella Hosted (Azure)
Capella Hosted (Azure) ←→ Capella Hosted (Azure)

Note that source and destination buckets must have the same conflict resolution method configured. If the buckets use different conflict resolution methods, then the replication will fail to be created.

A replication can be created from one Capella database to any other. This is described immediately below, in Create a Replication Between Capella Databases.

A replication can also be created from an on-premises (or self-managed) cluster to a Capella cluster: this procedure is described below, in Create a Replication to Capella from a Self-Managed Cluster.

Create a Replication Between Capella Databases

To create a replication using Capella databases, proceed as follows:

Go to Select Replication Direction. Choose to create a One-way or a Two-way replication, by selecting the appropriate option. A one-way replication replicates data only from the source to the target. A two-way replication replicates data both from the source to the target, and from the target to the source. For example, to define a one-way replication, select One-way:
Go to Select Databases.

The source database is already determined by the database on which you are currently configuring the replication. The target database must be selected from the Target Database menu.
Go to Select Buckets. Each replication proceeds from a source bucket to a target bucket. Select these buckets from the Source Bucket and Target Bucket menus.

For example, if the source database and bucket are respectively named db-1 and travel-sample, and the target database and bucket are named db-2 and travel-sample-backup, the UI should be configured as follows:
Go to Filter Replication. To apply a filter to the replication, select Enable.

This displays an interactive field into which the filter expression can be entered.

To ensure that only documents that contain the value France for the key country are replicated, enter the expression REGEXP_CONTAINS(country, "France"). For example:

Note that the expression must contain at least two keys. An error-notification appears on the console, if the expression is in any way invalid.

Check the syntax of your expression, by clicking on the Check Syntax button. If the syntax is correct, this is indicated by a notification on the UI. If the syntax is incorrect, an error is flagged.

Next, optionally, test the expression against a specific document in the source bucket, by clicking on the control at the right of the Test Document panel. This expands as follows;

Using the controls at the right of each of the fields, select an appropriate scope, collection, and document ID from the pulldown menus provided. When completed, the panel appears as follows;

Click on Test Document to run the test. If the filter provides a match with the specified document, a success message is displayed. Otherwise, a failure message is displayed.
Select Deletion Filters. The panel appears as follows:

The basic meaning of each option is described in the UI: for more detailed information, see Using Deletion Filters.
Go to Select Replicaton Priority.

Select High, Medium, or Low. The meaning of each option is described in the UI: for more detailed information, see XDCR Priority. Note that a setting only takes effect if there are multiple replications with different priorities.
Go to Set Network Usage Limit. This option allows an upper limit for network usage to be established. To establish an upper limit, select Enable. This displays an interactive field, into which the limit, in MiB per second, should be entered:

Note that an established limit applies to all replications for the entire database. If no limit need be established, select Disable.
Go to Replicate All Scopes and Collections. If the replication is intended to replicate to the target all scopes and collections that have been defined on the source, allow the default Yes setting to remain. Note that for scopes and collections to be replicated in this way, each scope and collection must already exist with the same name on source and target.

If the replication is intended to replicate documents to a differently named combination of scope and collection on the target, the target scope and collection can be individually specified. To do this, select the No setting. The UI now presents an interactive Add Scope and Collection tab. When this is clicked on, the UI expands as follows:

Two interactive fields are now displayed, allowing a scope and collection to be specified for both source and target. When clicked with the mouse, each field presents a menu, from which selections can be made. The first selection is for a scope. For example, the scope on the source can be selected as follows:

After a scope has been selected, a further menu allows selection of a collection within the selected scope. For example:

Once appropriate selections have been made for source and target, the Replication All Scopes and Collections UI might appear as follows:
Click on the Setup Replication button, at the bottom of the screen, to start the replication:

Notifications now appear, explaining the some time must now elapse, to allow setup to be performed. Once setup is complete, details of the new, ongoing replication are displayed in the UI.

Observe an Ongoing Replication

Once one or more replications are underway, these are displayed on the Replications screen. For example:

The 'Replications' screen, showing an ongoing replication.

The process for defining an additional replication can be started by clicking the Setup Replication button.

Pause and Resume a Replication

Pausing an XDCR replication temporarily suspends the replication of data from the source to the target. Pausing a replication always occurs on the source cluster. Subsequent to being paused, a replication can be resumed.

To pause a replication, you must have the Project Owner role for the project that contains the source cluster. Note that when pausing a bidirectional replication, only the replication from the current cluster will be paused. To pause both directions of a bidirectional replication, you will need to pause the replication on both clusters individually.

Proceed as follows:

On the Replications screen, identify the ongoing replication that you wish to pause.
Click on the name of the source bucket for the replication. A summary of the replication is now displayed. For example:
Locate the Pause button at the upper right. To pause the replication, click on the Pause button.

The replication is now paused. This is reflected in the upper area of the screen, which appears as follows:

Note that the button previously labeled Pause is now labeled Resume. To resume the replication, click the Resume button.

Modify a Replication

Replications cannot be modified after creation. Desired modifications must be configured for an entirely new replication: with the old replication being deleted, if appropriate.

Delete a Replication

Deleting an XDCR replication stops the replication of data, and removes the defined replication from the cluster. Any replicated data will not be deleted.

To delete a replication, you must have the Project Owner role for the projects that contain both the source and destination clusters. Note that if the replication is bidirectional, when the replication is deleted on the source, both directions of the replication are thereby deleted, and are removed from the Replications tab of both clusters.

Proceed as follows:

On the Replications screen, identify the ongoing replication that you wish to delete.
Click on the name of the source bucket for the replication. A summary of the replication is now displayed. For example:
Locate the Delete button at the upper right. To delete the replication, click on the Delete button.

A dialog appears, requesting confirmation:

To confirm, enter the word delete into the interactive field. Then click on Delete Replication.

The replication is now deleted, and is removed from the list of ongoing replications.

Create a Replication to Capella from a Self-Managed Cluster

You can create an XDCR replication to a Capella cluster from a self-managed cluster that is located either in an on-prem data center, or on a non-Capella cloud. If the replication goes across the public internet, you must add the IP addresses of the self-managed cluster Data-Service nodes to the Allowed IP list of the Capella cluster (this is covered in step 2 of the instructions below — for further information, see Configure Allowed IP Addresses). However, you do not need to do this if the replication goes over a private network (for example, if you have peered the self-managed cluster’s VPC with the Capella cluster’s VPC): in such cases, you can skip step 2 of the instructions provided below.

To set up XDCR from the on-premises cluster, follow these steps.

Add a Database User in your Cloud Cluster.
Whitelist the public IP of the VM
1. Get the public IP from your VM
  
  ssh into your VM (on which Couchbase Server is running).
  # dig +short myip.opendns.com @resolver1.opendns.com
  67.212.150.204
2. Allowlist this IP in the Cloud Cluster
  
  Clusters → < cluster name > → Connect → Add/View Allowed IPs

Get the hostname

Get the WAN Endpoint.

Clusters → < cluster name > → Connect → Copy

Get the hostname:

# nslookup -type=SRV _couchbases._tcp.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com

Server:		172.23.201.5
Address:	172.23.201.5#53

Non-authoritative answer:
_couchbases._tcp.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com	service = 0 0 11207 cb-0000.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com.

Authoritative answers can be found from:
cb-0000.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com	internet address = 100.26.70.215

The hostname is:

cb-0000.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com

When setting up a XDCR remote reference from your on-premise cluster Web Console, you must add port 18091 to the hostname.

Test Connectivity

# curl -u dbaas:P@ssw0rd https://cb-0000.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com:18091/pools --insecure

{"isAdminCreds":true,"isROAdminCreds":false,"isEnterprise":true,"allowedServices":["kv","n1ql","index","fts","cbas","eventing"],"isIPv6":false,"isDeveloperPreview":false,....

Copy the Root Certificate on the Cloud Server

Clusters → < cluster name > → Connect → [Security Certificate - Root Certificate] → Copy
Set up XDCR Remote Reference

From your on-premises cluster Web Console.
- Cluster Name: Any name
- IP Hostname: From Step 3.2 and add port 18091
  
  For example: cb-0000.8acfd09d-bb3a-4bfb-a278-7322e4902506.dataplane.nonprod-project-avengers.com:18091
- User name: From Step 1
- Password: From Step 1
- Cert pasted in Box: From Step 4
Set up XDCR Replication

From your on-premises cluster Web Console.

Then:

Create a Replication from Capella to a Self-Managed Cluster

A self-managed cluster is a cluster that is outside Capella. Data can be replicated from Capella to a self-managed cluster. To configure the replication, the Capella administrator must have the following:

The CA of the self-managed cluster.
Either of the following:
- The username and password for the self-managed cluster.
- A client certificate and private key provided for client access to the self-managed cluster.

Additionally, the self-managed cluster must be network-accessible to Capella. Ensure that:

The target cluster is accessible via the SSL ports 18091, 18092, and 11207.
Each Data-Service node on the Capella database can connect to each Data-Service node on the self-managed target.
All target-cluster firewalls allow access to the Capella Data-Service nodes.

To replicate data from Capella to a self-managed cluster; first, add the self-managed cluster as a target; then, select the self-managed cluster to be the target for a specific replication. These steps are described below. Note, however, that if the self-managed target is to be accessed over a private network, VPC Peering should be established for the connection, before the self-managed target is added. For information, see Configure a Private Network.

Add a Self-Managed Target

To add a self-managed target, proceed as follows:

Access the Replication screen. Under the main area provided for configuring replications, the Self-Managed Targets area is visible. Here, if no self-managed target has yet been added, the No Self-Managed Targets Found notification is displayed; and under this, the Add Self-Managed Target button.

Click on the Add Self-Managed Target button. This brings up the Add Self-Managed Target screen.
Take note of the information provided at the top of the screen: this contains essential requirements for establishing a replication from Capella to a self-managed target.
Enter a name for the self-managed target into the Target Name field.
Enter the IP address or Fully Qualified Domain Name of the self-managed target into the IP/Hostname field.
Optionally, add the appropriate administrative username and password for the self-managed target into the Username for Self Managed Target and Password fields. A username and password must be added unless you intend to authenticate by means of a client certificate and private key (described below).
Specify a Network Type, by selecting the appropriate radio button. This determines whether the internal or alternate network address of a target is used, in cases where the target can be contacted by means of either. When selected, the buttons Force using alternate address and Force using internal address each ensure that only its corresponding option can be used for establishing this replication. When selected, the button Auto ensures that Capella itself will make the decision.

For information on specifying internal and alternate addresses, see Using Alternate Addresses and Specifying Addresses.
Paste the CA of the self-managed target into the TLS Certificate field.
If you wish Capella to authenticate by means of a client certificate and private key, check the User Client Certificate Authentication checkbox. This displays the interactive fields Client Certificate and RSA Private Key, into which the certificate and key should be pasted. Note that checking this checkbox deactivates the Username and Password fields: anything previously entered into those fields will consequently be ignored.

If you wish to authenticate with a username and password, leave the User Client Certificate Authentication checkbox unchecked.
Click on the Add Self-Managed Target button. The self-managed target is now added to Capella.
The Replication screen is now displayed. The newly added self-managed target is now listed in the Self-Managed Targets area.

Replicate to a Self-Managed Target

After a self-managed cluster has been established as a target for Capella, a replication from Capella to the self-managed target can be configured. Proceed as follows:

On the Replication screen, click on the Setup Replication button. This brings up the Set Up Cross Data Center Replication screen.
Use the Select Replication Direction selector to establish One-Way as the replication direction to the self-managed targeted. Note that two-way replication cannot be used, when the target is self-managed.
Select the target database by accessing the pull-down menu in the Target Database field. Note that databases in the menu are divided into two groups, which are Self-Managed and Capella Managed. The self-managed database that is to be the target of the replication appears in the menu under the Self-Managed heading.
In the Select Buckets fields, add the names of the source and target buckets.
Complete the setup by appropriately configuring the sections Filter Documents, Select Replication Priority, Set Network Usage Limit, and Replicate All Scopes and Collections (as described previously, on this page).
Click on the Set Up Replication button. Capella now sets up the specified replication.

Error and Other Notifications

If a connectivity issue arises when replicating to a self-managed target, the issue is indicated in the Self-Managed Targets area of the Replication page: the notification One or more nodes has connectivity issues appears under the Connectivity Status column for the self-managed target:

Issues with connecting to self-managed target.

The replication itself is also represented in the list of all ongoing replications. A yellow circle, with an integer inside, appears in the Errors column, to indicate that a number of errors have occurred:

Click on the circle to see the errors: this displays the Replication Errors panel. Each error is listed, with its accompanying text.

Note that not all of the listings represent errors: some listings may be warnings or informational.