Manage Replications

  • Capella Operational
      +
      Configure XDCR to replicate data between source and destination buckets. An XDCR replication allows bucket data to be replicated continuously from a specified bucket on a source cluster to a specified bucket on a target cluster.

      Use the procedures on this page to create and manage XDCR (Cross Data Center Replication) with Capella operational clusters. For general information about XDCR in Capella and how it works, see Cross Data Center Replication (XDCR).

      Prerequisites

      • To view and manage replications on a cluster, you need the Project Owner role for your source cluster.

      • To delete or create a new replication, you need the Project Owner role for the projects that contain your source cluster and destination cluster.

      • You have created a single node or multi-node cluster that you want to use for replication, either as a source or destination cluster.

      Replication on single node clusters is only supported for development or test use cases.

      View Your Replications

      To view and manage replications:

      1. On the Operational Clusters page, select the cluster where you want to manage replications.

      2. Go to Settings  Replication.

      If you have not set up a replication for your cluster, you can click Set Up Replication. For more information about adding Self-Managed Targets, see Create a Replication from Capella to a Self-Managed Cluster.

      If you have already created replications for your cluster, you’ll see a summary of your replications in a table. For an example, see Observe an Ongoing Replication.

      Create a New Replication

      Source and destination buckets must have the same conflict resolution method configured. If your source and destination buckets use different conflict resolution methods, then you cannot create a replication.

      Data replication can be between:

      Clusters hosted by Couchbase Capella support the following replications:

      • Capella Hosted (AWS) ←→ Capella Hosted (AWS)

      • Capella Hosted (AWS) ←→ Own Cloud (AWS)

      • Capella Hosted (AWS) ←→ On-Premises

      • Capella Hosted (GCP) ←→ Capella Hosted (GCP)

      • Capella Hosted (GCP) ←→ Own Cloud (GCP)

      • Capella Hosted (GCP) ←→ On-Premises

      • Capella Hosted (Azure) ←→ Capella Hosted (Azure)

      • Capella Hosted (Azure) ←→ Own Cloud (Azure)

      • Capella Hosted (Azure) ←→ On-Premises

      Private Link connections do not support XDCR.

      Create a Replication Between Operational Clusters

      To create a replication between 2 operational clusters:

      1. On the Replication page, click Set Up Replication.

      2. Choose to create a One-way or a Two-way replication. A one-way replication replicates data only from the source to the target. A two-way replication replicates data both from the source to the target, and from the target to the source.

      3. Choose your target cluster from the Target Cluster menu.

      4. Under Select Buckets, choose the Source Bucket and Target Bucket to use in the replication.

      5. To add a filter to your replication, under Filter Replication, click Enable:

        1. In the Filter Expression field, enter a regex pattern or SQL++ statement to use to filter documents from your replication.

          For example, to only replicate documents that contain the value France for the key country, enter the expression REGEXP_CONTAINS(country, "France").

          The expression must contain at least 2 keys.

        2. Click Check Syntax to verify the syntax of your expression.

      6. (Optional) If you added a filter expression, use the Test Document panel to test your filter:

        1. From your source bucket, choose a scope, collection, and enter a document ID value for a document you want to use in your test.

        2. Click Test Document to run the test.

      7. Choose your Deletion Filters.

        For more information about deletion filters, see Deletion Filters in the Server documentation.

      8. If your operational cluster is on Couchbase Server version 7.2.2 or later, choose whether to Filter Binary Documents.

        For more information about filtering binary documents from XDCR, see Filtering Binary Documents in the Server documentation.

      9. Choose a High, Medium, or Low Replication Priority. For more information about each option, see XDCR Priority. A setting only takes effect if there are multiple replications with different priorities.

      10. To set a network usage limit, under Set Network Usage Limit, click Enable. Enter a limit in MiB per second for the maximum network usage of this replication.

        This limit applies to all replications for your source cluster.
      11. (Optional) If you want to replicate all scopes and collections on your source cluster to your target cluster, under Replicate All Scopes and Collections, click Yes.

        To replicate your scopes and collections, each scope and collection must already exist with the same name on the source and target buckets. If you want to replicate documents to a different target scope and collection from your source, click No.
        1. In the Source Name list, choose a scope and then a specific collection on your source cluster to replicate.

        2. In the Target Name list, choose the scope and specific collection on your target cluster to receive the replicated documents.

        3. (Optional) To add another scope and collection pairing on your source and target clusters, click Add Source and Target.

      12. To start the replication, click Setup Replication.

      It may take some time for your replication to be set up and start replicating documents.

      Observe an Ongoing Replication

      You can view the details for a current replication at any time from Settings  Replication on your cluster.

      Pause and Resume a Replication

      Pausing an XDCR replication temporarily suspends the replication of data from the source to the target. Pausing a replication always occurs on the source cluster. You can resume a paused replication at any time.

      When pausing a bidirectional replication, only the replication from the current cluster will be paused. To pause both directions of a bidirectional replication, you’ll need to pause the replication on both clusters individually.

      To pause or resume a replication:

      1. On the Replications page, click the Active replication you want to pause, or the Paused replication you want to resume.

      2. Under Pause Replication, click Pause Replication or Resume Replication.

      Modify a Replication

      You can modify specific replication settings after creation. You cannot change:

      • The replication direction.

      • The source cluster, source provider and region, source type, or source bucket.

      • The target cluster, target provider and region, target type, or target bucket.

      To modify your replication:

      1. On the Replications page, click the replication you want to modify.

      2. Modify the following settings:

        • Enable or disable filter replication.

        • Select High, Medium, or Low replication priority.

        • Enable or disable the network usage limit.

        • Choose to replicate all scopes and collections.

      3. Click Save.

      Delete a Replication

      Deleting an XDCR replication stops the replication of data, and removes the defined replication from the cluster. Any replicated data will not be deleted.

      If the replication is bidirectional, when you delete your replication on the source cluster, both directions of the replication are deleted and removed from the Replications tab of both clusters.

      To delete a replication:

      1. On the Replications page, click the replication you want to delete.

      2. Under Delete Replication, click Delete Replication.

      3. Confirm that you want to delete the replication.

      4. Click Delete Replication.

      Create a Replication to Capella from a Self-Managed Cluster

      Replicate your data to an operational cluster from a self-managed cluster that’s in an on-premises datacenter or a non-Capella cloud.

      If your XDCR replication is between an operational cluster and a self-managed cluster with the same cloud provider, you can connect them using a private network. If your XDCR replication is between an operational cluster and a self-managed cluster with different cloud providers or in an on-premises environment, you can only connect through the public Internet.

      If the replication goes across the public Internet, you must add the IP address of the self-managed cluster Data Service nodes to the Allowed IP list of the operational cluster, as shown in Step 2 of this procedure. For more information about adding IPs to the Allowed IP list, see Configure Allowed IP Addresses.

      You do not need to do this if the replication travels a private network. For example, if you peered the self-managed cluster’s VPC with the operational cluster’s VPC. In such cases, skip Step 2 of this procedure.

      To set up XDCR from an on-premises cluster to an operational cluster:

      1. Create a cluster access credential in your operational cluster.

        For more information about creating cluster access credentials, see Configure Cluster Access Credentials.

      2. If the replication crosses the public Internet, allowlist the public IP of the VM:

        1. Get the public IP from your VM.

          ssh into the VM where Couchbase Server is running.

          # dig +short myip.opendns.com @resolver1.opendns.com
          67.212.150.204
        2. Allowlist this IP in your operational cluster:

          With your cluster open in Capella, go to Settings  Allowed IP Addresses  Add Allowed IP.

          For more information about adding IPs to the Allowed IP list, see Configure Allowed IP Addresses.

      3. Get the hostname to use in the XDCR remote cluster reference:

        1. In Capella, with your cluster open, go to Connect  SDKs.

        2. In SDKs, copy the Public Connection String without the couchbases:// prefix. The public connection string is also the DNS SRV of the cluster.

          Although the connection string is sometimes referred to as the "public connection string," if you have VPC peering set up the connection string resolves to private addresses. For more information, see Configure a VPC Peering Connection.
      4. Copy the security certificate from the operational cluster:

        1. With your cluster open in Capella, go to Settings  Security Certificate  Copy.

      5. Set up XDCR Remote Reference:

        XDCR compatibility can vary between different versions of Couchbase Enterprise Server. To view and confirm compatibility, see XDCR Compatibility.
        1. In your on-premises cluster Couchbase Server Web Console, go to XDCR and click Add Replication:

          • Couchbase Server 7.X or later

          • Couchbase Server 6.6.X

          If your self-managed cluster uses Couchbase Server 7.X or later, you need the following information for your Capella cluster:

          Cluster Name

          Any name

          IP/Hostname

          The connection string from Step 3 without the couchbases:// prefix.

          As the hostname saves as a DNS SRV string in Couchbase Server 7.X and later, you do not need to specify a port.

          For example: cb.q2pi8ahm18gn4w2p.cloud.couchbase.com

          Username for Remote Cluster

          Cluster credentials from Step 1.

          Password

          Cluster credentials from Step 1.

          Certificate

          Paste the security certificate copied in Step 4.

          If your self-managed cluster uses Couchbase Server 6.6.X, you need the following information for your Capella cluster:

          Cluster Name

          Any name

          IP/Hostname

          The connection string from Step 3 without the couchbases:// prefix.

          Couchbase Server versions previous to Couchbase Server 7.x cannot use DNS SRV for the XDCR remote reference. Specify the port number 18091 with the Capella connection string so that XDCR resolves it to one of the operational cluster Data Service node hostnames.

          For example: cb.q2pi8ahm18gn4w2p.cloud.couchbase.com:18091

          Username for Remote Cluster

          Cluster credentials from Step 1.

          Password

          Cluster credentials from Step 1.

          Certificate

          Paste the security certificate copied in Step 4.

      6. Complete the setup for your replication.

        For more information about how to set up a replication in Couchbase Server, see Create a Replication.

      Create a Replication from Capella to a Self-Managed Cluster

      Replicate your data from an operational cluster to a self-managed cluster that’s in an on-premises datacenter or a non-Capella cloud.

      If your XDCR replication is between an operational cluster and a self-managed cluster with the same cloud provider, you can connect them using a private network. If your XDCR replication is between an operational cluster and a self-managed cluster with different cloud providers or in an on-premises environment, you can only connect through the public Internet.

      To configure the replication, the Capella administrator must have the following:

      • The CA of the self-managed cluster.

      • Either of the following:

        • The username and password for the self-managed cluster.

        • A client certificate and private key provided for client access to the self-managed cluster.

      Additionally, the self-managed cluster must be network-accessible to Capella. Ensure that:

      • The target cluster is accessible via the SSL ports 18091, 18092, and 11207.

      • Each Data-Service node on the operational cluster can connect to each Data-Service node on the self-managed target.

      • All target-cluster firewalls allow access to the Capella Data-Service nodes.

      To replicate data from Capella to a self-managed cluster; first, add the self-managed cluster as a target; then, select the self-managed cluster to be the target for a specific replication.

      If you want to access your self-managed target over a private network, set up VPC Peering for the connection before adding your self-managed target. For information, see Configure a Private Network.

      Add a Self-Managed Target

      To add a self-managed target:

      1. On the Operational Clusters page, select the cluster where you want to manage replications.

      2. Go to Settings  Replication.

      3. Click Add Self-Managed Target.

      4. Review the configuration information for setting up a self-managed target.

      5. In the Target Name field, enter a name for the self-managed target.

      6. In the IP/Hostname field, enter the IP address or Fully Qualified Domain Name of the self-managed target.

      7. (Optional) If you want to authenticate through a username and password, in the Username for Self-Managed Target and Password fields, enter your admin username and password for the self-managed target.

        You must add a username and password unless you authenticate with a client certificate and private key.

      8. Choose the Network Type for your connection.

        You can use either the alternate or internal network address for your self-managed target, or let Capella decide based on your provided hostname.

        For information about specifying internal and alternate addresses, see Using Alternate Addresses and Specifying Addresses.

      9. In the TLS Certificate field, paste the CA of the self-managed target.

      10. (Optional) If you want Capella to authenticate through a client certificate and private key, click Use Client Certificate Authentication.

        1. In the Client Certificate field, paste your client certificate from your self-managed target.

        2. In the RSA Private Key field, paste the RSA private key.

      11. (Optional) To verify the connection to your self-managed target using your provided information, click Check Connection.

      12. Click Add Self-Managed Target.

      Replicate to a Self-Managed Target

      To configure a replication from Capella to a self-managed target:

      1. On the Operational Clusters page, select the cluster where you want to manage replications.

      2. Go to Settings  Replication. Click Setup Replication.

      3. Choose a One-Way replication direction. You cannot use Two-Way replication with a self-managed target.

      4. In the Target Clusters list, select your self-managed target cluster. Clusters in the menu are divided into two groups: Self-Managed and Capella Managed.

      5. Under Select Buckets, choose the Source Bucket and Target Bucket to use in the replication.

      6. Configure other settings for your replication, such as Filter Documents, Select Replication Priority, Set Network Usage Limit, and Replicate All Scopes and Collections.

      7. Click Set Up Replication.

      Error and Other Notifications

      If a connectivity issue occurs while replicating to a self-managed target, you’ll see an error under Self-Managed Targets on the Replication page. Connectivity errors appear in the Connectivity Status column.

      For all replication types, any errors, warnings, or informational messages can be viewed in more detail from the Errors column under Replications. Click the icon in the Errors column for a listed replication to view error details.