Backup and Restore with Command Line Tools

      +
      Use Couchbase command line tools to manage ad hoc backups.

      You can use the cbbackupmgr command line tool included with Couchbase Server with a Couchbase Capella database. The command line tools are available as a separate package.

      Use cbbackupmgr to back up and restore data across multiple Capella databases, either in development or production environments. For more information about cbbackupmgr, see cbbackupmgr in the Couchbase Server documentation.

      For data migration between Couchbase databases, Cross Data Center Replication (XDCR) is generally recommended. When XDCR cannot be used, the recommended method is to use cbbackupmgr backup and cbbackupmgr restore. Using cbexport and cbimport to migrate data from a Couchbase bucket that holds Sync Gateway data can cause the data to be corrupted.

      Prerequisites

      All procedures and examples on this page assume the following:

      • You’ve configured database access by creating database access credentials. You’ll need the username and password for the database credentials to connect to the database.

      • You’ve added your IP address to the database’s list of allowed IPs.

      • You’ve downloaded your security certificate for your database.

      • You’ve downloaded and installed the command line tools package.

      • You’ve set the following --disable options when you created your repository:

        --disable-analytics

        --disable-cluster-analytics

        --disable-bucket-query

        --disable-cluster-query

      Backup and Restore Examples

      For full examples, see cbbackupmgr in the Couchbase Server documentation.

      For production environments, use the secure --cacert <cert_file> option shown in the examples. For development environments, replace --cacert <cert_file> with --no-ssl-verify.

      Use the cbbackupmgr backup and cbbackupmgr restore commands to back up and restore your Capella database.

      Use --disable options to disable backup or restore of data that can’t be read or written by your Capella database access credentials.

      With Capella databases running Couchbase Server version 7.1.3 and later, use the following --disable options with cbbackupmgr config to configure the backup repository:

      --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query
      If you did not set these --disable options when you created your repository, you must add them to the cbbackupmgr restore command when you restore a backup form your repository to Capella.

      With Capella databases running Couchbase Server version 7.0.4, use --disable-eventing when using cbbackupmgr.

      If you are restoring data backed up from a self-managed cluster to a Capella database with cbbackupmgr restore, use --disable-views. Capella databases don’t support views.

      If you want to restore a backup of the beer-sample and gamesim-sample data sets, you must use --disable-views.

      $ cbbackupmgr config --archive /home/couchbase/backups --repo capella_db1 --disable-eventing --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query
      Back up all buckets
      $ cbbackupmgr backup --archive /home/couchbase/backups --repo capella_db1 --cluster couchbases://cb.zjhxs-12ab3cd4e5.cloud.couchbase.com --username dbuser --password ‘******’ --cacert /root/capella.pem --full-backup --threads 4

      To only restore mybucket1 data, and restore it to bucket newbucket1:

      $ cbbackupmgr restore --archive /home/couchbase/backups --repo capella_db1 --cluster couchbases://cb.zjhxs-12ab3cd4e5.cloud.couchbase.com --username dbuser --password ‘******’ --cacert /root/capella.pem --include-data mybucket1 --map-data mybucket1=newbucket1 --threads 4

      Backup a Capella Trial Database and Restore to a Paid Provisioned Database

      This example uses cbbackupmgr CLI to migrate data from a trial database to a paid provisioned database.

      Backup a Trial Database

      To configure the cbbackupmgr archive repository to hold the backup:

      $ cbbackupmgr config --archive /home/couchbase/backups --repo capella_trial --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query --disable-views

      To backup a Capella trial database:

      $ cbbackupmgr backup --archive /home/couchbase/backups --repo capella_trial -c couchbases://cb.ubibrahu6aizk5rm.cloud.couchbase.com -u trialuser -p '******' --cacert /root/capella.pem --full-backup --threads 2

      Restore to a Paid Provisioned Database

      Ensure you have created the Couchbase buckets to restore to your paid provisioned database. Database access credentials do not provide have privileges to create a bucket, so you cannot use the cbbackupgmr restore option --auto-create-buckets when restoring to a Capella database.

      To restore to a paid provisioned database:

      $ cbbackupmgr restore --archive /home/couchbase/backups --repo capella_trial -c couchbases://cb.20xa2skdftoygjxe.cloud.couchbase.com -u dbuser -p '******' --cacert /root/capella.pem --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query --disable-views

      You do not need to use the same disable options again during the restore because you used them when configuring the archive repository. There may be cases where you may not know how the backup was done.

      The --purge option cleans up restore progress information from the previous restore attempt. Add the --purge option to the restore command to rerun the restore if you get an error at the start of the restore.

      See Allowed IPs Tips and Restore Tips for more restoring tips.

      Build Indexes

      Index definitions are restored but not built.

      To build the indexes in your database:

      1. Open your database in the Capella UI.

      2. Go to Data Tools  Indexes.

      3. Build the indexes that are in a “Created” state.

      Backup a Self-Managed Database and Restore to a Capella Provisioned Database

      This example uses cbbackupmgr CLI to migrate data from a self-managed database to a provisioned database.

      A self-managed database can be either an Enterprise or Community Edition. If you are using the Community Edition Couchbase Server, you need to use the cbbackupmgr CLI from the command line tools package to restore to the Capella provisioned database. You can also use the cbbackupmgr CLI from the command line tools package for backup.

      Backup a Self-Managed Database

      You can backup a self-managed database or use an existing backup. The example below is when you do not have an existing backup.

      To configure the cbbackupmgr archive repository to hold the backup:

      $ cbbackupmgr config --archive /home/couchbase/backups --repo repo_mydb

      To backup a self-managed database:

      $ cbbackupmgr backup --archive /home/couchbase/backups --repo repo_mydb -c localhost -u Administrator -p '******'

      Restore to a Paid Provisioned Database

      If your self-managed database is using the Community Edition Couchbase Server, you must use the cbbackupmgr CLI from the command line tools package to restore to the Capella provisioned database.

      Ensure you have created the Couchbase buckets to restore to your paid provisioned database. Capella database access credentials do not have privileges to create a bucket, so you cannot use the cbbackupgmr restore option --auto-create-buckets when restoring to a Capella database.

      To restore to a paid provisioned database:

      $ cbbackupmgr restore --archive /home/couchbase/backups --repo repo_mydb -c couchbases://cb.eqwqztkw5e5kf6l.cloud.couchbase.com --cacert /root/capella.pem -u dbuser -p '******' --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query --disable-views

      The --purge option cleans up restore progress information from the previous restore attempt. Add the --purge option to the restore command to rerun the restore if you get an error at the start of the restore.

      See Allowed IPs Tips and Restore Tips for more restoring tips.

      Build Indexes

      Index definitions are restored but not built.

      To build the indexes in your database:

      1. Open your database in the Capella UI.

      2. Go to Data Tools  Indexes.

      3. Build the indexes that are in a “Created” state.

      Allowed IPs Tips

      If you are running cbbackupmgr CLI from a VPC-peered environment, the Capella connection string may resolve to a private IP address in your environment. In this situation, you do not need to add the IP address of the machine you are running cbbackupmgr on to the Capella database’s list of allowed IPs.

      If you have a private endpoint set up in your Capella provisioned database, and you are using the Capella private endpoint DNS name to connect, you do not need to add the IP address of the machine you are running cbbackupmgr on to the list of allowed IPs.

      Example private endpoint DNS name connection:
      couchbases://private-endpoint.iml6stsfy4njxkx0.cloud.couchbase.com

      Restore Tips

      The --purge option cleans up restore progress information from the previous restore attempt.

      Add the --purge option to the restore command to rerun the restore if you get an error at the start of the restore because you omitted to do the following:

      • Create the buckets in the provisioned database.

      • Create the database access credential.

      • Specify the certificate option.

      If the restore process is running, and gets interrupted due to a temporary network issue, try restoring again from where you left off using the --resume option.

      Other specific restore options are shown in the following table:

      Option Description Example

      --map-data

      Restore to different bucket names.

      --map-data oldbucket1=newbucket1,oldbucket2=newbucket2

      --include-data

      Restore only specific buckets, scopes, and collections.

      --include-data bucket1.scope1,bucket2.scope2.collection3

      --exclude-data

      Exclude specific buckets, scopes, and collections from the restore.

      --exclude-data bucket2.scope1

      --threads

      Specify the number of concurrent clients to use during backup or restore. The default is 1. More clients means more resource usage. The value should not exceed the number of CPUs on the machine running cbbackupmgr.

      --threads 4

      There are other cbbackupmgr restore options. For example, if the backup you are restoring also includes FTS indexes, you may want to restore the data first, followed by restoring the FTS indexes. If you restore the data and the FTS indexes concurrently, the restore starts by creating the FTS indexes, followed by the restore data, which puts a very high load on the FTS nodes.

      The GSI indexes do not have the same issue as the FTS indexes because they are restored deferred. The index definitions are restored, but the indexes are not built. You must build the GSI indexes manually after the restore completes.

      To disable restoring FTS indexes, add the following disable options to your cbbackupmgr restore command:
      --disable-ft-indexes and --disable-ft-alias

      Example:

      $ cbbackupmgr restore --archive /home/couchbase/backups --repo repo_mydb -c couchbases://cb.eqwqztkw5e5kf6l.cloud.couchbase.com --cacert /root/capella.pem -u dbuser -p '******' --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query --disable-views --disable-ft-indexes --disable-ft-alias

      After restoring your data, GSI indexes, and eventing functions, to restore the FTS indexes by adding the disable options for data, GSI indexes, and eventing, run another cbbackupmgr restore command:
      --disable-data --disable-gsi-indexes --disable-eventing

      Example:

      $ cbbackupmgr restore --archive /home/couchbase/backups --repo repo_mydb -c couchbases://cb.eqwqztkw5e5kf6l.cloud.couchbase.com --cacert /root/capella.pem -u dbuser -p '******' --disable-analytics --disable-cluster-analytics --disable-bucket-query --disable-cluster-query --disable-views --disable-data --disable-gsi-indexes --disable-eventing