A newer version of this documentation is available.

View Latest

cbbackup

The cbbackup tool creates a copy of data from an entire running cluster, an entire bucket, a single node, or a single bucket on a single functioning node.

Syntax

The basic syntax is:

cbbackup [options] [source] [backup-dir] -u [admin] -p [password]

Where:

  • [backup-dir]

    The directory for the backup files to be stored. If an empty directory doesn’t exist, it will be created; the parent directory must exist.

The following syntax example includes a full backup and two incremental backups for a cluster:

cbbackup http://HOST:8091 /backup-42 -u Administrator -p password
cbbackup http://HOST:8091 /backup-42 -m diff -u Administrator -p password
cbbackup http://HOST:8091 /backup-42 -m diff -u Administrator -p password
If the backup directory exists, the following backup command will be automatically incremented. The only way to generate a full backup is to choose an unused directory. If backing up a Couchbase Server Community Edition cluster, then incremental backup is not available, all backups will be full backups.

The following syntax example include a full backup, two differential backups, and one accumulative backup for a single node.

cbbackup couchbase://HOST:8091 /backup-43 -m full --single-node -u Administrator -p password
cbbackup couchbase://HOST:8091 /backup-43 -m diff --single-node -u Administrator -p password
cbbackup couchbase://HOST:8091 /backup-43 -m diff --single-node -u Administrator -p password
cbbackup couchbase://HOST:8091 /backup-43 -m accu --single-node -u Administrator -p password

Syntax example of a full backup plus two differentials and one accumulative for a single node.

cbbackup couchbase://HOST:8091 /backup-43 -m full --single-node -u Administrator -p password \
cbbackup couchbase://HOST:8091 /backup-43 -m diff --single-nodecbbackup \
couchbase://HOST:8091 /backup-43 -m diff --single-node -u Administrator -p password \
cbbackup couchbase://HOST:8091 /backup-43 -m accu --single-node -u Administrator -p password]
A full backup task is always triggered for a new sink location no matter what backup mode is specified.

Description

The backup process writes a copy of data onto the disk. To create a backup using cbbackup, the node or cluster needs to be functional.

The cbbackup, cbrestore, and cbtransfer tools do not communicate with external IP addresses for server nodes outside of a cluster. Backup, restore, or transfer operations are performed on data from a node within a Couchbase Server cluster. They only communicate with nodes from a node list obtained within a cluster. This also means that if Couchbase Server is installed with a default IP address, an external hostname cannot be used to access it.

This tool has several different options which you can use to:

  • Backup all buckets in an entire cluster

  • Backup one named bucket in a cluster

  • Backup all buckets on a node in a cluster

  • Backup one named buckets on a specified node

We recommended that cbbackup output be generated on a file system local to the server. Specifically, back up to a dedicated partition. A dedicated partition prevents the backup from filling the Couchbase Server data stores and operating system partitions.
Avoid routing the cbbackup output to remote share file systems (NFS). This is because cbbackup files are based on sqlite format which have issues being written to remote file systems.

This tool is located in the following directories:

Operating system Location

Linux

/opt/couchbase/bin/cbbackup

Windows

C:\Program Files\Couchbase\Server\bin\cbbackup

Mac OS X

/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/bin/cbbackup

Options

The following are the command options:

Table 1. cbbackup options
Parameters Description

-h, --help

Command line help.

-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE

Single named bucket from source cluster to transfer. If the backup directory only contains a single bucket, then that bucket is automatically used.

--single-node

Use a single server node from the source only, not all server nodes from the entire cluster. This single server node is defined by the source URL.

-m MODE, --mode=MODE

Backup mode. The mode option takes any one of the following values:

full

Perform a full backup

diff

Perform a differential incremental backup, which backs up only the changes since the last full or incremental backup.

accu

Perform a cumulative incremental backup, which backs up all changes since the last full backup.

-i ID, --id=ID

Transfer only items that match a vbucket ID.

-k KEY, --key=KEY

Transfer only items with keys that match a regular expression.

-n, --dry-run

No actual transfer. Just validate parameters, files, connectivity and configurations.

-u USERNAME, --username=USERNAME

REST username for source cluster or server node.

-p PASSWORD, --password=PASSWORD

REST password for cluster or server node.

-t THREADS, --threads=THREADS

Number of concurrent workers threads performing the transfer.

-v, --verbose

Verbose logging. More v’s provide more verbosity. Max: -vvv.

-x EXTRA, --extra=EXTRA

Provide extra, uncommon configuration parameters. Comma-separated key=val(key-val)* pairs.

The following are extra, specialized command options with the cbbackup -x parameter.

Table 2. cbbackup -x options
-x options Description

backoff_cap=10

Maximum backoff time during the rebalance period.

batch_max_bytes=400000

Transfer this # of bytes per batch.

batch_max_size=1000

Transfer this # of documents per batch.

cbb_max_mb=100000

Split backup file on destination cluster if it exceeds the MB.

conflict_resolve=1

By default, disable conflict resolution.

* This option doesn’t work in Couchbase Server versions 4.0 and 4.1 but will be re-implemented in version 4.1.1 and in subsequent versions.

data_only=0

For value 1, transfer only data from a backup file or cluster.

design_doc_only=0

For value 1, transfer only design documents from a backup file or cluster. Default: 0.

max_retry=10

Max number of sequential retries if the transfer fails.

mcd_compatible=1

For value 0, display extended fields for stdout output.

nmv_retry=1

0 or 1, where 1 retries transfer after a NOT_MY_VBUCKET message. Default: 1.

recv_min_bytes=4096

Amount of bytes for every TCP/IP batch transferred.

rehash=0

For value 1, rehash the partition id’s of each item. This is required when transferring data between clusters with different number of partitions, such as when transferring data from an Mac OS X server to a non-Mac OS X cluster.

report=5

Number batches transferred before updating progress bar in console.

report_full=2000

Number batches transferred before emitting progress information in console.

seqno=0

By default, start seqno from beginning.

try_xwm=1

Transfer documents with metadata. Default: 1. Value of 0 is only used when transferring from 1.8.x to 1.8.x.

uncompress=0

For value 1, restore data in uncompressed mode.

Back up only design documents from a cluster or bucket with the option, design_doc_only=1. Restore only design documents with cbrestore design_doc_only=1.

Examples

Example 1:

An entire cluster can be backed up. This includes all of the data buckets and data at all nodes and all design documents. To backup an entire cluster and all buckets for that cluster:

cbbackup http://HOST:8091 ~/backups -u Administrator -p password

Where ~/backups is the directory where you want to store the data. When this operation is performed, be aware that cbbackup creates the following directory structure and files in the ~/backups directory assuming there two buckets in the cluster named my_name and sasl and two nodes N1 and N2 :

~/backups
        bucket-my_name
            N1
            N2
        bucket-sasl
            N1
            N2

Where bucket-my_name and bucket-sasl are directories containing data files and where N1 and N2 are two sets of data files for each node in the cluster.

Example 2:

To backup a single bucket in a cluster:

cbbackup http://HOST:8091 /backups/backup-20120501 -u Administrator -p password \
 -b default

In this case, the default bucket in the cluster is specified and the data from the default bucket is backed up.

Example 3:

To backup all data from multiple buckets on a single node:

> cbbackup http://HOST:8091 /backups/ -u Administrator -p password \
 --single-node

Example 4:

To backup data from a single bucket on a single node:

cbbackup http://HOST:8091 /backups -u Administrator -p password \
 --single-node  -b bucket_name

Example 5:

To specify the keys that are backed up using the - k option. For example, to backup all keys from a bucket with the prefix ‘object’:

> cbbackup http://HOST:8091 /backups/backup-20120501 -u Administrator -p password \
 -b bucket_name -k '^object.*'

Example 6:

The following example creates a backup copy of all design documents from foo-bucket and store this as design.json in the directory ~/backup/foo-bucket. If a bucket is not specified, design documents for all buckets in the cluster are backed up.

cbbackup http://10.5.2.117:8091 ~/backup -x design_doc_only=1 -u Administrator -p password

Response

The following example response shows only design documents for all buckets being backed up. In this case, the source node had two (2) buckets including the default bucket.

transfer design doc only. bucket msgs will be skipped.
transfer design doc only. bucket msgs will be skipped.
done

In the following output, two design documents were backed up.

[
   {
      "controllers":{
         "compact":"/pools/default/buckets/default/ddocs/_design%2Fddoc1/controller/compactView",
         "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fddoc1/controller/setUpdateMinChanges"
      },
      "doc":{
         "json":{
            "views":{
               "view1":{
                  "map":"function(doc){emit(doc.key,doc.key_num);}"
               },
               "view2":{
                  "map":"function(doc,meta){emit(meta.id,doc.key);}"
               }
            }
         },
         "meta":{
            "rev":"1-6f9bfe0a",
            "id":"_design/ddoc1"
         }
      }
   },
   {
      "controllers":{
         "compact":"/pools/default/buckets/default/ddocs/_design%2Fddoc2/controller/compactView",
         "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fddoc2/controller/setUpdateMinChanges"
      },
      "doc":{
         "json":{
            "views":{
               "dothis":{
                  "map":"function (doc, meta) {\n  emit(meta.id, null);\n}"
               }
            }
         },
         "meta":{
            "rev":"1-4b533871",
            "id":"_design/ddoc2"
         }
      }
   },
   {
      "controllers":{
         "compact":"/pools/default/buckets/default/ddocs/_design%2Fdev_ddoc2/controller/compactView",
         "setUpdateMinChanges":"/pools/default/buckets/default/ddocs/_design%2Fdev_ddoc2/controller/setUpdateMinChanges"
      },
      "doc":{
         "json":{
            "views":{
               "dothat":{
                  "map":"function (doc, meta) {\n  emit(meta.id, null);\n}"
               }
            }
         },
         "meta":{
            "rev":"1-a8b6f59b",
            "id":"_design/dev_ddoc2"
         }
      }
   }
]

Example 7:

The following example requests a full backup of all the data on the specified cluster:

cbbackup -m full http://example.com:8091 /backups/backup-1 -u Administrator -p password

After an initial full backup, incremental backups can be performed. This example requests a differential incremental backup of all the data on the specified cluster:

cbbackup -m diff http://example.com:8091 /backups/backup-1 -u Administrator -p password

This example requests a cumulative incremental backup of all the data on the specified cluster:

cbbackup -m accu http://example.com:8091 /backups/backup-1 -u Administrator -p password