A newer version of this documentation is available.

View Latest

cbbackupwrapper and cbrestorewrapper

Two new tools are included with Couchbase Server Enterprise Edition that improve throughput for the database backup process: cbbackupwrapper and cbrestorewrapper.

cbbackupwrapper and cbrestorewrapper were specifically created to address the single worker thread limitation for one backup/restore target (Bucket, Node). They should be used instead of the standard backup tools, cbbackup and cbrestore.

cbbackupwrapper and cbrestorewrapper are designed to provide a parallel processing approach to backing up a Couchbase Server cluster. At a technical level, they divide each vBucket space into a subset of vBuckets for each backup process and thus improve the speed and efficiency.

In a future release of Couchbase Server, these two tools will be folded into cbbackup and cbrestore respectively and then deprecated.

Syntax

The basic syntax is:

cbbackupwrapper BACKUPDIR CLUSTER OPTIONS
cbrestorewrapper BACKUPDIR CLUSTER OPTIONS

Options

The following are the command options:

Table 1. Basic command options for cbbackupwrapper
Option Description

-h, --help

Show this help message and exit

-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE

Specify the bucket to backup. Defaults to all buckets

--single-node

Use a single server node from the source only

-u USERNAME, --username=USERNAME

REST username for source cluster or server node. Default is Administrator

-p PASSWORD, --password=PASSWORD

REST password for source cluster or server node

-s, --ssl

Transfer data with SSL enabled

-v, --verbose

Enable verbose messaging

--path=PATH

Specify the path to cbbackup. Defaults to current directory

--port=PORT

Specify the bucket port. Defaults to 11210.

-n NUMBER, --number=NUMBER

Specify the number of vbuckets per process. Defaults to 100.

-P PARALLELISM, --parallelism=PARALLELISM

Number of vbucket backup jobs to run at a time. Defaults to 1.

-x EXTRA, --extra=EXTRA

Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.

-m MODE, --mode=MODE

Backup mode: full, diff or accu [default:diff]

Table 2. Additional configuration options for cbbackupwrapper
Option Description

backoff_cap=10

Max backoff time during rebalance period

batch_max_bytes=400000

Transfer this # of bytes per batch

batch_max_size=1000

Transfer this # of documents per batch

cbb_max_mb=100000

Split backup file on destination cluster if it exceeds MB

data_only=0

For value 1, only transfer data from a backup file or cluster

design_doc_only=0

For value 1, transfer design documents only from a backup file or cluster

max_retry=10

Max number of sequential retries if transfer fails

recv_min_bytes=4096

Amount of bytes for every TCP/IP call transferred

rehash=0

For value 1, rehash the partition IDs of each item. Rehashing is needed when transferring data between clusters with a different number of partitions, such as when transferring data from an OSX server to a non-OSX cluster.

report=5

Number batches transferred before updating progress bar in console

report_full=2000

Number batches transferred before emitting progress information in console

seqno=0

By default, start seqno from beginning

Table 3. Basic command options for cbrestorewrapper
Option Description

-h, --help

Show this help message and exit

-B

Allows you to specify the destination bucket for a restore (assuming that the bucket in the backup is named differently).

-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE

Specify the bucket to restore. Defaults to all buckets

--single-node

Use a single server node from the source only

-u USERNAME, --username=USERNAME

REST username for source cluster or server node. Default is Administrator

-p PASSWORD, --password=PASSWORD

REST password for source cluster or server node

-s, --ssl

Transfer data with SSL enabled

-v, --verbose

Enable verbose messaging

--path=PATH

Specify the path to cbrestore. Defaults to current directory

--port=PORT

Specify the bucket port. Defaults to 11210.

-n NUMBER, --number=NUMBER

Specify the number of vbuckets per process. Defaults to 100.

-P PARALLELISM, --parallelism=PARALLELISM

Number of vbucket backup jobs to run at a time. Defaults to 1.

-x EXTRA, --extra=EXTRA

Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.

-m MODE, --mode=MODE

Restore mode: full, diff or accu [default:diff]

Table 4. Additional configuration options for cbrestorewrapper
Option Description

backoff_cap=10

Max backoff time during rebalance period

batch_max_bytes=400000

Transfer this # of bytes per batch

batch_max_size=1000

Transfer this # of documents per batch

cbb_max_mb=100000

Split restore file on destination cluster if it exceeds MB

data_only=0

For value 1, only transfer data from a restgore file or cluster

design_doc_only=0

For value 1, transfer design documents only from a restore file or cluster

max_retry=10

Max number of sequential retries if transfer fails

recv_min_bytes=4096

Amount of bytes for every TCP/IP call transferred

report=5

Number batches transferred before updating progress bar in console

report_full=2000

Number batches transferred before emitting progress information in console

seqno=0

By default, start seqno from beginning