A newer version of this documentation is available.

View Latest

cbbackupwrapper and cbrestorewrapper

    +
    Two tools are included with Couchbase Server Enterprise Edition that improve throughput for the database backup process: cbbackupwrapper and cbrestorewrapper.

    cbbackupwrapper and cbrestorewrapper were specifically created to address the single worker thread limitation for one backup/restore target (Bucket, Node). They should be used instead of the standard backup tools, cbbackup and cbrestore.

    cbbackupwrapper and cbrestorewrapper are designed to provide a parallel processing approach to backing up a Couchbase Server cluster. At a technical level, they divide each vBucket space into a subset of vBuckets for each backup process and thus improve the speed and efficiency.

    In a future release of Couchbase Server, these two tools will be folded into cbbackup and cbrestore respectively and then deprecated.

    Syntax

    The basic syntax is:

    cbbackupwrapper CLUSTER BACKUPDIR OPTIONS
    cbrestorewrapper BACKUPDIR CLUSTER OPTIONS

    Options

    The following are the command options:

    Table 1. Basic command options for cbbackupwrapper
    Option Description

    -h, --help

    Show this help message and exit.

    -b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE

    Specify the bucket to backup. Defaults to all buckets.

    --single-node

    Use a single server node from the source only.

    -u USERNAME, --username=USERNAME

    REST username for source cluster or server node. Default is Administrator.

    -p PASSWORD, --password=PASSWORD

    REST password for source cluster or server node.

    -s, --ssl

    Transfer data with SSL enabled.

    -v, --verbose

    Enable verbose messaging.

    --path=PATH

    Specify the path to cbbackup. Defaults to current directory.

    --port=PORT

    Specify the bucket port. Defaults to 11210.

    -n NUMBER, --number=NUMBER

    Specify the number of vbuckets per process. Defaults to 100.

    -P PARALLELISM, --parallelism=PARALLELISM

    Number of vbucket backup jobs to run at a time. Defaults to 1.

    -x EXTRA, --extra=EXTRA

    Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.

    -m MODE, --mode=MODE

    Backup mode: full, diff or accu [default:diff].

    Table 2. Additional configuration options for cbbackupwrapper
    Option Description

    backoff_cap=10

    Max backoff time during rebalance period.

    batch_max_bytes=400000

    Transfer this # of bytes per batch.

    batch_max_size=1000

    Transfer this # of documents per batch.

    cbb_max_mb=100000

    Split backup file on destination cluster if it exceeds MB.

    data_only=0

    For value 1, only transfer data from a backup file or cluster.

    dcp_consumer_queue_length=1000

    A DCP client needs a queue for incoming documents/messages. A large length is more efficient, but memory proportional to length*avg. doc size. Below length 150, performance degrades significantly.

    design_doc_only=0

    For value 1, transfer design documents only from a backup file or cluster.

    flow_control=1

    For value 0, disable flow control to improve throughput.

    max_retry=10

    Max number of sequential retries if transfer fails.

    recv_min_bytes=4096

    Amount of bytes for every TCP/IP call transferred.

    rehash=0

    For value 1, rehash the partition IDs of each item. Rehashing is needed when transferring data between clusters with a different number of partitions, such as when transferring data from an OS X server to a non-OS X cluster.

    report=5

    Number batches transferred before updating progress bar in console.

    report_full=2000

    Number batches transferred before emitting progress information in console.

    seqno=0

    By default, start seqno from beginning.

    uncompress=0

    For value 1, restore data in uncompressed mode.

    This option is unsupported. To create backups with compression, you should use cbbackupmgr, which is available for Couchbase Server Enterprise Edition only.

    Table 3. Basic command options for cbrestorewrapper
    Option Description

    -h, --help

    Show this help message and exit.

    -B BUCKET_DESTINATION

    Allows you to specify the destination bucket for a restore (assuming that the bucket in the backup is named differently).

    -b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE

    Specify the bucket to restore. Defaults to all buckets.

    -u USERNAME, --username=USERNAME

    REST username for source cluster or server node. Default is Administrator.

    -p PASSWORD, --password=PASSWORD

    REST password for source cluster or server node.

    -s, --ssl

    Transfer data with SSL enabled.

    -v, --verbose

    Enable verbose messaging.

    --path=PATH

    Specify the path to cbrestore. Defaults to current directory.

    --port=PORT

    Specify the bucket port. Defaults to 11210.

    --from-date=FROM_DATE

    Restore data from the date specified as yyyy-mm-dd. By default, all data from the very beginning will be restored.

    --to-date=TO_DATE

    Restore data till the date specified as yyyy-mm-dd. By default, all data collected will be restored.

    -x EXTRA, --extra=EXTRA

    Provide extra, uncommon config parameters; comma-separated key=val(,key=val)* pairs.

    Table 4. Additional configuration options for cbrestorewrapper
    Option Description

    backoff_cap=10

    Max backoff time during rebalance period.

    batch_max_bytes=400000

    Transfer this # of bytes per batch.

    batch_max_size=1000

    Transfer this # of documents per batch.

    cbb_max_mb=100000

    Split backup file on destination cluster if it exceeds MB.

    conflict_resolve=1

    By defaault, enable conflict resolution.

    data_only=0

    For value 1, only transfer data from a backup file or cluster.

    dcp_consumer_queue_length=1000

    A DCP client needs a queue for incoming documents/messages. A large length is more efficient, but memory proportional to length*avg. doc size. Below length 150, performance degrades significantly.

    design_doc_only=0

    For value 1, transfer design documents only from a backup file or cluster.

    flow_control=1

    For value 0, disable flow control to improve throughput.

    max_retry=10

    Max number of sequential retries if transfer fails.

    recv_min_bytes=4096

    Amount of bytes for every TCP/IP call transferred.

    rehash=0

    For value 1, rehash the partition IDs of each item; this is needed when transferring data between clusters with different number of partitions, such as when transferring data from an OS X server to a non-OS X cluster.

    report=5

    Number batches transferred before updating progress bar in console.

    report_full=2000

    Number batches transferred before emitting progress information in console.

    seqno=0

    By default, start seqno from beginning.

    uncompress=0

    For value 1, restore data in uncompressed mode.

    This option is unsupported. To restore from compressed backups, you should use cbbackupmgr, which is available for Couchbase Server Enterprise Edition only.