Please use the form below to provide your feedback. Because your feedback is valuable to us, the information you submit in this form is recorded in our issue tracking system (JIRA), which is publicly available. You can track the status of your feedback using the ticket number displayed in the dialog once you submit the form.

cbmigrate

  • reference
Use the cbmigrate command-line tool to migrate your data from other platforms.

Description

The cbmigrate tool will migrate your existing data from the following platforms:

Installation

  1. Download the latest version of the cbmigrate package from its GitHub repository.

  2. Unpack the downloaded package to its own directory.

  3. Execute the tool by running the following from the command line:

    $ ./cbmigrate [command] [flags]

Syntax

$ cbmigrate [--version] [--help HELP]
$ cbmigrate [command] [flags]

Command options

cbmigrate takes one of three optional commands. Depending on the command used, the cbmigrate tool will also accept a range of flags for additional information required for its execution.

  • mongoDB

  • DynamoDB

  • Hugging Face

Table 1. Command options
Command Flags

mongo

Migrate the data from a MongoDB installation to Couchbase server.
--mongodb-uri string

The mongodb connection string.

--mongodb-database string

The name of the database that you wish to migrate.

--mongodb-collection

The name of the collection within the database you are migrating.

--cb-username string

The username granting access to the target cluster.

--cb-password string

The password (attached to --cb-username) for accessing the target cluster.

--cb-cluster string

The URL of the target cluster node for the import.

--cb-bucket string

The name of the target bucket.

--cb-scope string

The target scope for the migration.

--cb-collection string

The target collection name for the import.

--cb-generate-key string

Specifies a key expression used for generating a unique key for each imported document. It allows for the creation of document keys by combining static text, field values (denoted by %fieldname%), and custom generators (like #UUID#). For example, using a combination of static text, field names, and custom generators, you can generate a unique key of the form: "key::%name%::#UUID#"
(Default: "%_id%")

--cb-cacert string

Specifies a CA certificate that will be used to verify the identity of the server being connected to. Either this flag or the --cb-no-ssl-verify flag must be specified when using an SSL encrypted connection.

--cb-no-ssl-verify

Skips the SSL verification phase. Specifying this flag will allow a connection using SSL encryption but will not verify the identity of the server you connect to.

You are vulnerable to a man-in-the-middle attack if you use this flag.

Either this flag or the --cb-cacert flag must be specified when using an SSL encrypted connection

--cb-client-cert string

The path to a client certificate used to authenticate when connecting to a cluster. May be supplied with --cb-client-key as an alternative to the --cb-username and --cb-password flags.

--cb-client-cert-password

The password for the certificate provided to the --cb-client-cert flag, when using this flag, the certificate/key pair is expected to be in the PKCS#12 format

--cb-client-key string

The path to the client private key whose public key is contained in the certificate provided to the --cb-client-cert flag. May be supplied with --cb-client-cert as an alternative to the --username and --password flags.

--cb-client-key-password string

The password for the key provided to the --cb-client-key flag, when using this flag, the key is expected to be in the PKCS#8 format

--cb-buffer-size int

An integer value denoting the size of the memory buffer used during the import. (Default: 10000)

--cb-batch-size int

The number of documents processed as a batch during the import. (Default: 200)

--copy-indexes

Copy indexes for the collection (default: true)

--hash-document-key string

Hash the couchbase document key. Can be sha256 or sha512)

--keep-primary-key

Keep the non-composite primary key in the document. By default, if the key is a non-composite primary key, it is deleted.

--help

Help for the MongoDB migration parameters and flags

--debug

Enable debug output.
Table 2. Command options
Command Flags

dynamodb

Migrate the data from a DynamoDB installation to Couchbase server.
--aws-access-key-id string

Your AWS Access Key ID

--aws-ca-bundle string

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings

--aws-endpoint-url string

Override the AWS default endpoint url with the given URL

--aws-no-verify-ssl

By default, cbmigrate uses SSL when communicating with AWS services. For each SSL connection, cbmigrate will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

--aws-profile string

Use a specific aws profile from your credential file.

--aws-region string

The region to use. Overrides config/env settings.

--aws-secret-access-key string

The AWS secret access key.

--dynamodb-limit int

Specifies the maximum number of items to retrieve per page during a scan operation. Use this option to control the amount of data fetched in a single request, helping to manage memory usage and API call rates during scanning.

--dynamodb-segments int

Specifies the total number of segments to divide the DynamoDB table into for parallel scanning. Each segment is scanned independently, allowing multiple threads or processes to work concurrently for faster data retrieval. Use this option to optimize performance for large tables. By default, the entire table is scanned sequentially without segmentation (Default: 1)

--dynamodb-table-name string

The name of the table containing the requested item. You can also provide the Amazon Resource Name (ARN) of the table in this parameter.

--cb-username string

The username granting access to the target cluster.

--cb-password string

The password (attached to --cb-username) for accessing the target cluster.

--cb-cluster string

The URL of the target cluster node for the import.

--cb-bucket string

The name of the target bucket.

--cb-scope string

The target scope for the migration.

--cb-collection string

The target collection name for the import.

--cb-generate-key string

Specifies a key expression used for generating a unique key for each imported document. It allows for the creation of document keys by combining static text, field values (denoted by %fieldname%), and custom generators (like #UUID#). For example, using a combination of static text, field names, and custom generators, you can generate a unique key of the form: "key::%name%::#UUID#"
(Default: "%_id%")

--cb-cacert string

Specifies a CA certificate that will be used to verify the identity of the server being connected to. Either this flag or the --cb-no-ssl-verify flag must be specified when using an SSL encrypted connection.

--cb-no-ssl-verify

Skips the SSL verification phase. Specifying this flag will allow a connection using SSL encryption but will not verify the identity of the server you connect to.

You are vulnerable to a man-in-the-middle attack if you use this flag.

Either this flag or the --cb-cacert flag must be specified when using an SSL encrypted connection

--cb-client-cert string

The path to a client certificate used to authenticate when connecting to a cluster. May be supplied with --cb-client-key as an alternative to the --cb-username and --cb-password flags.

--cb-client-cert-password

The password for the certificate provided to the --cb-client-cert flag, when using this flag, the certificate/key pair is expected to be in the PKCS#12 format

--cb-client-key string

The path to the client private key whose public key is contained in the certificate provided to the --cb-client-cert flag. May be supplied with --cb-client-cert as an alternative to the --username and --password flags.

--cb-client-key-password string

The password for the key provided to the --cb-client-key flag, when using this flag, the key is expected to be in the PKCS#8 format

--cb-buffer-size int

An integer value denoting the size of the memory buffer used during the import. (Default: 10000)

--cb-batch-size int

The number of documents processed as a batch during the import. (Default: 200)

--copy-indexes

Copy indexes for the collection (default: true)

--hash-document-key string

Hash the couchbase document key. Can be sha256 or sha512)

--keep-primary-key

Keep the non-composite primary key in the document. By default, if the key is a non-composite primary key, it is deleted.

--help

Help for the MongoDB migration parameters and flags

--debug

Enable debug output.
Table 3. Command options
Command Flags

hugging-face

Migrate the data from a Hugging Face installation to Couchbase server.
--path string

The path or name of the dataset. (Required)

--name

Configuration name of the dataset. (Optional)

--data-files string

Path(s) to the source data file(s). (Optional)

--split string

The split of the data to load. (Optional)

--cache-dir string

The cache directory to store the datasets. (Optional)

--download-config string

Specific download configuration parameters. (Optional)

--download-mode reuse_dataset_if_exists | force_redownload

Specifies whether to reuse existing downloaded data or force a fresh download. (Optional)

--verification-mode no_checks | basic_checks | all_checks

Sets the level of verification during the migration. (Optional)

--keep-in-memory

Use this flag to keep the migrated dataset in memory.

--save-infos

Save the dataset information. (Default: false)

--revision string

The version of the dataset script to load. (Optional)

--token string

Authentication token for private datasets. (Optional)

--no-streaming

Disable streaming mode for dataset loading. (Default: false)

num-proc int

Number of processes to use for the migration. (Optional)

--storage-options string

Storage options for remote filesystems. (Optional)

--trust-remote-code

Allow loading arbitrary code from the dataset repository. (Optional)

--id-fields string

Comma-separated list of field names to use as the document ID.

--cb-url string

The URL for the target Couchbase cluster (e.g., couchbase://localhost)

--cb-username string

The username granting access to the target cluster.

--cb-password string

The password (attached to --cb-username) for accessing the target cluster.

--cb-bucket string

The name of the target bucket.

--cb-scope string

The target scope for the migration.

--cb-collection string

The target collection name for the import.

cb-batch-size int

The number of documents to insert per batch. (Default: 1000)

--help

Show the help screen for the hugging face migration.