cbft-bleve for Scorch Indexes

  • reference
    +
    The cbft-bleve tool supports commands that return information on Scorch index partitions.

    Using cbft-bleve on Scorch indexes.

    The cbft-bleve tool can be used to retrieve information on a Scorch index partition in either of the following ways:

    • By means of the subset of top level command-values that can also be used on Moss index partitions.

    • Specifically by means of the scorch top level command-value, which entails additional syntax, and can be used on Scorch index partitions only.

    These options are described below. Examples are for MacOS.

    Using Top Level Command-Values

    The top-level Syntax for cbft-bleve permits the following to be specified as values for the principal command, and to be used alike on Moss index partitions (on all Couchbase-Server supported platforms) and on Scorch index partitions (on all platforms except CentOS and Ubuntu): check, count, dictionary, fields, mapping, query, and registry.

    For a description of the syntax entailed by each value, and instances of command-output, see the examples provided for Moss index partitions, in cbft-bleve for Moss Indexes.

    Using the scorch Command-Value

    The scorch command-value specifies that the target index-partition is a Scorch index. The basic syntax is as follows:

    cbft-bleve scorch [command] [index-store-path] [snapshot] [flags]

    The available commands are listed in the sections below. A command must be specified. Note that a command summary can be printed to standard output, by entering cbft-bleve scorch --help (or -h).

    The index-store-path, which is mandatory, is the filesystem path of the store subdirectory, located under the directory for the Scorch index partition. See Specifying Index Partitions.

    The snapshot is a reference to a particular snapshot, within a Scorch-index partition, on which information is to be retrieved. See immediately below.

    The flags argument can always be --help, which prints help text to standard output. Additional values are indicated as appropriate, below.

    Understanding Snapshots

    Each Scorch index contains a number of snapshots, each of which represents the state of all indexed data at a particular point in time: this supports high-performance rollbacks of the index. By default, three snapshots are simultaneously available: each is referred to by a unique integer known as the snapshot epoch. To change the number of available snapshots, include the numSnapshotsToKeep key to the store object within the index-definition, specifying an appropriate integer as the value. For example, the following establishes 4 snapshots to be made simultaneously available:

       "store": {
         "kvStoreName": "scorchStore",
         "indexType": "scorch",
         "numSnapshotsToKeep": 4
       }

    For more information, see Index-Creation with the REST API.

    Some commands used by cbft-bleve scorch refer to and return information on a specific snapshot epoch. Information on currently available snapshot epochs and their unique identifying integers is returned by means of the snapshot command.

    snapshot

    The snapshot command prints a list of snapshot epochs that currently exist for a specified Scorch index partition. The syntax is as follows:

    cbft-bleve scorch snapshot [index-store-path] [flags]

    Example: Print Snapshot Epochs

    The following example prints the snapshot epochs for the specified index partition:

    /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
    /cbft-bleve scorch snapshot \
    /Users/<username>/Library/Application\ Support/Couchbase\
    /var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/

    If successful, the command prints the following:

    snapshot epoch: 37
    snapshot epoch: 36
    snapshot epoch: 1

    ascii

    The ascii command prints an ascii representation of the segments in a snapshot. Each segment is a batch of items from the snapshot. The syntax is as follows:

    cbft-bleve scorch ascii [index-store-path] [snapshot]

    The snapshot argument, which is required, must be the unique integer associated with a snapshot epoch for the specified index, as returned by the snapshot command.

    Example: Print an ASCII Representation

    The following example prints an ascii representation of the segments in the snapshot specified for an index partition.

    /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
    /cbft-bleve scorch ascii \
    /Users/<username>/Library/Application\ Support/Couchbase\
    /var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ \
    37

    If successful, the command returns the following:

    37    37:  5433 / 5433 -   .........................

    deleted

    The deleted command prints a bitmap that represents items that have been deleted from each segment of a specified snapshot, during the course of the segment’s existence. The syntax is as follows:

    cbft-bleve scorch deleted [index-store-path] [snapshot] [flags]

    The snapshot argument, which is required, must be the unique integer associated with a snapshot epoch for the specified index partition, as returned by the snapshot command.

    Example: Print the Deleted Bitmap

    The following example prints the bitmap of deleted items for the segments in snapshot 67:

    /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
    /cbft-bleve scorch deleted \
    /Users/<username>/Library/Application\ Support/Couchbase\
    /var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ \
    67

    If successful, the command returns a list that contains deleted items, if any exist:

    0 <nil>
    1 <nil>
    2 <nil>

    info

    The info command prints information on a specified index partition. The syntax is as follows:

    cbft-bleve scorch info [index-store-path] [flags]

    Example: Print Info on an Index Partition

    The following example prints information on the specified Scorch index partition.

    /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
    /cbft-bleve scorch info \
    /Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
    /scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/

    If successful, the command returns the following:

    doc count: 5433

    internal

    The internal command prints the internal kv pairs within a snapshot — these constitute meta data for the snapshot. The syntax is as follows:

    cbft-bleve scorch internal [index-store-path] [snapshot] [flags]

    The snapshot, which is mandatory, must be the unique integer associated with a snapshot epoch for the specified index, as returned by the snapshot command.

    The flags argument can be -a or --ascii; which indicates that the values are to be printed in ascii.

    Example: Print KV Pairs

    The following example provides an ascii print-out of the kv pairs for the specified index-snapshot:

    /Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/\
    cbft-bleve scorch internal \
    /Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
    /scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ 37 -a

    If successful, the command provides the following output:

    42 ?
    43 ?
    _mapping {"types":{"product":{"enabled":true,"dynamic":true,"properties":{"price":{"enabled":true,"dynamic":false,"fields":[{"name":"price","type":"number","index":true,"include_term_vectors":true,"include_in_all":true}]}}}},"default_mapping":{"enabled":false,"dynamic":true},"type_field":"_type","default_type":"_default","default_analyzer":"standard","default_datetime_parser":"dateTimeOptional","default_field":"_all","store_dynamic":false,"index_dynamic":true,"docvalues_dynamic":true,"analysis":{}}
    o:39 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":501,"failOverLog":[[149680438380220,0]]}
    36 ?
    41 ?
    38 ?
    o:33 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":484,"failOverLog":[[81822404584738,0]]}
    o:36 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":492,"failOverLog":[[174103028649261,0]]}
    o:37 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":486,"failOverLog":[[110924524680780,0]]}
    o:41 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":497,"failOverLog":[[155103402616817,0]]}
    o:43 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":507,"failOverLog":[[217538306806458,0]]}
    33 ?
    37 ?
    o:42 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":492,"failOverLog":[[47136605887494,0]]}
    34 ?
    o:35 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":485,"failOverLog":[[181174121062964,0]]}
    40 ?
    o:34 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":499,"failOverLog":[[128188523546156,0]]}
    o:38 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":486,"failOverLog":[[161601347095991,0]]}
    o:40 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":504,"failOverLog":[[38787600199365,0]]}
    35 ?
    39 ?