A newer version of this documentation is available.

View Latest

Storage Settings

      A Secondary Index can be saved in either of two ways: memory-optimized or standard.

      Memory-Optimized Index Storage

      Memory-optimized index-storage is supported by the Nitro storage engine, and is only available in Couchbase Server Enterprise Edition.

      Memory-optimized index-storage allows high-speed maintenance and scanning; since the index is kept fully in memory at all times. A snapshot of the index is maintained on disk, to permit rapid recovery if node-failures are experienced. To be consistently beneficial, memory-optimized index-storage requires that all nodes running the Index Service have a memory quota sufficient for the number and size of their resident indexes, and for the frequency with which the indexes will be updated and scanned.

      Memory-optimized index-storage may be less suitable for nodes where memory is constrained; since whenever the Index Service memory-quota is exceeded, indexes on the node can neither be updated nor scanned.

      If indexer RAM usage goes above 75% of the Index Service memory-quota, an error-notification is provided. If indexer RAM usage then goes above 95% of the Index Service memory-quota, the indexer goes into the Paused mode on that node; and although the indexes remain in Active state, traffic is routed away from the node.

      Before index-operations can resume, memory must be freed. When the indexer RAM usage drops below 80% of the Index Service memory-quota, the indexer goes into Active mode again on that node.

      To resume indexing operations on a node where the Indexer has paused due to low memory, consider taking one or more of the following actions:

      • Increase the index-memory quota, to give indexes additional memory for request-processing.

      • Remove less important indexes from the node, to free up memory.

      • Remove buckets with indexes: removing a bucket automatically removes all the dependent indexes.

      • Flush buckets that have indexes: flushing a bucket deletes all data in a bucket; and even if there are pending updates not yet processed, flushing causes all indexes to drop their own data.

        Note that attempting to delete bucket-data selectively during an out-of-memory condition does not succeed in decreasing memory-usage; since without memory, such requested deletions cannot themselves be processed.

      In cases where recovery requires an Index-Service node to be restarted, the node’s resident memory-optimized indexes are rebuilt from the snapshots retained on disk. Following the node’s restart, these indexes remain in the Warmup state until all information has been read into memory: then, final updates are made with the indexes in Active state. Note that once a rebuilt index is thus available, queries with consistency=request_plus or consistency=at_plus fail, if the specified timestamp exceeds the last timestamp processed by given index. [1] However, queries with consistency=unbounded execute normally. For information on these settings, see Index Availability and Performance.

      Standard Index Storage

      Standard is the default storage-setting for Secondary Indexes: the indexes are saved on disk; in a disk-optimized format that uses both memory and disk for index-update and scanning. The performance of standard index storage depends on overall I/O performance.

      In Couchbase Server Enterprise Edition, standard index-storage is supported by the Plasma storage engine. In this case, compaction is handled automatically.

      In Couchbase Server Community Edition, standard index-storage is supported by the ForestDB storage engine. In this case, each index saved with the standard option has two write modes:

      Circular Write Mode

      Writes changes to the end of the index-file, until the relative index fragmentation exceeds 65%. Block reuse is then triggered: new data is written into stale blocks where possible, rather than to the end of the file, so as to optimize I/O throughput. Full compaction runs in accordance with the value of the Circular write mode with day + time interval trigger setting: see Index Fragmentation. Note, however, that the index-fragmentation data-size is not significantly changed by compaction.

      Append-only Write Mode

      Writes changes to the end of the index-file, invalidating existing pages within the index file, and requiring frequent, full compaction.

      By default, Couchbase Server Community Edition uses Circular Write Mode for standard index storage. Append-only Write Mode is provided for backwards compatibility with previous versions.

      Other storage-settings are described in detail in Configuring Auto-Compaction.

      Changing Index-Storage Settings

      Settings are established at cluster-initialization for all indexes on the cluster, across all buckets. Following cluster-initialization, to change from one setting to the other, all nodes running the Index Service must be removed. If the cluster is single-node, uninstall and reinstall Couchbase Server. If the cluster is multi-node, and only some of the nodes host the Index Service, proceed as follows:

      1. Identify the nodes running the Index Service.

      2. Remove each of the nodes running the Index Service. Note that as Index-Service nodes are removed, so are the indexes they contain; and in consequence, any ongoing queries fail.

      3. Perform a rebalance.

      4. Change the Index-Storage Settings for the cluster.

      5. Add new Index-Service nodes, and confirm the revised storage mode.

      For information on adding and removing nodes, and on rebalancing a cluster, see Manage Nodes and Clusters.

      1. In fact, queries in this case wait for a consistent snapshot to be available and time out, rather than fail immediately.