A Secondary Index can be saved in either of two ways: standard or memory-optmized.
Memory-optimized index-storage allows high-speed maintenance and scanning; since the index is kept fully in memory at all times. A snapshot of the index is maintained on disk, to permit rapid recovery if node-failures are experienced. To be consistently beneficial, memory-optimized index-storage requires that all nodes running the Index Service have a memory quota sufficient for the number and size of their resident indexes, and for the frequency with which the indexes will be updated and scanned. Memory-optimized index-storage, which is only available in Couchbase Server Enterprise Edition, is supported by the Nitro storage engine.
Memory-optimized index-storage may be less suitable for nodes where memory is constrained; since whenever the Index Service memory-quota is exceeded, indexes on the node can neither be updated nor scanned.
In such circumstances, an error-notification is provided; and although the indexes remain in
ONLINE state, traffic is routed away from the node.
Before index-operations can resume, memory must be freed.
In cases where recovery requires an Index-Service node to be restarted, the node’s resident memory-optimized indexes are rebuilt from the snapshots retained on disk.
Following the node’s restart, these indexes remain in the
BUILDING state until all information has been read into memory: then, final updates are made with the indexes in
Note that once a rebuilt index is thus available, queries with
consistency=at_plus fail, if the specified timestamp exceeds the last timestamp processed by given index.
However, queries with
consistency=unbounded execute normally.
For information on these settings, see Index Availability and Performance.
To rescue a node from an out-of-memory situation, consider taking one or more of the following actions:
Increase the index-memory quota, to give indexes additional memory for request-processing.
Remove less important indexes from the node, to free up memory.
Remove buckets with indexes: removing a bucket automatically removes all the dependent indexes.
Flush buckets that have indexes: flushing a bucket deletes all data in a bucket; and even if there are pending updates not yet processed, flushing causes all indexes to drop their own data.
Note that attempting to delete bucket-data selectively during an out-of-memory condition does not succeed in decreasing memory-usage; since without memory, such requested deletions cannot themselves be processed.
Standard is the default storage-setting for Secondary Indexes: the indexes are saved on disk; in a disk-optimized format that uses both memory and disk for index-update and scanning. In Couchbase Server Enterprise Edition, standard index-storage is supported by the Plasma storage engine; in Couchbase Server Community Edition, it is supported by the ForestDB storage engine.
The performance of standard index storage depends on overall I/O performance. Each index saved with the standard option has two write modes:
Circular Write Mode: Writes changes to the end of the index-file, until the relative index fragmentation exceeds 65%. Block reuse is then triggered: new data is written into stale blocks where possible, rather than to the end of the file, so as to optimize I/O throughput. Full compaction runs in accordance with the value of the Circular write mode with day + time interval trigger setting: see Index Fragmentation. Note, however, that the index-fragmentation data-size is not significantly changed by compaction.
Append-only Write Mode: Writes changes to the end of the index-file, invalidating existing pages within the index file, and requiring frequent, full compaction.
By default, Couchbase Server uses Circular Write Mode for standard index storage. Append-only Write Mode is provided for backwards compatibility with previous versions. These and other storage-settings are described in detail in Configuring Auto-Compaction.
Settings are established at cluster-initialization for all indexes on the cluster, across all buckets. Following cluster-initialization, to change from one setting to the other, all nodes running the Index Service must be removed. If the cluster is single-node, uninstall and reinstall Couchbase Server. If the cluster is multi-node, and only some of the nodes host the Index Service, proceed as follows:
Identify the nodes running the Index Service.
Remove each of the nodes running the Index Service. Note that as Index-Service nodes are removed, so are the indexes they contain; and in consequence, any ongoing queries fail.
Perform a rebalance.
Change the Index-Storage Settings for the cluster.
Add new Index-Service nodes, and confirm the revised storage mode.
For information on adding and removing nodes, and on rebalancing a cluster, see Manage Nodes and Clusters.