Storage Engines
- Capella Operational
Capella supports two different backend storage mechanisms: Couchstore and Magma. It is important to understand which backend storage is best suited for your requirements.
Couchstore
Couchstore is the default bucket storage engine that has been in use for more than ten years. It’s optimised for high performance with large data sets while using fewer system resources (the minimum bucket size for Couchstore backend is 100 MiB). If you have a small data set that can fit in memory, then you should consider using Couchstore.
Magma
Magma is Capella’s latest storage engine that is designed for high performance with very large datasets that do not fit in memory. It is ideal for use-cases that rely primarily on disk access. The performance of disk access will be as good as the underlying disk subsystems. Magma can work with very low amounts of memory for large datasets: e.g. for a node holding 5 TiB of data, Magma can be used with only 64 GiB RAM. It is especially suited for data sets that will not fit into available memory.
Couchstore and Magma at a Glance
Couchstore | Magma | |
---|---|---|
Minimum bucket memory quota |
100 MiB |
1 GiB[1] |
Minimum memory to data ratio |
10% |
1% |
Maximum data per node |
1.6 TiB |
1 |
Magma’s minimum memory requirement is higher at 1GiB per node due to the more complex data structures it has to maintain. We are working on reducing this in a future release. |
When to Use Couchstore
The choice of Couchstore or Magma is set at the bucket level when the bucket is created. A single Capella cluster can have a mix of Couchstore and Magma buckets.
You should use the Couchstore backend if:
-
you have a dataset with a working set that will fit in available memory (and the working set is > 20%)
-
you’re running the Capella server on a low-end instance.
For single node operational clusters, your bucket is automatically set with the Couchstore backend. Single node clusters cannot use the Magma backend. |
When to Use Magma
You should use the Magma backend if:
-
Your working set is much larger than the available memory, and you need disk access speed only.
-
You need to store and access large amounts of data (several terabytes) using the lowest amount of memory.
-
Your application(s) make heavy use of transactions with persistence-based durability.
Migrating a Couchstore Bucket to Magma
-
Couchbase Server 7.6 or later
-
Couchbase Server 7.2
If your cluster in Capella uses Couchbase Server 7.6 or later, migrate your Couchstore buckets to Magma by making a PUT - Migrate Buckets call to the Management API.
PUT - Migrate Buckets can migrate 1 or more Couchstore buckets to Magma while your cluster is running. The API runs a swap rebalance of your cluster’s Data Service nodes. Your cluster shows a rebalancing state during the operation.
Before using PUT - Migrate Buckets, check that the buckets you’re migrating meet the requirements for Magma.
If the bucket does not meet or exceed those requirements, the API call returns "errorType": "MagmaBucketSizeTooSmall"
.