Storage Engines

      +
      Capella supports two different backend storage mechanisms: Couchstore and Magma. It is important to understand which backend storage is best suited for your requirements.

      Couchstore

      Couchstore is the default bucket storage engine that has been in use for more than ten years. It’s optimised for high performance with large data sets while using fewer system resources (the minimum bucket size for Couchstore backend is 100 MiB). If you have a small data set that can fit in memory, then you should consider using Couchstore.

      Magma

      Magma is Capella’s latest storage engine that is designed for high performance with very large datasets that do not fit in memory. It is ideal for use-cases that rely primarily on disk access. The performance of disk access will be as good as the underlying disk subsystems. Magma can work with very low amounts of memory for large datasets: e.g. for a node holding 5 TiB of data, Magma can be used with only 64 GiB RAM. It is especially suited for data sets that will not fit into available memory.

      Couchstore and Magma at a Glance

      Couchstore Magma

      Minimum bucket memory quota

      100 MiB

      1 GiB[1]

      Minimum memory to data ratio

      10%

      1%

      Maximum data per node

      1.6 TiB

      16 TiB

      1

      Magma’s minimum memory requirement is higher at 1GiB per node due to the more complex data structures it has to maintain. We are working on reducing this in a future release.

      When should you use Couchstore?

      The choice of Couchstore or Magma is set at the bucket level when the bucket is created. A single Capella cluster can have a mix of Couchstore and Magma buckets.

      You should use the Couchstore backend if:

      • you have a dataset with a working set that will fit in available memory (and the working set is > 20%)

      • you’re running the Capella server on a low-end instance.

      • your bucket needs to support full text searching, eventing, or analytics.

      When should you use Magma?

      You should use the Magma backend if:

      • Your working set is much larger than the available memory, and you need disk access speed only.

      • You need to store and access large amounts of data (several terabytes) using the lowest amount of memory.

      • Your application(s) make heavy use of transactions with persistence-based durability.

      Migrating a Couchstore bucket to Magma

      If you have an existing Couchstore bucket that you would like to migrate to Magma, you have to create a new bucket with Magma storage engine and then copy the data.

      To copy the data, you can either use XDCR or you can backup the Couchstore bucket and restore to the new Magma bucket.