Use Customer-Managed Encryption Keys (CMEK) at Rest

  • Capella Operational
  • how-to
    +
    Capella encrypts cluster volumes at rest. You can move control of the keys from Couchbase to your own key management system (KMS).

    By default, all clusters in Couchbase Capella use your cluster’s cloud provider KMS to encrypt cluster volumes at rest. Instead of using this Couchbase-managed solution, you can create your own encryption keys for data at rest using customer-managed encryption keys (CMEK). By managing your encryption keys, you control their configuration, rotation cycles, geographic storage location, and can directly revoke them.

    Backups and CMEK

    Customer-managed encryption keys for bucket backups are not available at this time. To manage your encryption keys for a bucket backup, you can run your backup outside Capella. If you use CMEK on your cluster for encryption at rest, Capella encrypts your cluster backups with the same key used to encrypt your cluster storage.

    You cannot use CMEK for Azure cluster backups with cross-region copies. Only AWS clusters support encrypting cross-region backup copies with CMEK.

    Do not delete a Key Management System (KMS) Key ID until any cluster backups using that Key ID have expired. If you delete the Key ID, your backup becomes unusable. Encryption Key IDs must be enabled and available to restore an encrypted backup to a Capella cluster.

    Key Management Provider Support

    Capella supports the following customer key management providers for encryption at rest:

    Capella supports both the Standard and Premium tier of Azure Key Vault, including Managed Hardware Security Module (HSM) keys.

    CMEK on Azure clusters is available upon request. Contact Couchbase Capella Support.

    Couchbase recommends using 1 key per cluster and not sharing keys across clusters.

    CMEK is available to clusters using the Enterprise plan.

    Configure Capella to Use Customer Key Management

    To use CMEK on Capella, you must:

    You must add and manage your keys through the Management API. Through the Management API, you can:

    • Enable CMEK for a specific cloud service provider.

    • Add a customer-managed encryption key.

    • Get the details of a key based on its ID.

    • Get a list of keys created in an organization.

    • Rotate a key to replace an existing key with a new one.

    • Delete an existing key.

    • Associate a key with a cluster to enable encryption.

    • Disassociate a key with a cluster.

    For the details and requirements of each call, see the Management API Reference. If you’re new to the Management API, see Get Started with the Management API

    Prerequisites

    • AWS

    • GCP

    • Azure

    • You must have the Organization Owner role to create, rotate, and delete customer-managed encryption keys. The Project Creator and Organization Owner roles can associate a key with a cluster. All organization members can retrieve the list of keys in an organization and view the details of a key.

    • You have a Management API key with the Organization Owner role.

    • You have deployed a cluster with AWS as your cloud service provider or enabled CMEK for AWS in your organization.

    • You have:

      • A symmetric encryption key in AWS KMS that’s in the same region as your cluster.

      • The unique Capella AWS account ID that’s associated with your organization. Get this account ID by making a GET - Get Cloud Accounts call to the Couchbase Capella Management API.

        Use this account ID when adding CMEK to other AWS clusters in your organization.

      • A key policy with the following permissions:

        • DescribeKey

        • GenerateDataKeyWithoutPlainText

        • Decrypt

        • ReEncrypt*

        • CreateGrant with the kms:GrantIsForAWSResource condition set to the Capella AWS account ID associated with your organization.

        • The principal set to the Capella AWS account ID that’s associated with your organization.

          Example
          {
                      "Sid": "Allow use of the key",
                      "Effect": "Allow",
                      "Principal": {
                          "AWS": "arn:aws:iam::<capella-account-id>:root"
                      },
                      "Action": [
                          "kms:DescribeKey",
                          "kms:GenerateDataKeyWithoutPlainText",
                          "kms:Decrypt",
                          "kms:ReEncrypt*"
                      ],
                      "Resource": "*"
                  },
                  {
                      "Sid": "Allow attachment of persistent resources",
                      "Effect": "Allow",
                      "Principal": {
                          "AWS": "arn:aws:iam::<capella-account-id>:root"
                      },
                      "Action": "kms:CreateGrant",
                      "Resource": "*",
                      "Condition": {
                          "Bool": {
                              "kms:GrantIsForAWSResource": "true"
                          }
                      }
                  }
          For more information about cross-account KMS encryption key access, see Creating KMS keys that other accounts can use.
      • The Amazon Resource Name (ARN) of your KMS encryption key.

    • You must have the Organization Owner role to create, rotate, and delete customer-managed encryption keys. The Project Creator and Organization Owner roles can associate a key with a cluster. All organization members can retrieve the list of keys in an organization and view the details of a key.

    • You have a Management API key with the Organization Owner role.

    • You have deployed a cluster with GCP as your cloud service provider or enabled CMEK for GCP in your organization.

    • You have:

      • A symmetric encryption key in GCP KMS.

      • The Key Version Resource ID of your GCP KMS encryption key.

      • The unique Capella Google service account ID that’s associated with your organization. Get this account ID by making a GET - Get Cloud Accounts call to the Couchbase Capella Management API.

        Use this service account ID when adding CMEK to other GCP clusters in your organization.

      • A service account on the key with the following permissions:

        • Cloud KMS CryptoKey Encrypter/Decrypter

        • The principal set to the Capella Google service account ID that’s associated with your organization.

          Add the principal to your key in the following format:

          rc-cluster-admin@<gcp-capella-project>.iam.gserviceaccount.com

          Replace <gcp-capella-project> with the value of "gcp-capella-project" from the GET - Get Cloud Accounts Management API call.

    • You must have the Organization Owner role to create, rotate, and delete customer-managed encryption keys. The Project Creator and Organization Owner roles can associate a key with a cluster. All organization members can retrieve the list of keys in an organization and view the details of a key.

    • You have a Management API key with the Organization Owner role.

    • You have enabled CMEK for Azure within your organization.

    • You have:

      • A Key Vault created in the same Azure region as your Azure cluster. For more information about how to create a Key Vault, see the Azure documentation.

      • A symmetric RSA encryption key in your Azure Key Vault. The key must be 2048-bit, 3072-bit, or 4096-bit.

        For more information about how to create a key in Azure Key Vault, see the Azure documentation.

      • Your vault or managed HSM URI, which you can view when you create a new key vault or on the Azure Portal.

    Enable Customer-Managed Encryption Keys on Capella

    To use CMEK on Capella, you need to enable customer-managed keys for your specific cloud service provider by making a call to the PUT - Enable CMEK for Cloud Services Provider endpoint.

    If have not yet deployed a cluster with AWS or GCP in your organization, and want to use customer managed-encryption keys for a new cluster, you need to enable customer managed-encryption keys for your cloud service provider, first.

    If you have already deployed a cluster with AWS or GCP, you do not need to enable CMEK for that cloud service provider.

    You must always enable customer-managed keys if you want to use a customer-managed key for Azure clusters. CMEK for Azure is currently available upon request.

    Using CMEK on Azure requires additional configuration after enabling customer-managed keys.

    • AWS

    • GCP

    • Azure

    • $ORGID is the organization ID.

    • $TOKEN is the API key token.

      HTTP Request
      curl --request POST \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/providers" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
          "cloudProvider": "aws"
        }'
    • $ORGID is the organization ID.

    • $TOKEN is the API key token.

      HTTP Request
      curl --request POST \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/providers" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
          "cloudProvider": "gcp"
        }'

    To use CMEK on Azure, you need to get and configure Capella’s Azure CMEK application in your Azure tenant:

    1. Enable CMEK for your organization:

      • $ORGID is the organization ID.

      • $TOKEN is the API key token.

        HTTP Request
        curl --request POST \
          --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/providers" \
          --header "Authorization: Bearer $TOKEN" \
          --header 'Content-Type: application/json' \
          --data '{
            "cloudProvider": "azure"
          }'
    2. Get your Azure CMEK application ID by making a GET - Get Azure Application ID call to the Couchbase Capella Management API:

      • $ORGID is the organization ID.

      • $TOKEN is the API key token.

        HTTP Request
        curl --request POST \
          --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmekAzureApplication" \
          --header "Authorization: Bearer $TOKEN" \
          --header 'Content-Type: application/json' \
        HTTP Response
        {
            "id": "$AZURE_APP_ID"
        }
    3. Create a service principal in your Azure tenant by running the following command in the Azure CLI, replacing $AZURE_APP_ID with the response from the previous API call:

      az ad sp create --id $AZURE_APP_ID
    4. Assign the Key Vault Crypto Service Encryption User role to this application ID as a service principal in your Key Vault’s Access Control (IAM). For more information about how to assign roles in Azure, see the Azure documentation.

    Add a Customer-Managed Encryption Key

    To add a customer-managed encryption key to your organization, make a POST - Create Key Metadata call to the Capella Management API. For example:

    • AWS

    • GCP

    • Azure

    • $ORGID is the organization ID.

    • $TOKEN is the API key token.

    • $ARN is the Amazon Resource Name (ARN) of the KMS encryption key.

    • $CMEKID is the Capella ID of the customer-managed encryption key.

      HTTP Request
      curl --request POST \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/cmek" \
        --header "Authorization: Bearer $apiKeySecret" \
        --header 'Content-Type: application/json' \
        --data '{
        "name": "Key name",
        "description": "Description of the key",
        "config": {
      	  "arn": "$ARN"
      	  }
        }'
      HTTP Response
      {
          "id": "$cmekId"
      }
    • $ORGID is the organization ID.

    • $TOKEN is the API key token.

    • $RESOURCE_NAME is the Key Version Resource ID of your GCP KMS encryption key.

    • $CMEKID is the Capella ID of the customer-managed encryption key.

      HTTP Request
      curl --request POST \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
        "name": "Key name",
        "description": "Description of the key",
        "config": {
      	  "resourceName": "$RESOURCE_NAME"
      	  }
        }'
      HTTP Response
      {
          "id": "$CMEKID"
      }
    • $ORGID is the organization ID.

    • $TOKEN is the API key token.

    • $VAULT_URI is your vault or managed HSM URI, which you can view when you create a new key vault or on the Azure Portal. Your URI must include your key name and key version.

    • $CMEKID is the Capella ID of the customer-managed encryption key.

      HTTP Request
      curl --request POST \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
        "name": "Key name",
        "description": "Description of the key",
        "config": {
      	  "keyLocation": "$VAULT_URI",
          "region": "eastus"
      	  }
        }'
      For Azure encryption keys, you must also provide the region where your Azure cluster and Key Vault are deployed.
      HTTP Response
      {
          "id": "$CMEKID"
      }

    Create a Cluster Using a Customer-Managed Encryption Key

    After adding a customer-managed encryption key, you can use this key when creating new clusters with the Management API. Do this by adding the cmekId field with the Capella ID of the customer-managed encryption key to the payload when creating a cluster.

    For more information, see POST - Create Cluster.

    Apply a Customer-Managed Encryption Key to an Existing Cluster

    To apply a customer-managed encryption key to an existing cluster, make a POST - Associate Key with Cluster call to the Capella Management API. This call needs:

    • Your organization ID.

    • The ID of the project where you created your cluster.

    • The ID of the cluster you want to apply the encryption to.

    • The CMEK ID—​the Capella ID of the customer-managed encryption key you’re using.

    The CMEK ID is in the response when you add a new key, and you can retrieve it by making a GET - List Key Metadata call to the Capella Management API.

    When the POST - Associate Key with Cluster call is successful, the cluster rebalances to encrypt the existing data with the customer-provided key. Rebalancing does not cause any downtime.

    You can confirm that your cluster is using the encryption key by making a GET - Get Cluster call and finding cmekId in the response.

    Key Rotation

    Do not use key rotation in AWS or GCP. KMS encryption key expiry is not supported through AWS or GCP. Only rotate keys out of Capella and remove them from your AWS or GCP key management provider once your cluster has a new KMS encryption key. If a customer-managed encryption key becomes unavailable, the related cluster experiences availability issues and data loss.

    For clusters on Azure, you can do an in-place rotation of your key in Azure Key Vault, instead of rotating your key through Capella. See Update Your Key Version in Azure Key Vault.

    Capella cannot rotate customer-managed encryption keys. You must do this using the Capella Management API.

    To rotate your customer-managed encryption keys:

    1. Create a new KMS encryption key in your cloud service provider.

    2. Make a PUT - Rotate Key call to the Capella Management API with the resource name of your new KMS encryption key.

      After associating a new key with your cluster, Capella rebalances all nodes in that cluster. Rebalancing does not cause any downtime. Only remove the old key from your key management provider once your cluster is using the new KMS encryption key.

    For example, to rotate your customer-managed encryption key:

    • AWS

    • GCP

    • Azure

    • $ORGID is the organization ID.

    • $CMEKID is the Capella ID of the customer-managed encryption key you want to rotate.

    • $TOKEN is the API key token.

    • $ARN is the Amazon Resource Name (ARN) of the KMS encryption key.

      HTTP Request
      curl --request PUT \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/$CMEKID" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
        "config": {
      	  "arn": "$ARN"
      	  }
        }'
    • $ORGID is the organization ID.

    • $CMEKID is the Capella ID of the customer-managed encryption key you want to rotate.

    • $TOKEN is the API key token.

    • $RESOURCE_NAME is the Key Version Resource ID of your GCP KMS encryption key.

      HTTP Request
      curl --request PUT \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/$CMEKID" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
        "config": {
      	  "resourceName": "$RESOURCE_NAME"
      	  }
        }'
    You can do an in-place rotation of your key in Azure Key Vault, instead of rotating your key through Capella. See Update Your Key Version in Azure Key Vault.
    • $ORGID is the organization ID.

    • $CMEKID is the Capella ID of the customer-managed encryption key you want to rotate.

    • $TOKEN is the API key token.

    • $VAULT_URI is your vault or managed HSM URI, which you can view when you create a new key vault or on the Azure Portal. Your URI must include your key name and key version.

      HTTP Request
      curl --request PUT \
        --url "https://cloudapi.cloud.couchbase.com/v4/organizations/$ORGID/cmek/$CMEKID" \
        --header "Authorization: Bearer $TOKEN" \
        --header 'Content-Type: application/json' \
        --data '{
        "config": {
      	  "keyLocation": "$VAULT_URI",
          "region": "eastus"
      	  }
        }'
      For Azure encryption keys, you must also provide the region where your Azure cluster and Key Vault are deployed.

    You can rotate customer-managed encryption keys once every 30 days. Contact Couchbase Support if you need to rotate keys earlier.

    Update Your Key Version in Azure Key Vault

    If you’re using CMEK on Azure clusters, you can do an in-place rotation of your key in Azure Key Vault and generate a new version of your key. Capella will automatically handle the new key version.

    To manually update the key version for your Azure clusters:

    1. Log in to the Azure Portal.

    2. Go to Key Vault.

    3. In your list of keys, click the key you want to update to a new version.

    4. Click New Version.

    Azure Key Vault generates a new version of your selected key. Capella starts using the new version of your key within 1 hour.

    For more information about how to automatically configure key version updates in your Key Vault, see the Microsoft Azure documentation.