Manage Log Streaming with the Management API

Capella Operational

how-to

You can configure and manage log streaming for App Services using the Couchbase Capella Management API.

For an overview of the Management API, see Manage Deployments with the Management API.
To get started with the Management API, see Get Started with the Management API.
To make an API call, see Make an API Call with the Management API.
For a full reference guide, see Management API Reference.

You can also configure log streaming using the Capella UI. See Enable Log Streaming and Configure Log Streaming for an App Endpoint.

About API Call Examples

In the examples on this page:

$organizationId is the organization ID.
$projectId is the project ID.
$clusterId is the cluster ID.
$appServiceId is the App Service ID.
$apiKeySecret is the API key secret, used as the Bearer token.

The endpoints described on this page all have the same base path: /v4/organizations/{organizationId}/projects/{projectId}/clusters/{clusterId}. For clarity, this is not shown in the instructions, but it’s included in the examples.

Prerequisites

Before you configure log streaming for an App Service:

You must have deployed a Couchbase Capella cluster and a linked App Service.
You must have set up your Log Collector and verified that it’s reachable. See Supported Log Collector Providers.
You must understand the resource considerations and cost implications of log streaming.

Configure Log Streaming for an App Service

You can enable and configure log streaming at the App Service level. This sets up the log collector connection and applies default log levels and filters.

Enable and Configure Log Streaming

To enable and configure log streaming for a specified App Service:

Use the POST /appservices/{appServiceId}/logStreaming endpoint.
Pass the App Service ID as a path parameter.
Pass the required log streaming configuration as the request body. For details, see the request body schema.

The configuration includes:

Log collector type and credentials.
Whether to enable streaming.
Optional App Endpoints to exclude from streaming.

You can exclude specific App Endpoints from streaming by listing them in the disabledAppEndpoints array. For an example, see Example 1.

In all of the following examples, $appServiceId is the App Service ID.

Datadog

In this example:

$YOUR_DATADOG_API_KEY is your Datadog API key.

To find your Datadog API key, see the Datadog Organization Settings documentation.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "datadog",
  "credentials": {
    "apiKey": "$YOUR_DATADOG_API_KEY",
    "url": "https://http-intake.logs.datadoghq.com"
  }
}'

Sumo Logic

Sumo Logic uses a signed collectURL for secure log transmission. To get your collector URL, create an HTTP Source in Sumo Logic and copy the provided HTTP Source Address. For more information, see the Sumo Logic HTTP Source documentation.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "sumologic",
  "credentials": {
    "url": "https://collectors.sumologic.com/receiver/v1/http/$YOUR_COLLECTOR_URL"
  }
}'

Elasticsearch

Capella App Services supports only Elasticsearch versions 8+ with basic auth.

In this example:

$YOUR_ELASTICSEARCH_INSTANCE is the URL of your Elasticsearch instance. Make sure to connect through port 9200.
$YOUR_ELASTIC_USER is the username of the Elastic user account you want to use to connect to your Elasticsearch instance.
$YOUR_ELASTIC_PASSWORD is the password for the account you want to use to connect.

To find your Elasticsearch instance URL, see the Elasticsearch documentation on finding your Elasticsearch connection details or your Elasticsearch deployment settings.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "elasticsearch",
  "credentials": {
    "url": "https://$YOUR_ELASTICSEARCH_INSTANCE.com:9200",
    "username": "$YOUR_ELASTIC_USER",
    "password": "$YOUR_ELASTIC_PASSWORD"
  }
}'

Grafana Loki

In this example:

$YOUR_LOKI_USER is the username of the Grafana Loki user account you want to use to connect.
$YOUR_LOKI_PASSWORD is the password for the Grafana Loki account you want to use to connect.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "loki",
  "credentials": {
    "url": "https://logs-prod-us-central1.grafana.net/loki/api/v1/push",
    "username": "$YOUR_LOKI_USER",
    "password": "$YOUR_LOKI_PASSWORD"
  }
}'

Splunk

In this example:

$YOUR_SPLUNK_HEC_TOKEN is your Splunk HTTP Event Collector (HEC) token.
$YOUR_SPLUNK_INSTANCE is the URL for your Splunk instance.

To find your Splunk HEC token and URL, see the Splunk documentation on setting up HTTP Event Collector.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "splunk",
  "credentials": {
    "token": "$YOUR_SPLUNK_HEC_TOKEN",
    "url": "https://$YOUR_SPLUNK_INSTANCE.com:8088/services/collector"
  }
}'

Dynatrace

In this example:

$YOUR_DYNATRACE_API_TOKEN is your Dynatrace API token.
$YOUR_ENVIRONMENT_ID is your Dynatrace environment ID.

To find your Dynatrace API token and environment ID, see the Dynatrace API authentication documentation and your Dynatrace environment settings.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "dynatrace",
  "credentials": {
    "apiKey": "$YOUR_DYNATRACE_API_TOKEN",
    "url": "https://$YOUR_ENVIRONMENT_ID.live.dynatrace.com/api/v2/logs/ingest"
  }
}'

Custom HTTP

In this example:

$YOUR_CUSTOM_COLLECTOR is the URL of your custom log collector.
$YOUR_COLLECTOR_USER is the username of the account for your custom log collector.
$YOUR_COLLECTOR_PASSWORD is the password of your custom log collector account.

The username and password fields are optional for custom HTTP collectors.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "https",
  "credentials": {
    "url": "https://$YOUR_CUSTOM_COLLECTOR.com/logs",
    "username": "$YOUR_COLLECTOR_USER",
    "password": "$YOUR_COLLECTOR_PASSWORD"
  }
}'

Example 1. Enable log streaming with excluded App Endpoints

In this example, the disabledAppEndpoints array excludes specific App Endpoints from streaming.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "streamingEnabled": true,
  "outputType": "datadog",
  "disabledAppEndpoints": ["test-endpoint", "staging-endpoint"],
  "credentials": {
    "apiKey": "YOUR_DATADOG_API_KEY",
    "url": "https://http-intake.logs.datadoghq.com"
  }
}'

Get Log Streaming Configuration

To view the current log streaming configuration for a specified App Service:

Use the GET /appservices/{appServiceId}/logStreaming endpoint.
Pass the App Service ID as a path parameter.

Example 2. Get log streaming configuration

In this example:

$appServiceId is the App Service ID.

Request

curl -X GET "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Authorization: Bearer $apiKeySecret"

Response

{
  "streamingEnabled": true,
  "logStreamingState": "enabled",
  "outputType": "datadog",
  "disabledAppEndpoints": []
}

Pause and Resume Log Streaming

You can pause log streaming temporarily without removing the configuration, and resume it later.

Pause Log Streaming

To pause log streaming for a specified App Service:

Use the DELETE /appservices/{appServiceId}/logStreaming/activationState endpoint.
Pass the App Service ID as a path parameter.

Example 3. Pause log streaming

In this example:

$appServiceId is the App Service ID.

Request

curl -X DELETE "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming/activationState" \
  -H "Authorization: Bearer $apiKeySecret"

Resume Log Streaming

To resume log streaming for a specified App Service:

Use the POST /appservices/{appServiceId}/logStreaming/activationState endpoint.
Pass the App Service ID as a path parameter.

Example 4. Resume log streaming

In this example:

$appServiceId is the App Service ID.

Request

curl -X POST "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming/activationState" \
  -H "Authorization: Bearer $apiKeySecret"

Disable Log Streaming

To disable log streaming for a specified App Service:

Use the DELETE /appservices/{appServiceId}/logStreaming endpoint.
Pass the App Service ID as a path parameter.

This removes the log streaming configuration and stops all streaming.

Example 5. Disable log streaming

In this example:

$appServiceId is the App Service ID.

Request

curl -X DELETE "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/logStreaming" \
  -H "Authorization: Bearer $apiKeySecret"

Change an App Endpoint’s Log Streaming Configuration

You can change the log streaming configuration on an App Endpoint to customize log levels and filters for specific endpoints.

Change App Endpoint Log Streaming Configuration

To change the log streaming configuration for a specified App Endpoint:

Use the PUT /appservices/{appServiceId}/appEndpoints/{appEndpointName}/logStreaming endpoint.
Pass the App Service ID and App Endpoint name as path parameters.
Pass the log level and filters configuration as the request body. For details, see the request body schema.

Changing log streaming configuration for an App Endpoint causes it to go offline briefly, which may result in temporary downtime for client applications.

Example 6. Configure App Endpoint with Info level and default filters

In this example:

$appServiceId is the App Service ID.
$appEndpointName is the App Endpoint name.

Request

curl -X PUT "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/appEndpoints/$appEndpointName/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "logLevel": "info",
  "logKeys": ["Cache", "Changes", "CRUD", "HTTP", "HTTP+", "Query"]
}'

Example 7. Configure App Endpoint with additional filters

In this example:

$appServiceId is the App Service ID.
$appEndpointName is the App Endpoint name.

This configuration includes additional filters for auth, sync, and DCP operations.

Request

curl -X PUT "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/appEndpoints/$appEndpointName/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "logLevel": "info",
  "logKeys": [
    "Admin",
    "Auth",
    "Cache",
    "Changes",
    "CRUD",
    "DCP",
    "HTTP",
    "HTTP+",
    "Query",
    "Sync",
    "SyncMsg"
  ]
}'

Example 8. Configure App Endpoint with Warning level

In this example:

$appServiceId is the App Service ID.
$appEndpointName is the App Endpoint name.

Setting the log level to warning or error reduces the volume of logs streamed.

Request

curl -X PUT "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/appEndpoints/$appEndpointName/logStreaming" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer $apiKeySecret" \
  -d '{
  "logLevel": "warning",
  "logKeys": ["HTTP", "CRUD", "Sync"]
}'

Get App Endpoint Log Streaming Configuration

To view the current log streaming configuration for a specified App Endpoint:

Use the GET /appservices/{appServiceId}/appEndpoints/{appEndpointName}/logStreaming endpoint.
Pass the App Service ID and App Endpoint name as path parameters.

Example 9. Get App Endpoint log streaming configuration

In this example:

$appServiceId is the App Service ID.
$appEndpointName is the App Endpoint name.

Request

curl -X GET "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/appEndpoints/$appEndpointName/logStreaming" \
  -H "Authorization: Bearer $apiKeySecret"

Response

{
  "logLevel": "info",
  "logKeys": ["Cache", "Changes", "CRUD", "HTTP", "HTTP+", "Query"]
}

Reset App Endpoint Log Streaming to Defaults

To reset the log streaming configuration for a specified App Endpoint to the App Service defaults:

Use the DELETE /appservices/{appServiceId}/appEndpoints/{appEndpointName}/logStreaming endpoint.
Pass the App Service ID and App Endpoint name as path parameters.

Example 10. Reset App Endpoint log streaming configuration

In this example:

$appServiceId is the App Service ID.
$appEndpointName is the App Endpoint name.

Request

curl -X DELETE "https://cloudapi.cloud.couchbase.com/v4/organizations/$organizationId/projects/$projectId/clusters/$clusterId/appservices/$appServiceId/appEndpoints/$appEndpointName/logStreaming" \
  -H "Authorization: Bearer $apiKeySecret"

Manage Log Streaming with the Management API

About API Call Examples

Prerequisites

Configure Log Streaming for an App Service

Enable and Configure Log Streaming

Datadog

Sumo Logic

Elasticsearch

Grafana Loki

Splunk

Dynatrace

Custom HTTP

Get Log Streaming Configuration

Pause and Resume Log Streaming

Pause Log Streaming

Resume Log Streaming

Disable Log Streaming

Change an App Endpoint’s Log Streaming Configuration

Change App Endpoint Log Streaming Configuration

Get App Endpoint Log Streaming Configuration

Reset App Endpoint Log Streaming to Defaults

See Also