Source Configuration Options

    +

    Reference of the source connector options.

    Connection

    couchbase.seed.nodes

    Addresses of Couchbase Server nodes, delimited by commas.

    If a custom port is specified, it must be the KV port (which is normally 11210 for insecure connections, or 11207 for secure connections).

    • Type: list

    • Importance: high

    couchbase.username

    Name of the Couchbase user to authenticate as.

    • Type: string

    • Importance: high

    couchbase.password

    Password of the Couchbase user.

    May be overridden with the KAFKA_COUCHBASE_PASSWORD environment variable.

    • Type: password

    • Importance: high

    couchbase.bucket

    Name of the Couchbase bucket to use.

    • Type: string

    • Importance: high

    couchbase.network

    The network selection strategy for connecting to a Couchbase Server cluster that advertises alternate addresses.

    A Couchbase node running inside a container environment (like Docker or Kubernetes) might be configured to advertise both its address within the container environment (known as its "default" address) as well as an "external" address for use by clients connecting from outside the environment.

    Setting the 'couchbase.network' config property to 'default' or 'external' forces the selection of the respective addresses. Setting the value to 'auto' tells the connector to select whichever network contains the addresses specified in the 'couchbase.seed.nodes' config property.

    • Type: string

    • Default: auto

    • Importance: medium

    couchbase.bootstrap.timeout

    On startup, the connector will wait this long for a Couchbase connection to be established. If a connection is not established before the timeout expires, the connector will terminate.

    • Type: string

    • Default: 30s

    • Valid Values: An integer followed by a time unit (ms = milliseconds, s = seconds, m = minutes, h = hours, d = days). For example, to specify 30 minutes: 30m

    • Importance: medium

    Security

    couchbase.enable.tls

    Use secure connection to Couchbase Server. If true, 'couchbase.trust.store.path' and 'couchbase.trust.store.password' must also be provided.

    couchbase.enable.hostname.verification

    Set this to false to disable TLS hostname verification for Couchbase connections. Less secure, but might be required if for some reason you can’t issue a certificate whose Subject Alternative Names match the hostname used to connect to the server. Only disable if you understand the impact and can accept the risks.

    • Since: 4.0.4

    • Type: boolean

    • Default: true

    • Importance: medium

    couchbase.trust.store.path

    Absolute filesystem path to the Java keystore holding the CA certificate used by Couchbase Server.

    If you want to use a PEM file instead of a Java keystore, specify couchbase.trust.certificate.path instead.

    • Type: string

    • Default: ""

    • Importance: medium

    couchbase.trust.store.password

    Password for accessing the trust store.

    May be overridden with the KAFKA_COUCHBASE_TRUST_STORE_PASSWORD environment variable.

    • Type: password

    • Default: [hidden]

    • Importance: medium

    couchbase.trust.certificate.path

    Absolute filesystem path to the PEM file containing the CA certificate used by Couchbase Server.

    If you want to use a Java keystore instead of a PEM file, specify couchbase.trust.store.path instead.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Since: 4.0.5

    • Type: string

    • Default: ""

    • Importance: medium

    couchbase.client.certificate.path

    Absolute filesystem path to a Java keystore or PKCS12 bundle holding the private key and certificate chain to use for client certificate authentication (mutual TLS).

    If you supply a value for this config property, the couchbase.username and couchbase.password properties will be ignored.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Since: 4.0.4

    • Type: string

    • Default: ""

    • Importance: medium

    couchbase.client.certificate.password

    Password for accessing the client certificate.

    May be overridden with the KAFKA_COUCHBASE_CLIENT_CERTIFICATE_PASSWORD environment variable.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Since: 4.0.4

    • Type: password

    • Default: [hidden]

    • Importance: medium

    Logging

    couchbase.log.redaction

    Determines which kinds of sensitive log messages from the Couchbase connector will be tagged for later redaction by the Couchbase log redaction tool. NONE = no tagging; PARTIAL = user data is tagged; FULL = user, meta, and system data is tagged.

    • Type: string

    • Default: NONE

    • Valid Values: One of [NONE, PARTIAL, FULL]

    • Importance: medium

    couchbase.log.document.lifecycle

    If true, document lifecycle milestones will be logged at INFO level instead of DEBUG. Enabling this feature lets you watch documents flow through the connector. Disabled by default because it generates many log messages.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Since: 4.0.5

    • Type: boolean

    • Default: false

    • Importance: medium

    Source Behavior

    couchbase.topic

    Name of the Kafka topic to publish data to. This is a format string that recognizes the following placeholders:

    ${bucket} refers to the bucket containing the document.

    ${scope} refers to the scope containing the document.

    ${collection} refers to the collection containing the document.

    The behavior of the "scope" and "collection" placeholders may change in a patch release without notice.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Type: string

    • Default: ${bucket}.${scope}.${collection}

    • Importance: medium

    couchbase.source.handler

    The class name of the source handler to use. The source handler determines how the Couchbase document is converted into a Kafka record.

    To publish JSON messages identical to the Couchbase documents, use com.couchbase.connect.kafka.handler.source.RawJsonSourceHandler and set value.converter to org.apache.kafka.connect.converters.ByteArrayConverter.

    • Type: class

    • Importance: medium

    couchbase.event.filter

    The class name of the event filter to use. The event filter determines whether or not a database change event is ignored.

    • Type: class

    • Default: com.couchbase.connect.kafka.filter.AllPassFilter

    • Importance: medium

    couchbase.batch.size.max

    Controls maximum size of the batch for writing into topic.

    • Type: int

    • Default: 2000

    • Importance: medium

    couchbase.no.value

    If true, Couchbase Server will omit the document content when telling the connector about a change. The document key and metadata will still be present.

    If you don’t care about the content of changed documents, enabling this option is a great way to reduce the connector’s network bandwidth and memory usage.

    • Type: boolean

    • Default: false

    • Importance: medium

    couchbase.connector.name.in.offsets

    When true, the connector’s offsets are saved under a key that includes the connector name. This is redundant, since the Kafka Connect framework already isolates the offsets of connectors with different names.

    Set this to true only if you’ve previously deployed the connector to production with this set to true, and you do not wish to restart streaming from the beginning. Otherwise you should ignore this property.

    • Type: boolean

    • Default: false

    • Importance: medium

    couchbase.stream.from

    Controls when in history then connector starts streaming from.

    • Type: string

    • Default: SAVED_OFFSET_OR_BEGINNING

    • Valid Values: One of [SAVED_OFFSET_OR_BEGINNING, SAVED_OFFSET_OR_NOW, BEGINNING, NOW]

    • Importance: medium

    couchbase.scope

    If you wish to stream from all collections within a scope, specify the scope name here.

    If you specify neither "couchbase.scope" nor "couchbase.collections", the connector will stream from all collections of all scopes in the bucket.

    Requires Couchbase Server 7.0 or later.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Type: string

    • Default: ""

    • Importance: medium

    couchbase.collections

    If you wish to stream from specific collections, specify the qualified collection names here, separated by commas. A qualified name is the name of the scope followed by a dot (.) and then the name of the collection. For example: "tenant-foo.invoices".

    If you specify neither "couchbase.scope" nor "couchbase.collections", the connector will stream from all collections of all scopes in the bucket.

    Requires Couchbase Server 7.0 or later.

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Type: list

    • Default: ""

    • Importance: medium

    Database Change Protocol

    couchbase.compression

    To reduce bandwidth usage, Couchbase Server 5.5 and later can send documents to the connector in compressed form. (Messages are always published to the Kafka topic in uncompressed form, regardless of this setting.)

    • Type: string

    • Default: ENABLED

    • Valid Values: One of [DISABLED, FORCED, ENABLED]

    • Importance: medium

    couchbase.persistence.polling.interval

    How frequently to poll Couchbase Server to see which changes are ready to be published. Specify 0 to disable polling, or an integer followed by a time qualifier (example: 100ms)

    • Type: string

    • Default: 100ms

    • Valid Values: An integer followed by a time unit (ms = milliseconds, s = seconds, m = minutes, h = hours, d = days). For example, to specify 30 minutes: 30m

    • Importance: medium

    couchbase.flow.control.buffer

    How much heap space should be allocated to the flow control buffer. Specify an integer followed by a size qualifier (example: 128m)

    • Type: string

    • Default: 128m

    • Valid Values: An integer followed by a size unit (b = bytes, k = kilobytes, m = megabytes, g = gigabytes). For example, to specify 64 megabytes: 64m

    • Importance: medium

    couchbase.xattrs

    Should filters and source handlers have access to a document’s extended attributes?

    UNCOMMITTED; this feature may change in a patch release without notice.

    • Since: 4.0.1

    • Type: boolean

    • Default: false

    • Importance: medium

    Parent topic: Kafka Connector

    Previous topic: Quickstart