The Kafka connector provides functionality to stream, filter, and transform events between Couchbase Server and Kafka.
The Couchbase Kafka connector is a plug-in for the Kafka Connect framework. It provides both "source" and "sink" components.
The source connector streams documents from Couchbase Server using the high-performance Database Change Protocol (DCP) and publishes the documents to Kafka topics in near real-time. Whenever you modify a document in Couchbase, the new version is published to Kafka.
The sink connector subscribes to Kafka topics and writes the messages to Couchbase Server.
The connector guarantees "at least one" delivery. In some situations the connector may "rewind" the document stream and restart from an earlier point in history.
Except for rewinding, the connector preserves the order of changes within the same Couchbase partition (or "virtual bucket"). The order between changes in different Couchbase partitions is not guaranteed.
Refer to the Compatibility page for information on compatible versions of Kafka and Couchbase Server.
Couchbase welcomes community contributions to the Kafka connector. The Kafka Connector source code is available on GitHub.