Release Notes

Release notes for the 2.0 version of the Kafka connector.

Couchbase Kafka Connector 2.0.0 GA (30 March 2016)

Version 2.0.0 is the general availability (GA) release of the Kafka connector. It contains important stability fixes, and also new API.

New features and behavioral changes

This release contains the following features:


    All these issues share a common root cause. If the Kafka consumer handles new events at a slower rate than the DCP producer emits them, then all additional messages are dropped silently. Eventually, because most of these messages carry native buffers that cannot be freed by GC, the connector eventually dies from out-of-memory exceptions.

    Fixing these issues required a new implementation of the DCP handler inside the JVM Core. Some classes were removed (such as BucketStreamAggregator and BucketStreamAggregatorState) and new, cleaner alternatives were introduced (such as ConnectorState and StreamState). These new classes encapsulate only one point in history, in contrast to the old classes that carried a starting and an ending sequence number. These alternatives replace the unintuitive direction and run-mode API with a method that accepts two states plus some helper methods that make it easier to initialize ConnectorState. For example, the following code:

    BucketStreamAggregatorState state = connector.buildState(Direction.TO_CURRENT);, RunMode.RESUME);

    can be simplified as

    ConnectorState startState = connector.startState();
    ConnectorState currentState = connector.currentState();, currentState);

    and the snippet

    BucketStreamAggregatorState state = connector.buildState(Direction.FROM_CURRENT);, RunMode.LOAD_AND_RESUME);


    ConnectorState loadedState = connector.loadState();
    ConnectorState endState = connector.endState();, endState);
  • KAFKAC-32:

    Zookeeper state serializer now uses the forward slash / as a path separator, which makes it platform independent. Before this change, a connector running on Windows used a backward slash and other platforms used a forward slash.

Known issues

This release does not handle cluster topology changes. For example, it does not add streams for partitions that are migrated between nodes during a rebalance. The API for the necessary enhancements to handle cluster topology changes is still under discussion. Corresponding Jira tickets are: JVMCBC-307 and KAFKAC-7.