In response to increasing volumes of data being sent over the wire, Couchbase Data Platform now provides data compression between the SDK and Couchbase Server.


      Documents may already be stored compressed with Snappy. Now documents may be passed from client applications to the server in compressed form, saving around 40% in bandwidth, and also transmission time. These operations take place automatically, after HELLO negotiation with the server, and do not require any changes on the client side, beyond updating to a recent SDK version with compression support (see release notes).


      The document must be below 20MiB in size in both compressed and uncompressed form. Compression is only available in the Enterprise Edition of Couchbase Data Platform.

      This size limit is enforced by Couchbase Server; in practice it will affect very few users, as most JSON documents are considerably smaller. A compressed doument of just under 20MB, which is greater than 20,971,520 bytes (20 MiB) when uncompressed, will be rejected by the server as follows:

      • Couchbase Server decompresses the document to check that it is valid JSON, and is correctly compressed with Snappy, and at this point measures it against max data size (20 MiB).

      • If the decompressed value’s size exceeds this limit, the mutation is failed with a "too big" error code (E2BIG code 3).

      Therefore, where necessary, enforce document size limits in your application on uncompressed documents.

      Operating Modes

      The three modes in which compression can be used, “off”, “passive” and “active”, are per bucket configuration settings on the cluster. Depending on how it is set, the HELLO negotiation will fail or succeed. The HELLO flag for compression has the value 0x0a and is defined as:

      Table 1. Compression Operating Modes

      Sending SNAPPY feature with HELLO




      Sending compressed data when SNAPPY feature enabled




      Sending compressed data when SNAPPY feature disabled

      reject as invalid

      reject as invalid

      reject as invalid

      Receiving data which were previously compressed on the server

      server inflates and sends plain data

      server sends compressed data

      server sends compressed data

      Receiving data which were not previously compressed on the server

      server sends plain data

      server sends plain data

      server might send compressed data (the compression is done in the background on the server)

      Acceptance Guarantee

      The server may change compression settings for the bucket at any time, but it is guaranteed that once the socket negotiates compression through HELLO, the server will never reject compressed data, even if the bucket setting has been changed.

      Minimum Size

      While the tiniest of documents will not be reduced in size by compressing, there is another category of slightly larger documents in whose case the time overhead of compressing and decompressing outweighs the slight advantage of marginally reduced transmission time from client to server or back.

      To safeguard against the case of several thousand documents stealing CPU time to barely discernable advantage, a threshold for minimum doument size to compress is set in the SDK, with a sensible default value of 32 bytes that you can modify in two ways:

      • MinSize — specifies the minimum size of the document to consider compression.

      • MinRatio — specifies the minimal compress ratio (compressed / original) acceptible, for the document to be sent compressed.

      Both of these settings are detailed in the API Reference.

      Configuring Compression

      As well as the changes to the document size threshold configuration, mentioned earlier, you can disable compression by setting Disabled = false

      This can be done in the ClusterOptions() added to gocb.Connect, and passed through the Connection String.