Compression

In response to increasing volumes of data being sent over the wire, Couchbase Data Platform now provides data compression between the SDK and Couchbase Server.

Overview

Documents may already be stored compressed with Snappy. Now documents may be passed from client applications to the server in compressed form, saving around 40% in bandwidth, and also transmission time. These operations take place automatically, after HELLO negotiation with the server, and do not require any changes on the client side, beyond updating to a recent SDK version with compression support (see release notes).

Limits

The document must be below 20MB in size in both compressed and uncompressed form. Compression is only available in the Enterprise Edition of Couchbase Data Platform.

Operating Modes

The three modes in which compression can be used, “off”, “passive” and “active”, are per bucket configuration settings on the cluster. Depending on how it is set, the HELLO negotiation will fail or succeed. The HELLO flag for compression has the value 0x0a and is defined as:

PROTOCOL_BINARY_FEATURE_SNAPPY = 0x0a
Table 1. Compression Operating Modes
OFF PASSIVE ACTIVE

Sending SNAPPY feature with HELLO

ignore

acknowledge

acknowledge

Sending compressed data when SNAPPY feature enabled

-

accept

accept

Sending compressed data when SNAPPY feature disabled

reject as invalid

reject as invalid

reject as invalid

Receiving data which were previously compressed on the server

server inflates and sends plain data

server sends compressed data

server sends compressed data

Receiving data which were not previously compressed on the server

server sends plain data

server sends plain data

server might send compressed data (the compression is done in the background on the server)

Acceptance Guarantee

The server may change compression settings for the bucket at any time, but it is guaranteed that once the socket negotiates compression through HELLO, the server will never reject compressed data, even if the bucket setting has been changed.

Minimum Size

While the tiniest of documents will not be reduced in size by compressing, there is another category of slightly larger documents in whose case the time overhead of compressing and decompressing outweighs the slight advantage of marginally reduced transmission time from client to server or back.

To safeguard against the case of several thousand documents stealing CPU time to barely discernable advantage, a threshold for minimum doument size to compress is set in the SDK, with a sensible default value - that value can be seen for your chosen SDK in its API documentation (normally 32 bytes), and you can override this to disable compression:

  • For Java, to disable compression from client to server, set compressionMinSize to max int (2147483647). Alternatively, to disable compression in both directions, set the Java system property com.couchbase.snappyEnabled to False.

  • For .NET, Snappy compression is not currently supported.

  • For SDKs using LCB’s connstr (Node.js, PHP, Python), override the default with "compression=off" in your connection string. Additionally, for Python you can set the couchbase.bucket.Bucket.compression property to COMPRESS_NONE.

  • For Go, can also be disabled by using compression=off in the connection string.