Migrating to SDK 3 API

The concept of a Cluster and a Bucket remain the same, but a fundamental new layer is introduced into the API: Collections and their Scopes. Collections are logical data containers inside a Couchbase bucket that let you group similar data just like a Table does in a relational database — although documents inside a collection do not need to have the same structure. Scopes allow the grouping of collections into a namespace, which is very usfeul when you have multilpe tenants acessing the same bucket. Couchbase Server includes support for collections as a developer preview in version 6.5, and as a first class concept of the programming model from version 7.0.

Note that the SDKs include the feature from SDK 3.0, to allow easier migration.

In the previous SDK generation, particularly with the KeyValue API, the focus has been on the codified concept of a Document. Documents were read and written and had a certain structure, including the id/key, content, expiry (ttl), and so forth. While the server still operates on the logical concept of documents, we found that this model in practice didn’t work so well for client code in certain edge cases. As a result we have removed the Document class/structure completely from the API. The new API follows a clear scheme: each command takes required arguments explicitly, and an option block for all optional values. The returned value is always of type Result. This avoids method overloading bloat in certain languages, and has the added benefit of making it easy to grasp APIs evenly across services.

As an example here is a KeyValue document fetch:

Compare this to a SQL++ (formerly N1QL) query:

Since documents also fundamentally handled the serialization aspects of content, two new concepts are introduced: the Serializer and the Transcoder. Out of the box the SDKs ship with a JSON serializer which handles the encoding and decoding of JSON. You’ll find the serializer exposes the options for methods like SQL++ queries and KeyValue subdocument operations,.

The KV API extends the concept of the serializer to the Transcoder. Since you can also store non-JSON data inside a document, the Transcoder allows the writing of binary data as well. It handles the object/entity encoding and decoding, and if it happens to deal with JSON makes uses of the configured Serializer internally. See the Serialization and Transcoding section below for details.

The SDKs are more proactive in retrying with certain errors and in certain situations, within the timeout budget given by the user — as an example, temporary failures or locked documents are now being retried by default — making it even easier to program against certain error cases. This behavior is customizable in a RetryStrategy, which can be overridden on a per operation basis for maximum flexibility if you need it.

Note, most of the bootstrap sequence is now lazy (happening behind the scenes). For example, opening a bucket is not raising an error anymore, but it will only show up once you perform an actual operation. The reason behind this is to spare the application developer the work of having to do error handling in more places than needed. A bucket can go down 2ms after you opened it, so you have to handle request failures anyway. By delaying the error into the operation result itself, there is only one place to do the error handling. There will still be situations why you want to check if the resource you are accessing is available before continuing the bootstrap; for this, we have the diagnostics and ping commands at each level which allow you to perform those checks eagerly.

The fundamental semantics of the CouchbaseEnvironment from SDK 2 remain the same, although somewhat more flexible, with its configuration cleaned up to be more easily discoverable. The CouchbaseEnvironment has been renamed to ClusterEnvironment and can be customized through a builder:

Similar to SDK 2, if you create your own ClusterEnvironment the SDK will not shut it down for you — you need to do this manually at the end of the program lifetime:

If a default environment is used, the default settings can still be customized through either the connection string or system properties. The SDK has elaborate reflection logic in place to parse "flat" string values and apply them to the builder, which means that you can now configure more properties than in SDK 2. Note that the property paths have changed.

The paths for each config start with the pattern com.couchbase.env. followed by either a toplevel or nested config. IoConfig is under the io path, similar to TimeoutConfig under timeout. So, if you want to modify a KV timeout you can use the com.couchbase.env.timeout.kvTimeout path. See the configuration section for full specifics.

At the end of this guide you’ll find a reference that describes the SDK 2 environment options and their SDK 3 equivalents where applicable.

Since SDK 2 supports Couchbase Server clusters older than 5.0, it had to support both Role-Based access control as well as bucket-level passwords. The minimum cluster version supported by SDK 3 is Server 5.0, which means that only RBAC is supported. This is why you can set the username and password when directly connecting:

This is just a shorthand for:

The reason why you can pass in a specific authenticator is that you can also use the same approach to configure certificate-based authentication:

Please see the documentation on certificate-based authentication for detailed information on how to configure this properly.

The exception hierachy is now flat and unified under a CouchbaseException. Each CouchbaseException has an associated ErrorContext which is populated with as much information as possible and then dumped alongside the stack trace if an error happens.

Here is an example of the error context if a SQL++ query is performed with an invalid syntax (i.e. select 1= from):

The expectation is that the application catches the CouchbaseException and deals with it as an unexpected error (e.g. logging with subsequent bubbling up of the exception or failing). In addition to that, each method exposes exceptions that can be caught separately if needed. As an example, consider the javadoc for the Collection.get API:

These exceptions extend CouchbaseException, but both the TimeoutException and the DocumentNotFoundException can be caught individually if specific logic should be executed to handle them.

One reason why the APIs do not expose a long list of exceptions is that the SDK now retries as many operations as it can if it can do so safely. This depends on the type of operation (idempotent or not), in which state of processing it is (already dispatched or not), and what the actual response code is if it arrived already. As a result, many transient cases — such as locked documents, or temporary failure — are now retried by default and should less often impact applications. It also means, when migrating to the new SDK API, you may observe a longer period of time until an error is returned by default.

When migrating your SDK 2 exception handling code to SDK 3, make sure to wrap every call with a catch for CouchbaseException (or let it bubble up immediately). You can likely remove your user-level retry code for temporary failures, backpressure exception, and so on. One notable exception from this is the CasMismatchException, which is still thrown since it requires more app-level code to handle (most likely identical to SDK 2).

The Key/Value (KV) API is now located under the Collection interface, so even if you do not use CollectionsExample, the defaultCollection() needs to be opened in order to access it.

The following table describes the SDK 2 KV APIs and where they are now located in SDK 3:

In addition, the datastructure APIs have been renamed and moved:

There are two important API changes:

The signatures now look very similar. The concept of the Document as a type is gone in SDK 3 and instead you need to pass in the properties explicitly. This makes it very clear what is returned, especially on the response side.

Thus, the get method does not return a Document but a GetResult instead, and the upsert does not return a Document but a MutationResult. Each of those results only contains the field that the specific method can actually return, making it impossible to accidentally try to access the expiry on the Document after a mutation, for example.

Instead of having many overloads, all optional params are now part of the Option block. All required params are still part of the method signature, making it clear what is required and what is not (or has default values applied if not overridden).

The timeout can be overridden on every operation and now takes a Duration from java 8. Compare SDK 2 and SDK 3 custom timeout setting:

In SDK 2, the getFromReplica method had a ReplicaMode argument which allowed you to customize its behavior on how many replicas should be reached. We have identified this as a potential source of confusion and as a result split it up in two methods that simplify usage significantly. There is now a getAllReplicas method and a getAnyReplica method.

Unless you want to build some kind of consensus between the different replica responses, we recommend getAnyReplica for a fallback to a regular get when the active node times out.

SQL++ querying is now available at the Cluster level instead of the bucket level, because you can also write SQL++ queries that span multiple buckets. Compare a simple SQL++ query from SDK 2 with its SDK 3 equivalent:

Note that there is no N1qlQuery.simple any more — parameter option have been moved to the queryOptions() for consistency reasons. The following shows how to do named and positional parameters in SDK 2, and their SDK 3 counterparts:

If you want to use prepared statements, the adhoc() method is still available on the QueryOptions, alongside every other option that used to be exposed on the SDK 2 Query options.

Much of the non-row metadata has been moved into a specific QueryMetaData section:

It is no longer necessary to check for a specific error in the stream: if an error happened during processing it will throw an exception at the top level of the query. The reactive streaming API will terminate the rows' Flux with an exception as well as soon as it is discovered. This makes error handling much easier in both the blocking and non-blocking cases.

While in SDK 2 you had to manually check for errors (otherwise you’d get an empty row collection):

In SDK 3 the top level query method will throw an exception:

Not only does it throw a CouchbaseException, it also tries to map it to a specific exception type and include extensive contextual information for a better troubleshooting experience.

Analytics querying, like SQL++, is also moved to the Cluster level: it is now accessible through the Cluster.analyticsQuery method. As with the Query service, parameters for the Analytics queries have moved into the AnalyticsOptions:

Also, errors will now be thrown as top level exceptions and it is no longer necessary to explicitly check for errors:

The Search API has changed a bit in SDK 3 so that it aligns with the other query APIs. The type of queries have stayed the same, but all optional parameters moved into SearchOptions. Also, similar to the other query APIs, it is now available at the Cluster level.

Here is a SDK 2 Search query with some options, and its SDK 3 equivalent:

Error handling for streaming is handled differently. While fatal errors will still raise top-level exceptions, any errors that happend during streaming (for example if one node is down, and only partial results are returned) they will not terminate the result. The reasoning behind this is that usually with search results, having partial results is better than none.

Here is a top level exception, for the index does not exist:

If you want to be absolutely sure that you didn’t get only partial data, you can check the error map:

Views have stayed at the Bucket level, because it does not have the concept of CollectionsExample and is scoped at the bucket level on the server as well. The API has stayed mostly the same, the most important change is that staleness is unified under the ViewConsistency enum.

Compare this SDK 2 view query with its SDK 3 equivalent:

Exceptions are exclusively raised at the top level: for example, if the design document is not found:

In SDK 2, the management APIs were centralized in the ClusterManager at the cluster level and the BucketManager at the bucket level. Since SDK 3 provides more management APIs, they have been split up in their respective domains. So for example when in SDK 2 you needed to remove a bucket you would call ClusterManager.removeBucket you will now find it under BucketManager.dropBucket. Also, creating a SQL++ index now lives in the QueryIndexManager, which is accessible through the Cluster.

The following table provides a mapping from the SDK 2 management APIs to those of SDK 3: