Migrating to SDK 3 API

concept

The 3.0 API breaks the existing 2.0 APIs in order to provide a number of improvements. Collections and Scopes are introduced. The Document class and structure has been completely removed from the API, and the returned value is now Result. Retry behavior is more proactive, and lazy bootstrapping moves all error handling to a single place. Individual behaviour changes across services are explained here.

In following Microsoft recommendations on .NET, the API has also been changed to surface many of the APIs as async using Tasks.

Fundamentals

Before this guide dives into the language-specific technical component of the migration, it is important to understand the high level changes first. As a migration guide, this document assumes you are familiar with the previous generation of the SDK and does not re-introduce SDK API 2 concepts. We recommend familiarizing yourself with the new SDK first by reading at least the getting started guide, and browsing through the other chapters a little.

Terminology

The concept of a Cluster and a Bucket remain the same, but a fundamental new layer is introduced into the API: Collections and their Scopes. Collections are logical data containers inside a Couchbase bucket that let you group similar data just like a Table does in a relational database — although documents inside a collection do not need to have the same structure. Scopes allow the grouping of collections into a namespace, which is very usfeul when you have multilpe tenants acessing the same bucket. Couchbase Server includes support for collections as a developer preview in version 6.5, and as a first class concept of the programming model from version 7.0.

Note that the SDKs include the feature from SDK 3.0, to allow easier migration.

In the previous SDK generation, particularly with the KeyValue API, the focus has been on the codified concept of a Document. Documents were read and written and had a certain structure, including the id/key, content, expiry (ttl), and so forth. While the server still operates on the logical concept of documents, we found that this model in practice didn’t work so well for client code in certain edge cases. As a result we have removed the Document class/structure completely from the API. The new API follows a clear scheme: each command takes required arguments explicitly, and an option block for all optional values. The returned value is always of type Result. This avoids method overloading bloat in certain languages, and has the added benefit of making it easy to grasp APIs evenly across services.

As an example here is a KeyValue document fetch:

IGetResult getResult = await collection.GetAsync("doc1");
var docContent = getResult.ContentAs<dynamic>();

Compared to the 2.x SDK, you will note that the result is now oriented around the result of the operation. Retrieval of the content of the document is accomplished through the ContentAs() method. Also, in most cases you do not need to check the result object returned as exceptions are thrown for most error conditions. Check the API reference for details.

Compare this to a SQL++ (formerly N1QL) query:

IQueryResult<dynamic> result = await cluster.QueryAsync<dynamic>("select 1 = 1", QueryOptions.Create().Timeout(TimeSpan.FromSeconds(3)));

Since documents also fundamentally handled the serialization aspects of content, two new concepts are introduced: the Serializer and the Transcoder. Out of the box the SDKs ship with a JSON serializer which handles the encoding and decoding of JSON. You’ll find the serializer exposes the options for methods like SQL++ queries and KeyValue subdocument operations,.

The KV API extends the concept of the serializer to the Transcoder. Since you can also store non-JSON data inside a document, the Transcoder allows the writing of binary data as well. It handles the object/entity encoding and decoding, and if it happens to deal with JSON makes uses of the configured Serializer internally. See the Serialization and Transcoding section below for details.

What to look out for

The SDKs are more proactive in retrying with certain errors and in certain situations, within the timeout budget given by the user — as an example, temporary failures or locked documents are now being retried by default — making it even easier to program against certain error cases. This behavior is customizable in a RetryStrategy, which can be overridden on a per operation basis for maximum flexibility if you need it.

Note, most of the bootstrap sequence is now lazy (happening behind the scenes). For example, opening a bucket is not raising an error anymore, but it will only show up once you perform an actual operation. The reason behind this is to spare the application developer the work of having to do error handling in more places than needed. A bucket can go down 2ms after you opened it, so you have to handle request failures anyway. By delaying the error into the operation result itself, there is only one place to do the error handling. There will still be situations why you want to check if the resource you are accessing is available before continuing the bootstrap; for this, we have the diagnostics and ping commands at each level which allow you to perform those checks eagerly.

Language Specifics

Now that you are familiar with the general theme of the migration, the next sections dive deep into the specifics. First, installation and configuration are covered, then we talk about exception handling, and then each service (i.e. Key/Value, Query,…) is covered separately.

Installation and Configuration

The .NET SDK 3.x is available for download from the same resources as the previous generation 2.0 SDK:

From NuGet (the most popular choice): Install-Package CouchbaseNetClient -Version 3.2.7
By downloading a zip file directly
(Not officially supported) By cloning and building the source code directly on github

Please see the Release Notes for up-to-date information.

Couchbase .NET SDK 3.2.7 build targets include .NET Standard 2.0, .NET Standard 2.1, .NET CoreApp 3.1, .NET 5 and .NET 6. This was chosen so .NET Full Framework 4.8 can be supported along with .NET Core 3.x and earlier and follows the suggested path for library authors as well as the latest .NET Versions. The goal is to use the latest and greatest .NET Core libraries available, but still allow fallback for developers still using .NET Full Framework as they progress towards .NET Standard compliance.

While the older target Frameworks are still targeted, Couchbase suggests that users target .NET 6 as it provides the latest, most performant .NET version.

Dependencies

There are few dependency changes from SDK 2.x. SDK 3.0 and later no longer has any dependencies on .NET Framework 4.5.2. It is based on .NET Standard 2.0 and .NET Standard 2.1. One dependency addition is DnsClient, which was previously in SDK 2.x through a transitive dependency via the Couchbase.Extensions.DnsDiscovery project.

Current dependencies (as of 3.2.7):

App.Metrics (>= 4.2.0)
App.Metrics.Abstractions (>= 4.2.0)
DnsClient (>= 1.3.2)
Microsoft.CSharp (>= 4.7.0)
Microsoft.Extensions.Logging.Abstractions (>= 3.1.21)
Microsoft.Extensions.ObjectPool (>= 6.0.0)
Newtonsoft.Json (>= 11.0.2)
System.Diagnostics.DiagnosticSource (>= 5.0.1)
System.IO.Pipelines (>= 5.0.1)
System.Linq.Async (>= 4.1.1)
System.Runtime.CompilerServices.Unsafe (>= 6.0.0)
System.Text.Json (>= 6.0.0)
System.Threading.Channels (>= 5.0.0)
System.Threading.Tasks.Dataflow (>= 5.0.0)

Note that if using the NuGet package these dependencies will all be handled for you by the NuGet Package Manager tool in Visual Studio or Visual Studio Code. See the Start Using document for information on adding it via NuGet.

Configuring the Environment

Configuration is essentially the same as SDK 2.x retaining capabilites with less tunable properties. Instead of using a ClientConfiguration object, you would use a ClusterOptions object. For example, to use a custom timeout for Key/Value (K/V) operations in SDK 2.x you would do something like this:

// SDK 2.0 custom k/v timeout
var config = new ClientConfiguration
{
    DefaultOperationLifespan = (uint)TimeSpan.FromMilliseconds(5).TotalMilliseconds
};

You can perform the same custom K/V timeout in SDK 3.0, however, you will use the ClusterOptions class:

// SDK 3.0 custom k/v timeout
var options = new ClusterOptions
{
    KvTimeout = TimeSpan.FromMilliseconds(5)
};

The configuration options are passed into one of the static Cluster.ConnectAsync(…) methods:

var cluster = await Cluster.ConnectAsync("couchbase://your-ip", options);

In order to free all resources associated with a cluster, simply call the .Dispose() method:

cluster.Dispose();

When creating a configuration, you may customize settings through the ClusterOptions or the connection string. See client configuration for further configuration details.

  // Will set the max http connections to 23
  var config = new ClusterOptions()
  {
      UserName = "user",
      Password = "pass",
      MaxHttpConnections =  23
  };

  config.WithConnectionString("couchbase://your-ip");

  cluster = await Cluster.ConnectAsync(config);

The above has this equivalent with a connection string

// Will set the max http connections to 23
var config = new ClusterOptions()
{
    Password = "pass",
    Username = "user"
};

config.WithConnectionString("couchbase://your-ip?max_kv_connections=23");

cluster = await Cluster.ConnectAsync(config);

The SDK offers a configuration API for customizing bootstrapping, timeouts, reliability and performance tuning. Configuration options have changed since the 2.x release. See the configuration section for specifics.

At the end of this guide you’ll find a reference that describes the SDK 2 environment options and their SDK 3 equivalents where applicable.

Authentication

Since SDK 2 supports Couchbase Server clusters older than 5.0, it had to support both Role-Based access control as well as bucket-level passwords. The minimum cluster version supported by SDK 3 is Server 5.0, which means that only RBAC is supported. This is why you can set the username and password when directly connecting:

var cluster = await Cluster.ConnectAsync("couchbase://your-ip", "user", "pass");

This is just a shorthand for:

var cluster = await Cluster.ConnectAsync("couchbase://your-ip", new ClusterOptions
{
    UserName = "user",
    Password = "pass"
});

Configuring TLS/SSL is done by using the "couchbases://" scheme in the connection string:

var cluster = await Cluster.ConnectAsync("couchbases://your-ip", new ClusterOptions
{
    UserName = "user",
    Password = "pass"
});

You may also use this approach to configure certificate-based authentication:

var cluster = await Cluster.ConnectAsync("couchbases://127.0.0.1", new ClusterOptions().
    WithX509CertificateFactory(CertificateFactory.GetCertificatesFromStore(
    new CertificateStoreSearchCriteria
    {
        FindValue = "value",
        X509FindType = X509FindType.FindBySubjectName,
        StoreLocation = StoreLocation.CurrentUser,
        StoreName = StoreName.CertificateAuthority
    })));

Note that we are using the scheme "couchbases://" as opposed to "couchbase://" this is an indication to the SDK to use TLS/SSL encryption on the wire. Note that you can also set the ClusterOptions.EnableTls flag as well to do this.

Please see the documentation on certificate-based authentication for detailed information on how to configure this properly.

Connection Lifecycle

From a high-level perspective, bootstrapping and shutdown is very similar to SDK 2. One notable difference is that the Collection is introduced and that the individual methods like bucket immediately return, and do not throw an exception. Compare SDK 2: the openBucket method would not work if it could not open the bucket.

The reason behind this change is that even if a bucket can be opened, a millisecond later it may not be available any more. All this state has been moved into the actual operation so there is only a single place where the error handling needs to take place. This simplifies error handling and retry logic for an application.

In SDK 2, you connected, opened a bucket, performed a KV op, and disconnected like this:

var cluster = new Cluster(new ClientConfiguration
{
    Servers = new List<Uri> { new Uri("http://localhost:8091") }
});

var authenticator = new PasswordAuthenticator("user", "pass");
cluster.Authenticate(authenticator);
var bucket = cluster.OpenBucket("travel-sample");

var result = bucket.Get<dynamic>("airline_10");

bucket.Dispose();
cluster.Dispose();

Here is the SDK 3 equivalent:

var cluster = await Cluster.ConnectAsync("127.0.0.1", "user", "pass");
var bucket = await cluster.BucketAsync("travel-sample");
var collection = bucket.DefaultCollection();

var getResult = await collection.GetAsync("airline_10");

cluster.Dispose();

Collections will be generally available with an upcoming Couchbase Server release, but the SDK already encodes it in its API to be future-proof. If you are using a Couchbase Server version which does not support Collections such as 6.0, always use the DefaultCollection() method to access the KV API; it will map to the full bucket.

You’ll notice that BucketAsync(String) returns immediately, even if the bucket resources are not completely opened. This means that the subsequent Get operation may be dispatched even before the connection is opened in the background. The SDK will handle this case transparently, and reschedule the operation until the bucket is opened properly. This also means that if a bucket could not be opened (say, because no server was reachable) the operation will time out. Please check the logs to see the cause of the timeout. In this example case, you’ll see network socket connection failures.

Also note you will now find Query, Search, and Analytics at the Cluster level. This is where they logically belong as they are Cluster level services as opposed to KV which is bucket specific. If you are using Couchbase Server 6.5 or later, you will be able to perform cluster-level queries even if no bucket is open. If you are using an earlier version of the cluster you must open at least one bucket, otherwise cluster-level queries will fail.

Async and Await by Default

SDK 2 followed a pattern of having a Method() and a MethodAsync() as was the common approach in most C# code at the time. Subsequently, it has become more common in contemporary C# code for all methods to return a Task from all asynchronous methods. This is now considered to be the best practice. Read more about it in MSDN’s post Async All the Way. With this change, the SDK follows the same idioms in the base class library’s HttpClient. The application then waits as appropriate on these calls with either the await keyword or by calling the .Wait() method, depending on how it fits into the rest of the applciation.

Serialization and Transcoding

In SDK 2 the API was oriented around an IDocument or using the IOperationRequest interface with additonal methods to get to metadata. As earlier mentioned, in SDK 3 many methods return 'result' objects with all metadata and content, where transcoding to either a C# POCO or to a dynamic object is done via method calls. While most of the time decoding to a dynamic object or a POCO is all that is necessary, these can, optionally, have custom deserializers.

In SDK 2 the main method to control transcoding was through providing different IDocument or IOperationRequest instances (which in turn had their own transcoder associated), such as the JsonDocument. This only worked for the KV APIs though — Query, Search, Views, and other services exposed their JSON rows/hits in different ways. All of this has been unified in SDK 3 under a single concept: serializers and transcoders.

By default, all KV APIs transcode to and from JSON either with dynamic objects or POCOs. In general this will suffice for the vast number of use cases, however, sometimes another transcoder will be needed.

If you want to write already-encoded JSON, you would pass in the RawJsonTranscoder in the options block:

var content = System.Text.Encoding.UTF8.GetBytes("{}");
var upsertTranscoded = await collection.UpsertAsync(
    "foo",
    content,
    new UpsertOptions().Transcoder(new RawJsonTranscoder())
);

Here is a mapping table from the SDK 2 Document<T> and/or IOperationResult<T> types to the new transcoder types:

Table 1. SDK 2.x Document vs. SDK 3.x Transcoder*
SDK 2	SDK 3
`object`	`JsonTranscoder` (default)
`string`	`JsonTranscoder` (default)
`array`	`JsonTranscoder` (default)
`number`	`JsonTranscoder` (default)
`byte[]`	`RawJsonTranscoder`
`byte[]`	`RawStringTranscoder`
`byte[]`	`RawBinaryTranscoder`

*Note that .NET SDK 2 had a DefaultTranscoder for storing JSON, a BinaryTranscoder for storing binary content and a BinaryToJsonTranscoder for handling legacy upgrades from very early server versions.

Serializers and transcoders can also be customized and overwritten on a per-operation basis, please see the appropriate documentation section for details.

The JSON Transcoders use a Serializer underneath. While a transcoder can handle many different storage types, the serializer is specialized for JSON encoding and decoding. On all JSON-only APIs (i.e. Sub-doc, Query, Search,…) you’ll only find a Serializer, not a Transcoder, in the operation options. Usually there is no need to override it unless you want to provide your own implementation (i.e. if you have your own POCO mapping json logic in place, and want to reuse it).

Exception Handling

While how to handle exceptions is unchanged from SDK 2, using a try/catch clause, the way common errors are exposed to the application layer by SDK3 is significantly different. In SDK2, exceptions are not thrown for most operations, only for terminal failures within the SDK. Instead an IResult implementation was returned which contained a Success, Status and Exception fields. In SDK3, this has changed and now all errors are thrown from the SDK (as well as terminal failures) and must be handled by the application code.

There have been changes made in the following areas:

Exception hierachy and naming.
Proactive retry where possible.

Exception hierachy

CouchbaseException is the root of all Couchbase-specific exceptions thrown by the SDK, serving as a base exception for many more detailed exceptions. Each CouchbaseException has an associated ErrorContext which is populated with as much information as possible and then dumped alongside the stack trace if an error happens.

Here is an example of the error context if a SQL++ query is performed with an invalid syntax (i.e. select 1= from):

try
{
    IQueryResult<dynamic> result = await cluster.QueryAsync<dynamic>("select 1");
}
catch (CouchbaseException ex) {
    Console.WriteLine(ex);
}

Besides the stacktrace and exception message, there is also additional context information returned by the exceptions which derive from CouchbaseException.

-----------------------Context Info---------------------------
{"Statement":"[{\"statement\":\"select 1 = \",\"timeout\":\"3000ms\",\"client_context_id\":\"e3003393-e54b-4f5b-b620-f91903556282\"}]","ClientContextId":"e3003393-e54b-4f5b-b620-f91903556282","Parameters":"{\"Named\":{},\"Raw\":{},\"Positional\":[]}","HttpStatus":400,"QueryStatus":6,"Errors":[{"msg":"syntax error - at end of input","Code":3000,"Name":null,"Severity":0,"Temp":false}],"Message":null}

The expectation is that the application catches the CouchbaseException and deals with it as an unexpected error (e.g. logging with subsequent bubbling up of the exception or failing). In addition to that, each method exposes exceptions that can be caught separately if needed. For example, a GetAsync() may throw a DocumentNotFoundException or a TimeoutException in addition to a more generic CouchbaseException. These exceptions extend CouchbaseException, but both the TimeoutException and the DocumentNotFoundException can be caught individually if specific logic should be executed to handle them.

try
{
    IQueryResult<dynamic> result = await cluster.QueryAsync<dynamic>("select 1");
}
catch(DocumentNotFoundException ex){
  //Handle the case where the document does not exist
}
catch (CouchbaseException ex) {
    Console.WriteLine(ex);
}

Proactive Retry

One reason why the APIs do not expose a long list of exceptions is that the SDK now retries as many operations as it can if it can do so safely. This depends on the type of operation (idempotent or not), in which state of processing it is (already dispatched or not), and what the actual response code is if it arrived already. As a result, many transient cases — such as locked documents, or temporary failure — are now retried by default and should less often impact applications. It also means, when migrating to the new SDK API, you may observe a longer period of time until an error is returned by default.

Operations are retried by default as described above with the default BestEffortRetryStrategy. Like in SDK 2 you can configure fail-fast retry strategies to not retry certain or all operations. The RetryStrategy interface has been extended heavily in SDK 3 — please see the error handling documentation.

When migrating your SDK 2 exception handling code to SDK 3, make sure to wrap every call with a catch for CouchbaseException (or let it bubble up immediately). You can likely remove your user-level retry code for temporary failures, backpressure exception, and so on. One notable exception from this is the CasMismatchException, which is still thrown since it requires more app-level code to handle (most likely identical to SDK 2).

Logging and Events

Configuring and consuming logs has not greatly changed.

The SDK is still compatible with multiple loggers and works well with the .NET Core Runtime abstraction interface in Microsoft.Extensions.Logging. The biggest impact you’ll see from it is that the log messages now look very structured and contain contextual information where possible.

The logger may be configured from the ClusterOptions.

using Microsoft.Extensions.Logging;
…
var loggerFactory = new LoggerFactory().AddConsole();

var config = new ClusterOptions()
{
    UserName = "user",
    Password = "pass",
    Logging = loggerFactory
};

config.WithConnectionString("couchbase://your-ip");

Couchbase recommends Serilog, however, any 3rd party logging library (Log4Net and others) will work as long as it is based off Microsoft.Extensions.Logging. Please see the 3rd party logging libraries documentation for details on its idiomatic logging configuration.

2022-02-18T13:46:48.1829565-08:00  [DBG] Waiting for 00:00:02.5000000 before polling. (c8639b24)
2022-02-18T13:46:50.0150580-08:00  [DBG] Setting TCP Keep-Alives using SocketOptions - enable keep-alives True, time 00:01:00, interval 00:00:01. (d66a37aa)

Please see the logging documentation for further information.

Migrating Services

The following section discusses each service in detail and covers specific bits that have not been covered by the more generic sections above.

Key Value

The Key/Value (KV) API is now located under the Collection interface, so even if you do not use collections, the DefaultCollection() needs to be opened in order to access it.

The following table describes the SDK 2 KV APIs and where they are now located in SDK 3:

Table 2. SDK 2.x KV API vs. SDK 3.x KV API
SDK 2	SDK 3
`Bucket.Upsert` and `Bucket.UpsertAsync`	`Collection.UpsertAsync`
`Bucket.Get` and `Bucket.GetAsync`	`Collection.GetAsync`
`Bucket.Exists` and `Bucket.ExistsAsync`	`Collection.ExistsAsync`
`Bucket.GetFromReplica` and `Bucket.GetFromReplicaAsync`	`Collection.GetAnyReplicaAsync` and `Collection.GetAllReplicasAsync`
`Bucket.GetAndLock` and `Bucket.GetAndLockAsync`	`Collection.GetAndLockAsync`
`Bucket.GetAndTouch` and `Bucket.GetAndTouchAsync`	`Collection.GetAndTouchAsync`
`Bucket.Insert` and `Bucket.InsertAsync`	`Collection.InsertAsync`
`Bucket.Upsert` and `Bucket.UpsertAsync`	`Collection.UpsertAsync`
`Bucket.Replace` and `Bucket.ReplaceAsync`	`Collection.ReplaceAsync`
`Bucket.Remove` and `Bucket.RemoveAsync`	`Collection.RemoveAsync`
`Bucket.Unlock` and `Bucket.UnlockAsync`	`Collection.UnlockAsync`
`Bucket.Touch` and `Bucket.TouchAsync`	`Collection.TouchAsync`
`Bucket.LookupIn`	`Collection.LookupInAsync`
`Bucket.MutateIn`	`Collection.MutateInAsync`
`Bucket.Increment`, `Bucket.IncrementAsync`
`Bucket.Decrement` and `Bucket.DecrementAsync`	`Collection.Binary.IncrementAsync` and `Collection.Binary.DecrementAsync`
`Bucket.Append` and `Bucket.AppendAsync`	`Collection.Binary.IncrementAsync.AppendAsync`
`Bucket.Prepend` and `Bucket.PrependAsync`	`Collection.Binary.IncrementAsync.PrependAsync`

In addition, the datastructure APIs have been renamed and moved:

Table 3. Datastructure API Changes
SDK 2	SDK 3
`Bucket.mapAdd`	`Collection.Dictionary<T>`
`Bucket.mapGet`	`Collection.Dictionary<T>`
`Bucket.mapRemove`	`Collection.Dictionary<T>`
`Bucket.mapSize`	`Collection.Dictionary<T>`
`Bucket.listGet`	`Collection.List<T>`
`Bucket.listAppend`	`Collection.List<T>`
`Bucket.listRemove`	`Collection.List<T>`
`Bucket.listPrepend`	`Collection.List<T>`
`Bucket.listSet`	`Collection.List<T>`
`Bucket.listSize`	`Collection.List<T>`
`Bucket.setAdd`	`Collection.Set<T>`
`Bucket.setContains`	`Collection.Set<T>`
`Bucket.setRemove`	`Collection.Set<T>`
`Bucket.setSize`	`Collection.Set<T>`
`Bucket.queuePush`	`Collection.Queue<T>`
`Bucket.queuePop`	`Collection.Queue<T>`

There are two important API changes:

On the request side, overloads have been reduced and moved under a Options block
On the response side, the return types have been unified.

The signatures now look very similar. The concept of the IDocument as a type is gone in SDK 3 and instead you need to pass in the properties explicitly. This makes it very clear what is returned, especially on the response side.

Thus, the GetAsync method does not return a IDocumentResult or a IOperationResult but a IGetResult instead, and the UpsertAsync does not return a IDocumentResult or a IOperationResult but a IMutationResult. Each of those results only contain the field that the specific method can actually return, making it impossible to accidentally try to access the expiry on the IDocumentResult after a mutation, for example.

Instead of having many overloads, all optional params are now part of the Option block. All required params are still part of the method signature, making it clear what is required and what is not (or has default values applied if not overridden).

The timeout can be overridden on every operation and now takes a TimeSpan. Compare SDK 2 and SDK 3 custom timeout setting:

// SDK 2 custom timeout
IOperationResult getResult = bucket.Get<dynamic>("mydoc-id", TimeSpan.FromMilliseconds(2250));

IGetResult getaResult = await collection.GetAsync("mydoc-id", options => options.Timeout(TimeSpan.FromMilliseconds(2250)));

In SDK 2, the GetFromReplica method was available for replica reads, this has been split into two methods that simplify usage significantly. There is now a GetAllReplicasAsync method and a GetAnyReplicaAsync method.

GetAllReplicasAsync asks the active node and all available replicas and returns the results as a stream.
GetAnyReplicaAsync uses GetAllReplicasAsync, and returns the first result obtained.

Unless you want to build some kind of consensus between the different replica responses, we recommend GetAnyReplicaAsync for a fallback to a regular GetAsync when the active node times out.

Operations which cannot be performed on JSON documents have been moved to the IBinaryCollection, accessible through ICollection.Binary(). These operations include AppendAsync, PrependAsync, IncrementAsync, and DecrementAsync (previously called counter in SDK 2). These operations should only be used against non-json data. Similar functionality is available through MutateIn on JSON documents.

Query

SQL++ querying is now available at the Cluster level instead of the bucket level, because you can also write SQL++ queries that span multiple buckets. Querying is now async by default as discussed earlier. One related change is that query results come back in an async stream by default as well. To convert results to IEnumerable to iterate like you would in the 2.x SDK, you can call .ToEnumerable() on the results.

Compare a simple SQL++ query from SDK 2 with its SDK 3 equivalent:

// SDK 2 simple query
var query = new QueryRequest("SELECT * FROM `travel-sample` LIMIT 10");
foreach (var row in bucket.Query<dynamic>(query))
{
    Console.WriteLine(JsonConvert.SerializeObject(row));
}

try {
    var queryResult = await cluster.QueryAsync<dynamic>("SELECT `travel-sample`.* FROM `travel-sample` LIMIT 10").ConfigureAwait(false);

    await foreach (var o in queryResult.ConfigureAwait(false))
    {
        Console.WriteLine(Newtonsoft.Json.JsonConvert.SerializeObject(o, Newtonsoft.Json.Formatting.None));
    }
}
catch (CouchbaseException) {
    // SDK 3 throws exceptions where possible
}

An example using named parameters:

try {
    var queryResult = await cluster.QueryAsync<dynamic>("SELECT `travel-sample`.* FROM `travel-sample` LIMIT 10 WHERE type=$type",
        options =>
        {
            options.Parameter("type", "airline");
        }).ConfigureAwait(false);

    await foreach (var o in queryResult.ConfigureAwait(false))
    {
        Console.WriteLine(Newtonsoft.Json.JsonConvert.SerializeObject(o, Newtonsoft.Json.Formatting.None));
    }
}
catch (CouchbaseException) {
    // SDK 3 throws exceptions where possible
}

Another similar query using positional parameters:

try {
    var queryResult = await cluster.QueryAsync<dynamic>("SELECT `travel-sample`.* FROM `travel-sample` LIMIT 10 WHERE type=$1",
        options =>
        {
            options.Parameter("airline");
        }).ConfigureAwait(false);

    await foreach (var o in queryResult.ConfigureAwait(false))
    {
        Console.WriteLine(Newtonsoft.Json.JsonConvert.SerializeObject(o, Newtonsoft.Json.Formatting.None));
    }
}
catch (CouchbaseException) {
    // SDK 3 throws exceptions where possible
}

If you want to use prepared statements, the AdHoc() method is still available on the QueryOptions, alongside every other option that used to be exposed on the SDK 2 Query options.

Much of the non-row metadata has been moved into a specific QueryMetaData section:

Table 4. Query Metadata Changes
SDK 2	SDK 3
`IQueryResult.Signature`	`QueryResult.QueryMetaData.Signature`
`IQueryResult.Metrics`	`QueryResult.QueryMetaData.Metrics`
`IQueryResult.Profile`	`QueryResult.QueryMetaData.Profile`
`IQueryResult.Success`	removed
`IQueryResult.Status`	`QueryResult.QueryMetaData.status` containing any warnings
`IQueryResult.Errors`	throws an Exception on `QueryResult`
`IQueryResult.RequestId`	`QueryResult.QueryMetaData.RequestId`
`IQueryResult.ClientContextId`	`QueryResult.QueryMetaData.ClientContextId`

It is no longer necessary to check for a specific error in the stream: if an error happened during processing it will throw an exception at the top level of the query. The stream with terminate with an error as soon as one is received.

In SDK 2 you had to manually check for errors, otherwise you would get an empty row collection:

var queryResult = await bucket.QueryAsync("select 1=");
if (queryResult.Errors().Any()) {
  // errors contain [{"msg":"syntax error - at end of input","code":3000}]
}

In SDK 3 the top level query method will throw an exception:

var results = await cluster.QueryAsync<dynamic>("SELECT * FROM DOESNOTEXIST");

Throws the following exception:

Couchbase.Core.Exceptions.IndexFailureException: 'Keyspace not found in CB datastore: default:DOESNOTEXIST - cause: No bucket named DOESNOTEXIST [12003]'

Not only does it throw a CouchbaseException, it also tries to map it to a specific exception type and include extensive contextual information for a better troubleshooting experience.

Analytics

Analytics querying, like SQL++, is also moved to the Cluster level: it is now accessible through the Cluster.AnalyticsQueryAsync method. As with the Query service, parameters for the Analytics queries have moved into the AnalyticsOptions:

var analyticsResult = await cluster.AnalyticsQueryAsync<TestRequest>("SELECT \"hello\" as greeting;").ConfigureAwait(false);
var result = await analyticsResult.ToListAsync().First().Greeting;

// SDK 3 named parameters for analytics
var result  var result = await cluster.AnalyticsQueryAsync(
  "SELECT * FROM dataset WHERE type = $type",
  options=>options.Parameter("type", "airline"))
);

Also, errors will now be thrown as top level exceptions and it is no longer necessary to explicitly check for errors:

// SDK 2 error check
var result  var result = await cluster.AnalyticsQueryAsync("select * from foo"));
if (analyticsQueryResult.Errors.Any()) {
  // errors contain [{"msg":"Cannot find dataset foo in dataverse Default nor an alias with name foo! (in line 1, at column 15)","code":24045}]
}

// SDK 3 top level exception
Analytics query failed: 24045

Search

The Search API has changed a bit in SDK 3 so that it aligns with the other query APIs. The type of queries have stayed the same, but all optional parameters moved into SearchOptions. Also, similar to the other query APIs, it is now available at the Cluster level.

Here is a SDK 2 Search query with some options, and its SDK 3 equivalent:

//  SDK 2 search query
var searchResult = bucket.Query(new SearchQuery
      {
          Index = "indexname",
          Query = new QueryStringQuery("airports"),
      }.Limit(5).Fields("a", "b", "c").Timeout(TimeSpan.FromSeconds(2)));
foreach(var hit in results.Hits)
{
  //
}

// SDK 3 search query
var result  = await cluster.SearchQueryAsync("indexname", new QueryStringQuery("airports"),
          options =>
      {
          options.Timeout(TimeSpan.FromSeconds(2));
          options.Limit(5);
          options.Fields("a", "b", "c");
      });

foreach (var hit in results.Hits)
{
    //
}

Error handling for streaming is handled differently. While fatal errors will still raise top-level exceptions, any errors that happend during streaming (for example if one node is down, and only partial results are returned) they will not terminate the result. The reasoning behind this is that usually with search results, having partial results is better than none.

Here is a top level exception, for the index does not exist:

Exception of type 'Couchbase.Core.Exceptions.IndexNotFoundException' was thrown.

If you want to be absolutely sure that you didn’t get only partial data, you can check the error map:

var result  = await cluster.SearchQueryAsync("indexname", new QueryStringQuery("airports"),
    options =>
{
    options.Timeout(TimeSpan.FromSeconds(2));
    options.Limit(5);
    options.Fields("a", "b", "c");
});
if (!result.MetaData.Errors.Any()) {
  // no errors present, so full data got returned
}

Views

Views have stayed at the Bucket level, because it does not have the concept of collections and is scoped at the bucket level on the server as well. The API has stayed mostly the same, the most important change is that staleness is unified under the ViewConsistency enum.

Table 5. View Staleness Mapping
SDK 2	SDK 3
`Stale.TRUE`	`ViewScanConsistency.NotBounded`
`Stale.FALSE`	`ViewScanConsistency.RequestPlus`
`Stale.UPDATE_AFTER`	`ViewScanConsistency.UpdateAfter`

Compare this SDK 2 view query with its SDK 3 equivalent:

// SDK 2 view query
var query = bucket.CreateQuery("design", "view")
                    .Limit(1)
                    .Skip(2)
                    .ConnectionTimeout(10);

var result bucket.Query<dynamic>(query);
foreach (var row in result.Rows)
{
}

var result = await bucket.ViewQueryAsync<string[], object>("design", "view", options =>
{
    options.Limit(1);
    options.Skip(2);
    options.Timeout(TimeSpan.FromSeconds(10));
}).ConfigureAwait(false);

await foreach (var row in result)
{
  //
}

Exceptions are exclusively raised at the top level: for example, if the design document is not found:

Couchbase.Core.Exceptions.View.ViewNotFoundException: 'http://localhost:8092/default/_design/design/_view/view?stale=false&descending=true&on_error=continue&limit=1&skip=2'

Management APIs

In SDK 2, the management APIs were centralized in the ClusterManager at the cluster level and the BucketManager at the bucket level. Since SDK 3 provides more management APIs, they have been split up in their respective domains. So for example when in SDK 2 you needed to remove a bucket you would call ClusterManager.removeBucket you will now find it under BucketManager.dropBucket. Also, creating a SQL++ index now lives in the QueryIndexManager, which is accessible through the Cluster.

The following table provides a mapping from the SDK 2 management APIs to those of SDK 3:

Table 6. SDK 2.x vs SDK 3.x ClusterManager
SDK 2	SDK 3
`ClusterManager.ClusterInfo`	removed
`ClusterManager.ListBuckets`	`BucketManager.GetAllBucketsAsync`
DNE	`BucketManager.GetBucketAsync`
`ClusterManager.InsertBucket`	`BucketManager.CreateBucketAsync`
`ClusterManager.UpdateBucket`	`BucketManager.UpdateBucketAsync`
`ClusterManager.RemoveBucket`	`BucketManager.DropBucketAsync`
`ClusterManager.UpsertUser`	`UserManager.UpsertUserAsync`
DNE	`UserManager.DropUserAsync`
DNE	`UserManager.GetAllUsersAsync`
DNE	`UserManager.GetUserAsync`

*DNE: Does not exist

Table 7. SDK 2.x vs SDK 3.x BucketManager
SDK 2	SDK 3
`BucketManager.Flush`	`BucketManager.FlushBucketAsync`
`BucketManager.GetDesignDocuments`	`ViewIndexManager.GetAllDesignDocumentsAsync`
`BucketManager.GetDesignDocument`	`ViewIndexManager.GetDesignDocumentAsync`
`BucketManager.InsertDesignDocument`	`ViewIndexManager.UpsertDesignDocumentAsync`
`BucketManager.UpsertDesignDocument`	`ViewIndexManager.UpsertDesignDocumentAsync`
`BucketManager.RemoveDesignDocument`	`ViewIndexManager.DropDesignDocumentAsync`
`BucketManager.PublishDesignDocument`	`ViewIndexManager.PublishDesignDocumentAsync`
`BucketManager.ListN1qlIndexes`	`QueryIndexManager.GetAllIndexesAsync`
`BucketManager.CreateN1qlIndex`	`QueryIndexManager.CreateIndexAsync`
`BucketManager.CreateN1qlPrimaryIndex`	`QueryIndexManager.CreatePrimaryIndexAsync`
`BucketManager.DropN1qlIndex`	`QueryIndexManager.DropIndexAsync`
`BucketManager.DropN1qlPrimaryIndex`	`QueryIndexManager.DropPrimaryIndexAsync`
`BucketManager.BuildN1qlDeferredIndexes`	`QueryIndexManager.BuildDeferredIndexesAsync`
`BucketManager.WatchN1qlIndexes`	`QueryIndexManager.WatchIndexesAsync`

Note: SDK2 has both synchronous and asynchronous methods; the asynchronous methods have been omitted for brevity.

Configuration Options Reference

The following table provides commonly used configuration options in SDK 2 and where they can be now applied in SDK 3. Note that some options have been removed, and others have different ways to configure them.

Table 8. SDK 2.x vs SDK 3.x Environment Configs
SDK 2	SDK 3
`QueryRequestTimeout`	`QueryTimeout`
`AnalyticsRequestTimeout`	`AnalyticsTimeout`
`EnableQueryTiming`	`removed`
`UseSsl`	`EnableTls`
`SslPort`	`removed`
`ApiPort`	`removed`
`DirectPort`	`removed`
`MgmtPort`	`BootstrapHttpPort`
`HttpsMgmtPort`	`BootstrapHttpPortTls`
`HttpsApiPort`	`removed`
`ObserveInterval`	`removed`
`ObserveTimeout`	`KvDurableTimeout`
`MaxViewRetries`	`removed`
`ViewHardTimeout`	`removed`
`ConfigPollInterval`	`ConfigPollInterval`
`ConfigPollCheckFloor`	`ConfigPollFloorInterval`
`ConfigPollEnabled`	`EnableConfigPolling`
`ViewRequestTimeout`	`ViewTimeout`
`SearchRequestTimeout`	`SearchTimeout`
`VBucketRetrySleepTime`	`removed`
`DefaultConnectionLimit`	`MaxKvConnections`
`Expect100Continue`	`EnableExpect100Continue`
`MaxServicePointIdleTime`	`IdleHttpConnectionTimeout`
`EnableOperationTiming`	`removed`
`BufferSize`	`removed`
`DefaultOperationLifespan`	`removed`
`QueryFailedThreshold`	`removed`
`EnableTcpKeepAlives`	`EnableTcpKeepAlives`
`TcpKeepAliveTime`	`TcpKeepAliveTime`
`TcpKeepAliveInterval`	`TcpKeepAliveInterval`
`NodeAvailableCheckInterval`	`removed`
`IOErrorCheckInterval`	`removed`
`IOErrorThreshold`	`removed`
`UseConnectionPooling`	`removed`
`EnableDeadServiceUriPing`	`removed`
`ForceSaslPlain`	`removed`
`OperationTracingEnabled`	`TracingOptions.Enabled`
`OperationTracingServerDurationEnabled`	`ThresholdOptions.Enabled`
`OrphanedResponseLoggingEnabled`	`OrphanOptions.Enabled`
`EnableCertificateRevocation`	`EnableCertificateRevocation`
`EnableCertificateAuthentication`	`EnableCertificateAuthentication`
`NetworkType`	`NetworkResolution`
`IgnoreRemoteCertificateNameMismatch`	`KvIgnoreRemoteCertificateNameMismatch`
`KvServerCertificateValidationCallback`	`KvCertificateCallbackValidation`
`PoolConfiguration`	`removed`
`CertificateFactory`	`X509CertificateFactory`
`Serializer`	`Serializer`

Comparing Older Documentation

You may want to visit documentation for older versions of the SDK, to help to understand application code that you are migrating. Versions that have reached end of life can be found in the archive. In the release notes pages of these older docs, you will also find links to the API reference for each no-longer-supported release.