Migrating from SDK2 to SDK3 API

The 3.0 API breaks the existing 2.0 APIs in order to provide a number of improvements. Collections and Scopes are introduced. The Document class and structure has been completely removed from the API, and the returned value is now Result. Retry behaviour is more proactive, and lazy bootstrapping moves all error handling to a single place. Individual behaviour changes across services are explained here.

Fundamentals

The Couchbase SDK team takes semantic versioning seriously, which means that API should not be broken in incompatible ways while staying on a certain major release. This has the benefit that most of the time upgrading the SDK should not cause much trouble, even when switching between minor versions (not just bugfix releases). The downside though is that significant improvements to the APIs are very often not possible, save as pure additions — which eventually lead to overloaded methods.

To support new server releases and prepare the SDK for years to come, we have decided to increase the major version of each SDK and as a result take the opportunity to break APIs where we had to. As a result, migration from the previous major version to the new major version will take some time and effort — an effort to be counterbalanced by improvements to coding time, through the simpler API, and performance. The new API is built on years of hands-on experience with the current SDK as well as with a focus on simplicity, correctness, and performance.

Before this guide dives into the language-specific technical component of the migration, it is important to understand the high level changes first. As a migration guide, this document assumes you are familiar with the previous generation of the SDK and does not re-introducing SDK 2.0 concepts. We recommend familiarizing yourself with the new SDK first by reading at least the getting started guide, and browsing through the other chapters a little.

Terminology

The concept of a Cluster and a Bucket remain the same, but a fundamental new layer is introduced into the API: Collections and their Scopes. Collections are logical data containers inside a Couchbase bucket that let you group similar data just like a Table does in a relational database — although documents inside a collection do not need to have the same structure. Scopes allow the grouping of collections into a namespace, which is very usfeul when you have multilpe tenants acessing the same bucket. Couchbase Server is including support for collections as a developer preview in version 6.5 — in a future release, it is hoped that collections will become a first class concept of the programming model. To prepare for this, the SDKs include the feature from SDK 3.0.

In the previous SDK generation, particularly with the KeyValue API, the focus has been on the codified concept of a Document. Documents were read and written and had a certain structure, including the id/key, content, expiry (ttl), and so forth. While the server still operates on the logical concept of documents, we found that this model in practice didn’t work so well for client code in certain edge cases. As a result we have removed the Document class/structure completely from the API. The new API follows a clear scheme: each command takes required arguments explicitly, and an option block for all optional values. The returned value is always of type Result. This avoids method overloading bloat in certain languages, and has the added benefit of making it easy to grasp APIs evenly across services.

As an example here is a KeyValue document fetch:

$getResult = $collection->get("key", (new GetOptionsl())->timeout(3000000));

Compare this to a N1QL query:

$queryResult = $cluster->query("select 1=1", (new QueryOptions())->timeout(3000000));

Since documents also fundamentally handled the serialization aspects of content, two new concepts are introduced: the Serializer and the Transcoder. Out of the box the SDKs ship with a JSON serializer which handles the encoding and decoding of JSON. You’ll find the serializer exposes the options for methods like N1QL queries and KeyValue subdocument operations,.

The KV API extends the concept of the serializer to the Transcoder. Since you can also store non-JSON data inside a document, the Transcoder allows the writing of binary data as well. It handles the object/entity encoding and decoding, and if it happens to deal with JSON makes uses of the configured Serializer internally. See the Serialization and Transcoding section below for details.

What to look out for

The SDKs are more proactive in retrying with certain errors and in certain situations, within the timeout budget given by the user — as an example, temporary failures or locked documents are now being retried by default — making it even easier to program against certain error cases. This behavior is customizable in a RetryStrategy, which can be overridden on a per operation basis for maximum flexibility if you need it.

Note, most of the bootstrap sequence is now lazy (happening behind the scenes). For example, opening a bucket is not raising an error anymore, but it will only show up once you perform an actual operation. The reason behind this is to spare the application developer the work of having to do error handling in more places than needed. A bucket can go down 2ms after you opened it, so you have to handle request failures anyway. By delaying the error into the operation result itself, there is only one place to do the error handling. There will still be situations why you want to check if the resource you are accessing is available before continuing the bootstrap; for this, we have the diagnostics and ping commands at each level which allow you to perform those checks eagerly.

Language Specifics

Now that you are familiar with the general theme of the migration, the next sections dive deep into the specifics. First, installation and configuration are covered, then we talk about exception handling, and then each service (i.e. Key/Value, Query,…​) is covered separately.

Installation and Configuration

As with 2.x release, the primary source of artifacts is the release notes page, where we publish links to pre-built binaries, as well as to source tarballs.

SDK 3.x supports PHP interpreters from 7.2 upwards.

From 3.0 onwards, binaries are available for Windows with the OpenSSL dependency. Note, that OpenSSL DLLs are not distributed in the archive and must be installed separately (see the official OpenSSL page for more details).

Connection Lifecycle

Bootstrapping the SDK is staged now, so the application has to create a Cluster object first, and then open the Bucket and Collection if necessary.

As in SDK 2.x there is no explicit shutdown, and all underlying connections still kept in the cache, for reusing in future requests. The connection idle time is controlled by the couchbase.pool.max_idle_time_sec PHP INI setting.

SDK 3.x allows the performance of Queries on the Cluster level, so it is not necessary to open a bucket anymore.

SDK 3.x does not allow the use of SASL PLAIN mechanism by default, instead it restricts to SCRAM-SHA{1,256,512}.

Exception Handling

SDK 3.x actively uses Exceptions to signal errors. Instead of using single \Couchbase\Exception, as in SDK 2.x, now we use a hierarchy of exceptions, which allows the handling of errors in a more reliable way:

try {
  $collection->get("foo");
} catch (\Couchbase\KeyNotFoundException $ex) {
  $collection->upsert("foo", ["bar" => 42]);
}

Instead of SDK 2.x’s:

try {
  $bucket->get("foo");
} catch (\Couchbase\Exception $ex) {
  if ($ex->getCode() == COUCHBASE_KEYNOTFOUND) {
    $bucket->upsert("foo", ["bar" => 42]);
  }
}

Serialization and Transcoding

SDK 3.x still relies on native types and supports the json_encode API from the standard json.so module (therefore it still has to be loaded before couchbase.so). But the igbinary.so transcoder is no longer supported.

Migrating Services

Key Value

Most of the KV APIs have moved from bucket-level (in SDK 2.x) to collection-level (in SDK 3.x). For servers which don’t support collections, the application should obtain the default collection using the bucket->defaultCollection() function.

The following table describes the SDK 2 KV APIs and their equivalents in SDK 3:

Table 1. SDK 2.x KV API vs. SDK 3.x KV API
SDK 2 SDK 3

Bucket->upsert

Collection->upsert

Bucket->get

Collection->get

-

Collection->exists

Bucket->getFromReplica

Collection->getAnyReplica and Collection.getAllReplicas

Bucket->getAndLock

Collection->getAndLock

Bucket->getAndTouch

Collection->getAndTouch

Bucket->insert

Collection->insert

Bucket->upsert

Collection->upsert

Bucket->replace

Collection->replace

Bucket->remove

Collection->remove

Bucket->unlock

Collection->unlock

Bucket->touch

Collection->touch

Bucket->lookupIn

Collection->lookupIn

Bucket->mutateIn

Collection->mutateIn

Bucket->counter

BinaryCollection->increment and BinaryCollection->decrement

Bucket->append

BinaryCollection->append

Bucket->prepend

BinaryCollection->prepend

The BinaryCollection mentioned above could be retrieved from the regular collection object using the $collection->binary() method.

Query

Ib SDK 3.x, the API for Query was improved and now it is more consistent with other endpoints.

SDK 2
$query = N1qlQuery::fromString('SELECT airportname FROM `travel-sample` WHERE city=$city AND type=$type');
$query->namedParams(['city' => "Los Angeles", 'type' => "airport"]);
$result = $bucket->query($query);
foreach ($result->rows as $row) {
  printf("%s\n", $row->airportname);
}
SDK 3
$options = new QueryOptions();
$options->namedParameters(['city' => "Los Angeles", 'type' => "airport"]);
$result = $cluster->query('SELECT airportname FROM `travel-sample` WHERE city=$city AND type=$airport', $options);
foreach ($result->rows() as $row) {
  printf("%s\n", $row['airportname']);
}

Analytics

Analytics queries in SDK3 have their own API entry point.

SDK 2
$query = AnalyticsQuery::fromString('SELECT * FROM dataset WHERE type = $type');
$query->namedParams(['type' => "airport"]);
$result = $bucket->query($query);
foreach ($result->rows as $row) {
  printf("%s\n", $row->airportname);
}
SDK 3
$options = new AnalyticsQueryOptions();
$options->namedParameters(['type' => "airport"]);
$result = $cluster->analyticsQuery('SELECT * FROM dataset WHERE type = $type', $options);
foreach ($result->rows() as $row) {
  printf("%s\n", $row['airportname']);
}

In SDK 3, query options and index name has been extracted from the query object.

SDK 2
$queryPart = SearchQuery::matchPhrase("hop beer");
$query = new SearchQuery("beer-search", $queryPart);
$query->limit(3)->fields("name");

$result = $this->bucket->query($query);
foreach ($result->hits() as $hit) {
  printf("%s - %f\n", $hit->id, $hit->score);
}
SDK 3
$query = new MatchPhraseSearchQuery("hop beer");
$options = new SearchOptions();
$options->limit(3);
$result = $cluster->search("beer-search", $query, $options);
foreach ($result->rows() as $row) {
  printf("%s - %f\n", $row['id'], $row['score']);
}

Views

The most noticeable change in the Views API for SDK 3.x is the change of names for consistency control settings.

SDK 2
$query = ViewQuery::from('design_name', 'test');
$query->consistency(ViewQuery::UPDATE_BEFORE);
$res = $bucket->query($query);
foreach ($res->rows as $row) {
  printf("%s\n", $row->id);
}
SDK 3
$options = new ViewOptions();
$options->scanConsistency(ViewScanConsistency::REQUEST_PLUS);
$res = $bucket->viewQuery('design_name', 'test', $options);
foreach ($res->rows() as $row) {
  printf("%s\n", $row->id());
}

Management APIs

In SDK 2, the management APIs were centralized in the ClusterManager at the cluster level and the BucketManager at the bucket level. Since SDK 3 provides more management APIs, they have been split up into their respective domains. For example, when in SDK 2 you needed to remove a bucket you would call ClusterManager.removeBucket — you will now find it under BucketManager.dropBucket. And, creating a N1QL index now lives in the QueryIndexManager, which is accessible through the Cluster.

The following table provides a mapping from the SDK 2 management APIs to those of SDK 3:

Table 2. SDK 2.x vs SDK 3.x ClusterManager
SDK 2 SDK 3

ClusterManager->info

removed

ClusterManager->listBuckets

BucketManager->getAllBuckets

-

BucketManager->getBucket

ClusterManager->createBucket

BucketManager->createBucket

ClusterManager->removeBucket

BucketManager->removeBucket

ClusterManager->upsertUser

UserManager->upsertUser

ClusterManager->removeUser

UserManager->dropUser

ClusterManager->listUsers

UserManager->getAllUsers

ClusterManager->getUser

UserManager->getUser

Table 3. SDK 2.x vs SDK 3.x BucketManager
SDK 2 SDK 3

BucketManager->info

removed

BucketManager->flush

BucketManager->flushBucket

BucketManager->listDesignDocuments

ViewIndexManager->getAllDesignDocuments

BucketManager->getDesignDocument

ViewIndexManager->getDesignDocument

BucketManager->removeDesignDocument

ViewIndexManager->dropDesignDocument

BucketManager->insertDesignDocument

ViewIndexManager->upsertDesignDocument

BucketManager->upsertDesignDocument

ViewIndexManager->upsertDesignDocument

BucketManager->listN1qlIndexes

QueryIndexManager->getAllIndexes

BucketManager->createN1qlIndex

QueryIndexManager->createIndex

BucketManager->createN1qlPrimaryIndex

QueryIndexManager->createPrimaryIndex

BucketManager->dropN1qlIndex

QueryIndexManager->dropIndex

BucketManager->dropN1qlPrimaryIndex

QueryIndexManager->dropPrimaryIndex