CRUD Document Operations Using the .NET SDK with Couchbase Server

You can access documents in Couchbase using methods of the Couchbase.CouchbaseBucket object. The methods for retrieving documents are Get<T>() and LookupIn<TDocument>() and the methods for mutating documents are Upsert<T>(), Insert<T>(), Replace<T>() and MutateIn<TDocument>().

Examples are shown using the synchronous methods. See the section on Async Programming for information on other methods.

Additional options

Update operations also accept a TTL (expiry) value (Expiry) on the passed IDocument which will instruct the server to delete the document after a given amount of time. This option is useful for transient data (such as sessions). By default documents do not expire. See Expiration Overview for more information on expiration.

Update operations can also accept a CAS (Cas) value on the passed document to protect against concurrent updates to the same document. See CAS for a description on how to use CAS values in your application. Since CAS values are opaque, they are normally retrieved when a Document is loaded from Couchbase and then used subsequently (without modification) on the mutation operations. If a mutation did succeed, the returned Document will contain the new CAS value.

Document input and output types

Couchbase stores documents. From an SDK point of view, those documents contain the actual value (like a JSON object) and associated metadata. Every document in the Java SDK contains the following properties, some of them optional depending on the context:

Name Description

Name	Description
`Id`	The (per bucket) unique identifier of the document.
`Content`	The actual content of the document.
`Cas`	The CAS (Compare And Swap) value of the document.
`Expiry`	The expiration time of the document.
`Token`	The optional MutationToken after a mutation.

Id

The (per bucket) unique identifier of the document.

Content

The actual content of the document.

Cas

The CAS (Compare And Swap) value of the document.

Expiry

The expiration time of the document.

Token

The optional MutationToken after a mutation.

There are a few different results using IDocument you may encounter:

Document: The default one in most methods and is backed by a JSON document when stored to Couchbase. Commonly this is a Document<dynamic> object, which the API will handle automatically on behalf of your program.
DocumentResult: The return type for any IDocument centric operation requests.
OperationResult<T>: The return type for any binary operations that return a value.

Because Couchbase Server can store anything and not just JSON, there is the possibility for other implementations of the IDocument interface or the IOperationResult interface. These are usually returned by other methods on the Bucket or are implemented using an ITypeTranscoder, which is outside the scope of this introduction.

Creating and updating full documents

Documents may be created and updated using the IBucket.Upsert<T>(), IBucket.Insert<T>(), and IBucket.Replace<T>() family of methods. Read more about the difference between these methods at Primitive Key-Value Operations in the Couchbase developer guide.

These methods accept an IDocument instance where the following properties are considered if set:

Id (mandatory): The ID of the document to modify (String).
Content (mandatory): The desired new content of the document, this varies per document type used. This Type of the content is specified by the generic Type T parameter.
Expiry (optional): Specify the expiry time for the document. If specified, the document will expire and no longer exist after the given number of seconds. See Expiration Overview for more information.
Cas (optional): The CAS value for the document. If the CAS on the server does not match the CAS supplied to the method, the operation will fail. See Concurrent Document Mutations for more information on the usage of CAS values. Note that the Cas is represented here as a UInt64, but you should consider it an opaque value in your programs.

Other optional arguments are also available for more advanced usage:

TimeSpan: A value for the Expiry on this document.
PersistTo, ReplicateTo: Specify durability requirements for the operations.

Upon success, the returned IDocumentResult instance will contain the new CAS value of the document. If the document is not mutated successfully, an exception is raised depending on the type of error. Further details on the exceptional case may be on the Exception property of the returned IDocumentResult.

Inserting a document works like this:

var doc = new Document<dynamic>{ Id = "document_id", Content = new {Some="value"} };
var result = bucket.Insert(doc);
Console.WriteLine(JsonConvert.SerializeObject(result.Document));

Output: {"Id":"document_id","Cas":1466044876366741504,"Expiry":0,"Content":null,"Token":null}

If the same code is called again, a ResponseStatus.KeyExists will be returned. If you don’t care that the document is overridden, you can use Upsert instead:

var doc = new Document<dynamic>{ Id = "document_id", Content = new {Some="other value"} };
var result = bucket.Upsert(doc);
Console.WriteLine(JsonConvert.SerializeObject(result.Document));

Output: {"Id":"document_id","Cas":1466044876366741599,"Expiry":0,"Content":null,"Token":null}

Finally, a full document can be replaced if it existed before. If it didn’t exist, then a ResponseStatus.KeyExists will be returned:

var doc = new Document<dynamic>{ Id = "document_id", Content = new {Some="other value"} };
var result = bucket.Replace(doc);
Console.WriteLine(result.Status);

Output: KeyExists

Retrieving full documents

Documents may be retrieved using the IBucket.Get(), IBucket.GetWithLock(), IBucket.GetAndTouch() and IBucket.GetFromReplica()methods. All of those serve different distinct purposes and accept different parameters.

Most of the time you use the get() method. It accepts one mandatory argument:

id: The document ID to retrieve

Console.WriteLine(bucket.GetDocument<dynamic>("document_id").Content);

Output: { "some": "value" }

It is also possible to read from a replica if you want to explicitly trade availability for consistency during the timeframe when the active partition is not reachable (for example during a node failure or netsplit).

GetFromReplica has one mandatory argument as well:

id: The document ID to retrieve

var result = bucket.getFromReplica("document_id");
Console.WriteLine(result.Status);

Since a replica is updated asynchronously and eventually consistent, reading from it may return stale and/or outdated results!

If you need to use pessimistic write locking on a document you can use the GetWithLock which will retreive the document if it exists and also return its CAS value. You need to provide a time that the document is maximum locked (and the server will unlock it then) if you don’t update it with the valid cas. Also note that this is a pure write lock, reading is still allowed.

// Get and Lock for max of 10 seconds
var ownedDoc = bucket.GetWithLock<dynamic>("document_id", new TimeSpan(0, 0, 10));

// Do something with your document
var modifiedDoc = ModfiyDocument(ownedDoc.Document);

// Write it back with the correct CAS
bucket.Replace(modifiedDoc);

It is also possible to fetch the document and reset its expiration value at the same time. See Modifying Expiration for more information.

Removing full documents

Documents may be removed using the IBucket.Remove() method. This method takes a single mandatory argument:

id: The ID of the document to remove.

Some additional options:

PersistTo, ReplicateTo: Specify durability requirements for the operations.
Timeout: Specify a custom timeout which overrides the default timeout setting.

If the cas value is set on the Document overload, it is used to provide optimistic currency, very much like the Replace operation.

// Remove the document
var result = bucket.Remove("document_id");

var loaded = bucket.GetDocument<dynamic>("document_id");

// Remove and take the CAS into account
var removed = bucket.Remove(loaded);

Modifying expiraton

Modifying the Document expiration can be performed using the IBucket.Touch() method. In addition, many methods support setting the expiry value as part of their other primary operations:

IBucket.Touch: Resets the expiry time for the given document ID to the value provided.
IBucket.GetAndTouchDocument: Fetches the document and resets the expiry to the given value provided.
IBucket.Insert, IBucket.Upsert, IBucket.Replace: Stores the expiry value alongside the actual mutation when set on the Document<T> instance.

The following example stores a document with an expiry, waits a bit longer and as a result no document is found on the subsequent get:

uint expiry = 2000; // milliseconds
var stored = bucket.Upsert(new Document<dynamic>
{
    Id = "expires",
    Expiry = expiry,
    Content = new {Some = "value"}
});

Thread.Sleep(3000);

//will be expired, thus KeyNotFound
Console.WriteLine(bucket.GetDocument<dynamic>("expires").Status);

KeyNotFound

Values are converted to seconds before being sent to the server. A time of zero will set the document to never expire (a negative number will set expiry to immediate — creating a tombstone). Values above 0ms but below 1000ms are rounded up to one second before being sent to the server.

You may also use the IBucket.Touch() method to modify expiration without fetching or modifying the document:

bucket.Touch("expires", TimeSpan.FromSeconds(2));

Atomic document modifications

Additional atomic document modifications can be performing using the .NET SDK. You can increase or decrease the value of a document using the IBucket.Increment or IBucket.Decrement methods. You can also use the IBucket.Append() and Bucket.Prepend() methods to perform raw byte concatenation.

Batching Operations

The .NET SDK supports the async/await keywords so batching can easily be done by utilizing the Task.WhenAll method:

var tasks = new List<Task<IDocumentResult<string>>>();
ids.ForEach(x => tasks.Add(bucket.GetDocumentAsync<string>(x)));

var results = await Task.WhenAll(tasks);
results.ToList().ForEach(doc => Console.WriteLine(doc.Status));

In addition, there are overloads of the standard CRUD methonds on IBucket which take a list or dictionary of keys and/or documents and internally batch them asynchronously:

var keys = new []{"key1", "key2", "key3"};
var results = bucket.Get<dynamic>(keys).Values;
results.ToList().ForEach(x=>Console.WriteLine(x.Value));

The main difference between the two is that the second way will block while the results are returned (even if they are retrieved in parrallel), and the first is non-blocking from the calling thread.

Operating with sub-documents

Sub-Document API is available starting Couchbase Server version 4.5. See Sub-Document Operations for an overview.

Sub-document operations save network bandwidth by allowing you to specify paths of a document to be retrieved or updated. The document is parsed on the server and only the relevant sections (indicated by paths) are transferred between client and server. You can execute sub-document operations in the .NET SDK using the IBucket.LookupIn() and IBucket.MutateIn() methods.

Each of these methods accepts a key as its mandatory first argument and give you a builder that you can use to chain several command specifications, each specifying the path to be impacted by the specified operation and a document field operand. You may find all the operations in the LookupInBuilder and MutateInBuilder classes.

bucket.LookupIn("docid")
    .Get("path.to.get")
    .Exists("check.path.exists")
    .Execute();

boolean createParents = true;
bucket.MutateIn("docid")
    .Upsert("path.to.upsert", value, createParents)
    .Remove("path.to.del"))
    .Execute();

All sub-document operations return a special IDocumentFragment<T> object rather than a IDocument<T>. It shares the Id, Cas and MutationToken fields of a document, but in contrast with a normal IDocument<T> object, a IDocumentFragment<T> object contains multiple results with multiple statuses, one result/status pair for every input operation. So it exposes method to get the Content() and Status() of each spec, either by index or by path. It also allows to check that a response for a particular spec Exists():

var res =
bucket.LookupIn("docid")
    .Get("foo")
    .Exists("bar")
    .Exists("baz")
    .Execute();

// First result
res.Content("foo");

// or
res.Content(0);

Using the Content(...) methods will raise an exception if the individual spec did not complete successfully. You can also use the Status(...) methods to return an error code (a ResponseStatus) rather than throw an exception.

Formats and Non-JSON Documents

See Non-JSON Documents for a general overview of using non-JSON documents with Couchbase

The .NET SDK supports documents in the form of POCOs (plain old csharp objects) as long as they are serializable (public getter/setter properties), dynamic types, and most of the representations of JSON objects from the 3rd party API’s such as NewtonSoft.JSON. In general, if the Type you are storing is an object capable of being serialized, it will be stored natively as JSON in couchbase. Exceptions include non-JSON strings, byte arrays, and any value that is not representable as JSON. If the value is not serializable to JSON, then it will be treated as a binary document. Note that binary documents cannot be queried using N1QL nor do the Sub-Document methods work on them.