CRUD Document Operations using the Go SDK with Couchbase Server

This topic covers the basic CRUD operations of creating, retrieving, updating, and deleting documents. It also describes how to do bulk operations and atomic operations.

Document Types

SDK operations use basic Go data types and JSON marshaling functionality.

All Go data types that can be serialized via the JSON encoding library are supported by default. All textual data is represented by UTF-8 when stored in Couchbase Server. If you do not wish to use Go’s JSON functionality and handle your own encoding and decoding, you can implement your own transcoder.

When retrieving documents, documents are received via an out pointer: simply pass a pointer to an object of the appropriate type (or an empty interface) and the decoder will set it to an object of the appropriate type. If you wish to convert to a specific type, simply pass a pointer of the appropriate type and object will be populated with the appropriate values.

var value interface{}
cas, err := bucket.Get("document_id", &value)

var value MyObject{}
cas, err := bucket.Get("document_id", &value)

Creating and Updating Full Documents

You can create or modify documents by using the Bucket.Insert, Bucket.Upsert, and Bucket.Replace methods.

The following example shows how to create a new document with the Insert() method:

myDoc := "Hello World"
cas, err := myBucket.Insert("document_name", &myDoc, 0)
myDoc := "New Value"
cas, err := myBucket.Replace("document_name", &myDoc, cas, 0)

The methods returns an error and the document’s current CAS which may be used for subsequent mutations

The last argument to Upsert and Insert is the document expiration time, while the last arguments to Replace are the CAS and expiration time. Note that only Replace accepts the CAS as input.

If the document does not exist (and Replace was used) then the returned error will be gocb.ErrKeyNotFound. If the document exists and Insert was used, the returned error will be gocb.ErrKeyExists.

Retrieving Full Documents

You may retrieve full documents using the Bucket.Get method. The Get method receives the document ID and a value pointer (see above) to receive the document itself:

var value interface{}
cas, err := myBucket.Get("document_name", &value)

The error and CAS of the document are returned. If the document does not exist, the error will be gocb.ErrKeyNotFound. Note that if any error occurs, the value pointer will not be set.

Note that the value pointer need not be of an empty interface. You can make the value of a more specific type, so long as the document decoder (by default, Go’s JSON unmarshaller) can properly convert the serialized document to the target object.

You may retrieve documents while simultaneously modifying the document’s expiration by using the Bucket.GetAndTouch method. This functions exactly like the Get method, except it takes a different set of arguments: the second argument is the new expiration time of the document, and the third is the value pointer:

var value interface{}
// expires in 5 minutes
cas, err := myBucket.GetAndTouch("document_name", 300, &value)

You may perform a replica read by using the Bucket.GetReplica() method. The last argument is the replica index, and should be set to 0 unless you desire accessing a specific replica. Note that the results from a replica read may not be consistent with the latest version of the document within the cluster and should only be used if the active node is unavailable.

var value interface{}
cas, err := myBucket.GetReplica("document_name", &value, 0)

Deleting Documents

You can delete documents by using the Bucket.Remove() method. You may supply the CAS as a third argument to Remove, in which case the operation will fail if the CAS has since been modified on the server.

cas, err := myBucket.Remove("document_name", 0)

Bulk operations

You can perform bulk operations by using the various *Op (e.g. GetOp, UpsertOp) structures along with the Bucket.Do method.

The *Op structures work by accepting the same parameters that are accepted by the individual methods. Following the execution of the Do method against the list of *Op structures, the remaining fields will be filled out with the results of the operations.

Here is an example that shows how to batch two Insert operations

var items []gocb.BulkOp
items = append(items, &gocb.InsertOp{Key: "document_name_1", Value: "Hello World 1"})
items = append(items, &gocb.InsertOp{Key: "document_name_2", Value: "Hello World 2"})
err := bucket.Do(items)

Note that you may batch operations of different types into the client.

Ensure that the error field of each operation object is inspected, and note that some operations may fail while others may succeed. Even if the Do method returns nil as its error, some errors may have still occurred.

Modifying Expiration

A document’s expiration time (in seconds) may be modified using the Bucket.Touch method

myBucket.Touch("document_id", 500)

Sub-Document Operations

Sub-Document API is available starting Couchbase Server version 4.5. See Sub-Document Operations for an overview.

Sub-document operations save network bandwidth by allowing you to specify paths of a document to be retrieved or updated. The document is parsed on the server and only the relevant sections (indicated by paths) are transferred between client and server. You can execute sub-document operations in the Golang SDK using the Bucket.MutateIn, and Bucket.LookupIn methods.

frag, err := bucket.LookupIn("document_id").
        Get("path.to.get").
        Exists("check.path.exists").Execute()
var fragValue interface{}
err = frag.Content("path.to.get", &fragValue)
if frag.Exists("check.path.exists") {
        // ...
}

Each of these methods accepts a key as its mandatory first argument and returns a builder object which can be used to add one or more command specifications specifying an operation and a document field operand. To submit the operation, invoke the Execute() method on the builder, which returns an DocumentFragment that contains the results and contents of the paths (where applicable).

You can use the Get() or Exists() methods on the builder object returned by LookupIn. Get will instruct the server to return the contents of the path, whereas Exists will merely check if the path exists.

For operations which return values, you may use the DocumentFragment.Content or DocumentFragment.ContentByIndex methods to retrieve the actual value. These methods accept the name (or index) of the path to retrieve, and a value pointer that will be populated with the contents of the path. Like Bucket.Get, the value pointer should be of a type capable of conversion from the serialized JSON.

For mutation operations, you may specify the expiration time and CAS alongside the document ID to retrieve

bucket.MutateIn("document_id", cas, 300).
        Upsert("email", "user@site.com", false).
        AddUnique("sessions.active", "0xeadbeef", true).
        Counter("sessions.loginCount", 1, true).
        Execute()

The above shows some mutation using the subdocument API via Bucket.MutateIn. The methods of the returned builder objects may accept a final boolean parameter indicating whether the path’s parents should be created if they do not exist.

Atomic Document Modifications

You may perform atomic document modifications on special document types: You can use Bucket.Counter for counters:

newValue, cas, err := myBucket.Counter("counter_document", 1, 100, 300)

Will increment the value of counter_document by 1. If the document does not exist, it will be created with an initial value of 100. The expiration time for the document is set to 300 seconds.

Upon successful completion, the Counter method returns the new value of the document.

Specifying Durability Requirements

Durability requirements allow you to place constraints on the success of a mutation based on how redundant the document is. In the Go SDK you should use various *Dura() functions which allow you to specify durability requirements. Methods such as Bucket.Upsert and Bucket.Replace have their *Dura variants as Bucket.UpsertDura and Bucket.ReplaceDura. The durability-enabled variants function like their simple counterparts, but also block until the durability requirement is achieved by requiring two additional arguments, indicating the persistence and replication of the performed mutation:

myBucket.UpsertDura("document_id", "value", 0, 1, 2)

The above snippet will perform an upsert operation with a persist-to of 2 and a replicate-to of 2

Non-JSON Documents

The term Non-JSON documents refers to any kind of document which, when stored on the server in serialized form, cannot be parsed as valid JSON. While non-JSON encodings may offer more CPU and space efficiency for certain documents, some Couchbase services (such as Query) may not be able to search or query non-JSON documents.

In the Go SDK you can either use an existing []byte (in which case the contents will be stored as-is) or implement the Transcoder interface to automatically convert items of given types to your own format.

The snippet below impelements a Transcoder which adds Zlib compression to serialized JSON

type ZlibJsonTranscoder struct {
}

const FmtZLib = 1<<24 // i.e. FmtPrivate

func (ZlibJsonTranscoder) Decode(content []byte, formatFlags uint32, out interface{}) error {
        if ((formatFlags & FmtZLib) != 0) {
                r, err := zlib.NewReader(bytes.NewReader(content))
		        if err != nil {
                        return err
		         }
		         jsonDecoder := json.NewDecoder(r)
		         return jsonDecoder.Decode(out)
        } else {
                return gocb.DefaultTranscoder{}.Decode(content, formatFlags, out)
        }
}

func (ZlibJsonTranscoder) Encode(input interface{}) (encoded []byte, flags uint32, err error) {
        // Always compress
        buf := bytes.NewBuffer([]byte{})
        zw := zlib.NewWriter(buf)
        e := json.NewEncoder(zw)
        err = e.Encode(input)
        zw.Flush()
        log.Printf("Buffer is %d bytes long\n", buf.Len())
        return buf.Bytes(), FmtZLib, err
}

The above example makes use of the private flag, so as to disambiguate it from other recognized SDK value formats. When encoding a value (to be stored on the server) it applies the flag. When retrieving the value, it checks for the presence of that flag and decodes it as Zlib-compressed JSON. If the flag is absent, it delegates the decoding to the default transcoder in the Go SDK.

To actually use the transcoder, you can use the Bucket.SetTranscoder function

bucket.SetTranscoder(ZlibJsonTranscoder{})