A newer version of this documentation is available.

View Latest

Managing Clusters Using the .NET SDK with Couchbase Server

The Couchbase .NET SDK provides an API for managing a Couchbase cluster programmatically: list, configure and create buckets, get cluster information, manipulate nodes in a cluster...

The primary means for managing clusters is through the Couchbase Web UI which provides an easy to use interface for adding, removing, monitoring and modifying buckets. In some instances you may wish to have a programmatic interface. For example, if you wish to manage a cluster from a setup script, or if you are setting up buckets in test scaffolding.

The .NET SDK also comes with some convenience functionality for common Couchbase management requests.

Management operations in the .NET SDK may be performed through several interfaces depending on the object:

  • ClusterProvisioner class which encapsulates some of the complexity of creating a cluster.

  • ClusterManager class (obtained via ICluster.CreateManager(string, string) with the administrative username and password).

  • BucketManager class (obtained via IBucket.CreateManager(string, string) with the bucket name/password or administrative username and password).

Getting Information About the Cluster

When you call ClusterManager.ClusterInfo(), the Result<> it returns is an IClusterInfo object that contains typed information about the general status and configuration of the cluster (as a Pools object) and the configuration of all the buckets (as a List<IBucketConfig>).

Creating and Removing Buckets

The ClusterManager class may be used to create and delete buckets from the Couchbase cluster. It is instantiated through the Cluster's CreateManager method, providing the administrative username and password.

Cluster cluster = new Cluster();
ClusterManager clusterManager = cluster.CreateManager("Administrator", "123456");

To create a bucket, use the ClusterManager#CreateBucket(string) method. The method has additional parameters that each have a default value, but can be set explicitly in order to use a custom value.

Only name parameter is mandatory. Here is the list of parameters available:

  • name: The name of the bucket (mandatory to create one, cannot be updated).

  • bucketType: The type of the bucket (mandatory to create one, cannot be changed). Defaults to BucketTypeEnum.Couchbase, but can also be BucketTypeEnum.Memcached to create a cache bucket.

  • ramQuota: How much memory should each node use for the bucket. This number is specified in megabytes.

  • saslPassword: If specified, makes this bucket password protected, forcing future connects (using the Bucket) class to specify the password parameter.

  • flushEnabled: Enables the BucketManager.Flush() operation to be performed on this bucket (see the Flushing Buckets section below).

  • replicaNumber: The number of replicas to use for the bucket as a ReplicaNumber enum.

  • indexReplicas: Whether or not to replicate indexes.

  • parallelDbAndViewCompaction: Indicates whether database and view files on disk can be compacted simultaneously.

  • threadNumber: Integer from 2 to 8. Change the number of concurrent readers and writers for the data bucket.

The following example creates a "new_bucket" password-protected bucket:

manager.CreateBucket("new_bucket",
    bucketType: BucketTypeEnum.Couchbase,
    saslPassword: "s3cr3t",
    ramQuota: 120, // megabytes
    replicaNumber: ReplicaNumber.TWO,
    indexReplicas: true,
    flushEnabled: true);

Once you no longer need to use the bucket, you may delete the bucket using the ClusterManager.RemoveBucket(string) method:

clusterManager.RemoveBucket("new_bucket");

Flushing Buckets

When a bucket is flushed, all content is removed. Because this operation is potentially dangerous it is disabled by default for each bucket. Bucket flushing may be useful in test environments where it becomes a simpler alternative to removing and creating a test bucket. You may enable bucket flushing on a per-bucket basis using the Couchbase Web Console or when creating a bucket.

You may flush a bucket in the .NET SDK by using the BucketManager.Flush() method:

bucket.CreateManager("new_bucket", "s3cr3t").Flush()

The Flush operation may fail if the bucket does not have flush enabled, returning an error IResult .

N1QL Index Management

You can create and drop N1QL indexes using the SDK. This is especially useful when setting up new applications, or simply when ensuring that a given bucket has certain indexes defined. Indexes can be defined using actual N1QL statements or by using convenience functions within the SDK.

You can manage indexes in the .NET SDK using the BucketManager class, with its various N1QL related methods: ListN1qlIndexes(), CreateN1qlIndex(...), etc...

The following example creates a N1QL secondary index named "fooBar" on the "test" bucket, indexing fields "foo" and "bar":

//getting the bucket manager is omitted
testManager.CreateN1qlIndex("fooBar", defer: false, "foo", "bar");

Note that the methods return an IResult, which could represent a failure notably if the index you attempt to create already exists (or when attempting to delete an index that doesn’t exist).

View Management

Views are stored in design documents. The SDK provides convenient methods to create, retrieve, and remove design documents. To set up views, you create design documents that contain one or more view definitions, and then insert the design documents into a bucket. Each view in a design document is represented by a name and a set of MapReduce functions. The mandatory map function describes how to select and transform the data from the bucket, and the optional reduce function describes how to aggregate the results.

In the .NET SDK, design documents are simply represented as a string, the JSON representation of the design document. All operations on design documents are performed on a IBucketManager instance.

To inspect design documents, you can either retrieve them by name (bucketManager.GetDesignDocument("landmarks")) or iterate through a list of documents (bucketManager.GetDesignDocuments()). The later can include design documents that are still in development mode (that is, running only on a small fraction of the documents) by using the includeDevelopment parameter.

To create or update design documents, use the InsertDesignDocument(string, string) and UpdateDesignDocument(string, string) methods.

The following example inserts a design document with two regular views and one spatial view into a bucket named travel-sample:

//shortcut for the purpose of this snippet only
BucketManager bucketManager = new Cluster().OpenBucket("travel-sample").CreateManager("travel-sample", "");

// the design document string
string designDoc = "{"
 + "    \"views\": {"
 + "        \"by_country\": {"
 + "            \"map\": \"function (doc, meta) { if (doc.type == 'landmark') { emit([doc.country, doc.city], null); } }\""
 + "        }, \"by_activity\": {"
 + "            \"map\": \"function (doc, meta) { if (doc.type == 'landmark') { emit(doc.activity, null); } }\","
 + "            \"reduce\": \"_count\""
 + "        }"
 + "    }, \"spatialViews\": {"
 + "        \"by_coordinates\": {"
 + "            \"map\": \"function (doc, meta) { if (doc.type == 'landmark') { emit([doc.geo.lon, doc.geo.lat], null); } }\""
 + "        }"
 + "    }"
 + "}";

// Insert design document into the bucket
bucketManager.InsertDesignDocument("landmarks", designDoc);

When you want to update an existing document with a new view (or a modification of a view’s definition), you can use the upsertDesignDocument method.

However, this method needs the list of views in the document to be exhaustive, meaning that if you just create the new view definition as previously and add it to a new DesignDocument that you upsert, all your other views will be erased!

The solution is to perform a getDesignDocument, add your view definition to the DesignDocument’s views list, then upsert it. This also works with view modifications, provided the change is in the map or reduce functions (just reuse the same name for the modified View), or for deletion of one out of several views in the document.

To remove a design document from a bucket, pass its name to the RemoveDesignDocument method.

Using the ClusterProvisioner to Create a Whole Cluster

Alternatively to the methods exposed above for dealing with buckets creation, the .NET SDK offers a ClusterProvisioner API that can also create buckets, but goes a little bit beyond as it can be used to provision an entire cluster, including adding/removing nodes.

In order to follow this example, you will need to provision the nodes that will make up the Couchbase cluster. These can be any supported OS, but each must have Couchbase Server installed and the basic networking done so that they can communicate to each other over common networking protocols such as TCP and HTTP.

In this example, we will use Vagrant, Puppet and the "vagrants" project found here: https://github.com/couchbaselabs/vagrants to provision a cluster. Additional steps for installing Puppet, Vagrant and VirtualBox can be found here: http://nitschinger.at/A-Couchbase-Cluster-in-Minutes-with-Vagrant-and-Puppet

Should you need a .Net Core compatible DNS SRV mechanism for discovering a Couchbase cluster dynamically, follow the instructions at Couchbase Labs.

Once the nodes are provisioned, we can continue with the rest of the example.

Using the IP addresses of your nodes that have been provisioned, the client can bootstrap to the entry-point or "EP" node. This done either through a configuration object or through the App.config or Web.config (for ASP.NET projects). In this example we will use a ClientConfiguration object to programmatically create a configuration and bootstrap to the EP node:

var config = new ClientConfiguration
{
    Servers = new List<Uri>
    {
        new Uri("http://192.168.77.101:8091/"),
        new Uri("http://192.168.77.102:8091/"),
        new Uri("http://192.168.77.103:8091/"),
        new Uri("http://192.168.77.104:8091/")
    }
};

var cluster = new Cluster(config);
var provisioner = new ClusterProvisioner(cluster, "Administrator", "password");
var results = await provisioner.ProvisionEntryPointAsync();
foreach (var res in results.Results)
{
    Console.WriteLine(res.Message);
}

In this example, you create a ClientConfiguration and specify the URI’s for each of the four nodes you provisioned in the previous step. The entry-point node or EP node will be the first node in the list. You then create a Cluster object passing in the ClientConfiguration and then create a ClusterProvisioner object passing in the Cluster references and the user and password. Then we call ProvisionEntryPointAsync() which will create an EP node from the first URI in the Servers list that has a "default" Couchbase (persistent) bucket. The password and username will be the administrative password and since we are using the defaults, all three services (data, query, and index) will be enabled. Finally we output the results.

Next you will optionally add two sample buckets: travel-sample and beer-sample.

var result = await provisioner.ProvisionSampleBucketAsync("beer-sample");
Console.WriteLine(result.Message);

var result = await provisioner.ProvisionSampleBucketAsync("travel-sample");
Console.WriteLine(result.Message);

Once again optionally, you may want to add additional buckets. In this case we will add a SASL authenticated Couchbase bucket and a MemcachedBucket:

var result = await provisioner.ProvisionBucketAsync(new BucketSettings
{
    Name = "authenticated",
    SaslPassword = "secret",
    AuthType = AuthType.Sasl,
    BucketType = BucketTypeEnum.Couchbase
});
Console.WriteLine(result.Message);

As you can see the provisioner uses a BucketSettings object rather than named method parameter to define options. The name of the bucket is "authenticated", the password is "secret", and the BucketType is Couchbase, so this will create a persistent bucket.

var result = await provisioner.ProvisionBucketAsync(new BucketSettings
{
    Name = "memcached",
    SaslPassword = "",
    AuthType = AuthType.Sasl,
    BucketType = BucketTypeEnum.Memcached
});
Console.WriteLine(result.Message);

The name of the bucket is "memcached" with no password and the BucketType is Memcached, so this will create an in-memory bucket.

Adding nodes to the cluster: Once you have provisioned and bootstrapped to an EP node, you can add nodes to create a cluster. In order to do this, the ClusterProvisioner object created earlier will be used to add the rest of the nodes defined in the ClientConfiguration in the first step (102, 103, and 104).

var results = await provisioner.ProvisionNodesAsync(
    CouchbaseService.Index,
    CouchbaseService.KV,
    CouchbaseService.N1QL);

This will provision the rest of the nodes in the ClientConfiguration. If you want finer grained control of which services are enabled on each node, you can you use ProvisionNodeAsync. ProvisionNodeAsync provisions a single node.