Introduction

A newer version of this software is available

You are viewing the documentation for an older version of this software. To find the documentation for the current version, visit the Couchbase documentation home page.

This guide provides information for developers who want to use the Couchbase C SDK to build applications that use Couchbase Server.

Getting Started

Now that you’ve installed Couchbase and have created a cluster of servers, you need a client library to read and write data from the cluster. The Couchbase C client library, also known as libcouchbase, can be used from your application to access Couchbase Server.

Here’s a quick outline of what you’ll learn in this chapter:

  • Get the client library.

  • Build the client library from source (optional).

  • Write a simple program to connecting to Couchbase and save some data.

Downloading the Couchbase Client Library

Download the latest client library from the Couchbase download site. It is available as a source archive in both .zip and .tar.gz

Installing on Linux using Packages

Packages are provided for RedHat and Ubuntu Linux systems by providing your package manager with the Couchbase repository information. The method for installation is dependent on your platform:

RedHat/CentOS

You must populate RPM with a new source, which is dependent on your RedHat version:

Then to install libcouchbase with libevent backend, run:

shell> sudo yum check-update
shell sudo yum install -y  libcouchbase2-libevent libcouchbase-devel

Ubuntu

You must update the apt-get repository to install the client library:

Also make sure you have the GPG key installed:

shell> wget -O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add -

Then to install libcouchbase with libevent backend, run:

shell> sudo apt-get update
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev

Installing using packages on Mac OS X

This client library is available via a homebrew recipe. After you install homebrew, install libcouchbase:

shell> brew update
shell> brew install libcouchbase

Installing from Source: Linux and Mac OS X

Installation is like many common libraries, following the ./configure, make, make install conventions.

For libvbucket, extract the archive then cd into the directory and run:

shell> ./configure
shell> make install

Standard configure options such as --prefix can be passed to the configure command. For additional information on configuration options, run./configure with –help.

For libcouchbase, extract the archive, then cd into the directory and run:

shell> ./configure CPPFLAGS="-I/opt/couchbase/include" --disable-couchbasemock
shell> make install

The --disable-couchbasemock simply disables some tests which are common during the development of libcouchbase, but not required when installing a release.

Installing from Source: Microsoft Windows

Spin up your visual studio shell and run cmake from there. It is best practice that you make an out-of-tree build; thus like so:

Assuming Visual Studio 2010

C:\> git clone git://github.com/couchbase/libcouchbase.git
C:\> mkdir lcb-build
C:\> cd lcb-build
C:\> cmake -G "Visual Studio 10" ..\libcouchbase
C:\> msbuild /M libcouchbase.sln

This will generate and build a Visual Studio .sln file.

Windows builds are known to work on Visual Studio versions 2008, 2010 and 2012.

Hello C Couchbase

The C client library, libcouchbase, is a callback-oriented client which makes it very easy to write high performance programs. There are a few ways you can drive IO with the library. The simplest approach is to use the synchronous interface over the asynch internals of the library. More advanced programs will either call the libcouchbase_wait() function after generating some operations or drive the event loop themselves.

To connect, you first configure the connection options and then create an instance of the connection to the cluster:

struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;

memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";

err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
    fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);

/*
 * Initiate the connect sequence in libcouchbase
 */
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
    fprintf(stderr, "Failed to initiate connect: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Run the event loop and wait until we've connected */
lcb_wait(instance);

Callbacks are used by the library and are simple functions which handle the result of operations. For example:

struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;

memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";

err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
    fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);

/*
 * Initiate the connect sequence in libcouchbase
 */
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
    fprintf(stderr, "Failed to initiate connect: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Run the event loop and wait until we've connected */
lcb_wait(instance);

Callbacks can be set up for all of your operations called in libcouchbase. In the API, you’ll note the use of a cookie. This is metadata from your application which is associated with the request. The libcouchbase library will not inspect any cookie or send the cookie to the server.

When you put the connect logic and the get callback together and plug them into a complete program with the include headers, you get:

#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

static void error_callback(lcb_t instance,
                           lcb_error_t err,
                           const char *errinfo)
{
    fprintf(stderr, "Error %s: %s", lcb_strerror(instance, err),
            errinfo ? errinfo : "");
    exit(EXIT_FAILURE);
}

/* the callback invoked by the library when receiving a get response */
static void get_callback(lcb_t instance,
                         const void *cookie,
                         lcb_error_t error,
                         const lcb_get_resp_t *resp)
{
    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to retrieve \"");
        fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
        fprintf(stderr, "\": %s\n", lcb_strerror(instance, error));
    } else {
        fprintf(stderr, "Data for key: \"");
        fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
        fprintf(stderr, "\" is : ");
        fwrite(resp->v.v0.bytes, 1, resp->v.v0.nbytes, stderr);
    }
    (void)cookie; /* ignore */
}

int main(void)
{
    struct lcb_create_st create_options;
    lcb_t instance;
    lcb_error_t err;

    memset(&create_options, 0, sizeof(create_options));
    create_options.v.v0.host = "myserver:8091";
    create_options.v.v0.user = "mybucket";
    create_options.v.v0.passwd = "secret";
    create_options.v.v0.bucket = "mybucket";

    err = lcb_create(&instance, &create_options);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }

    /* set up the handler to catch all errors */
    lcb_set_error_callback(instance, error_callback);

    /* initiate the connect sequence in libcouchbase */
    err = lcb_connect(instance);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to initiate connect: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }

    /* run the event loop and wait until we've connected */
    lcb_wait(instance);

    /* set up a callback for our get requests  */
    lcb_set_get_callback(instance, get_callback);

    {
        lcb_get_cmd_t cmd;
        const lcb_get_cmd_t *commands[1];

        commands[0] = &cmd;
        memset(&cmd, 0, sizeof(cmd));
        cmd.v.v0.key = "foo";
        cmd.v.v0.nkey = 3;

        err = lcb_get(instance, NULL, 1, commands);
        if (err != LCB_SUCCESS) {
            fprintf(stderr, "Failed to get: %s\n",
                    lcb_strerror(NULL, err));
            return 1;
        }
    }

    lcb_wait(instance);

    lcb_destroy(instance);
    exit(EXIT_SUCCESS);
}

To compile this sample program, you must link to libcouchbase :

shell> gcc -o hellocb -lcouchbase hellocb.c

Tutorial

This tutorial assumes you have installed libcouchbase on your systems per the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.

If you need to set up a server node or data bucket, you can do so with Couchbase Administrative Console, Couchbase Command-Line Interface (CLI), or Couchbase REST API. For information and instructions, see:

After you’ve set up your Couchbase Server and installed the needed client libraries, you can compile and run the following basic program.

Simple Example

Before getting into a more complex example of the programming model to this library, we will walk through a straightforward example of a program which builds with libcouchbase, connects to a server, and sets and gets a value:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libcouchbase/couchbase.h>

static void die(const char *msg, lcb_error_t err)
{
    fprintf(stderr, "Got error %s. Code=0x%x, %s\n",
            msg, err, lcb_strerror(NULL, err));
    exit(EXIT_FAILURE);
}

static void get_callback(
    lcb_t instance,
    const void *cookie,
    lcb_error_t err,
    const lcb_get_resp_t *resp
);

static const char *key = "My Key";
static const char *value = "My Value";

int main(void)
{
    lcb_t instance;
    lcb_error_t err;
    lcb_store_cmd_t s_cmd = { 0 }, *s_cmdlist = &s_cmd;
    lcb_get_cmd_t g_cmd = { 0 }, *g_cmdlist = &g_cmd;
    char scratch[4096];
    
    err = lcb_create(&instance, NULL);
    if (err != LCB_SUCCESS) {
        die("Couldn't create instance", err);
    }
    
    lcb_set_get_callback(instance, get_callback);
    err = lcb_connect(instance);
    if (err != LCB_SUCCESS) {
        die("Couldn't schedule connection", err);
    }
    lcb_wait(instance);
    
    s_cmd.v.v0.key = key;
    s_cmd.v.v0.nkey = strlen(key);
    s_cmd.v.v0.bytes = value;
    s_cmd.v.v0.nbytes = strlen(value);
    s_cmd.v.v0.operation = LCB_SET;
    
    err = lcb_store(instance, NULL, 1,
            (const lcb_store_cmd_t * const *)&s_cmdlist);

    if (err != LCB_SUCCESS) {
        die("Couldn't schedule store operation!", err);
    }
    lcb_wait(instance);
    
    g_cmd.v.v0.key = s_cmd.v.v0.key;
    g_cmd.v.v0.nkey = s_cmd.v.v0.nkey;
    err = lcb_get(instance, scratch, 1,
            (const lcb_get_cmd_t * const *)&g_cmdlist);
    if (err != LCB_SUCCESS) {
        die("Couldn't schedule get operation!", err);
    }
    lcb_wait(instance);
    
    /**
     * Inside the get callback we've copied over the value data from the
     * callback into the scratch buffer which we passed as a cookie
     */
    {
        char reversed[4096];
        int ii;
        int curlen = strlen(scratch);
        curlen--;
        for (ii = curlen; ii >= 0; ii--) {
            reversed[curlen-ii] = scratch[ii];
        }
        s_cmd.v.v0.bytes = reversed;
        s_cmd.v.v0.operation = LCB_APPEND;
        /** Since it's reversed there's no need to re-set the size again */
        err = lcb_store(instance, NULL, 1,
                (const lcb_store_cmd_t * const *)&s_cmdlist);
        if (err != LCB_SUCCESS) {
            die("Couldn't schedule second APPEND", err);
        }
        lcb_wait(instance);
    }
    /** Now get the key back again */
    err = lcb_get(instance, scratch, 1,
            (const lcb_get_cmd_t * const *)&g_cmdlist);
    if (err != LCB_SUCCESS) {
        die("Could not schedule final get operation", err);
    }
    lcb_wait(instance);
    printf("Buffer in server is now %s\n", scratch);
    lcb_destroy(instance);
}

static void get_callback(
    lcb_t instance,
    const void *cookie,
    lcb_error_t err,
    const lcb_get_resp_t *resp)
{
    char *target = (char *)cookie;
    if (err != LCB_SUCCESS) {
        die("Got error inside get callback", err);
    }
    memcpy(target, resp->v.v0.bytes, resp->v.v0.nbytes);
    target[resp->v.v0.nbytes] = '\0';
}

Additional Resources

In addition to the other sections of this manual, such as the Getting Started guide and the API reference, the libcouchbase package includes an examples directory and a tools directory. Each of these show simple Couchbase tools and an example libcouchbase programs.

Error Handling and Diagnostics

This section contains information about handling errors and logging error messages.

Inspecting Return Codes

Most operations return an lcb_error_t status code. A successful operation is defined by the return code of LCB_SUCCESS. You can find a full list of error codes in the <libcouchbase/error.h> header.

To handle the errors properly, you must understand what the errors mean and whether they indicate a data error, that the operation can be retried, or a fatal error.

Data errors are errors received from the cluster as a result of a requested operation on a given item. For example, if an lcb_get is performed on a key that does not exist, the callback will receive an LCB_KEY_ENOENT error. Other examples of conditions that can result in data errors include:

  • Adding a key that already exists
  • Replacing a key that does not already exist
  • Appending, prepending, or incrementing an item that does not already exist
  • Modifying an item while specifying a CAS, where the CAS on the server has already been modified

Errors related to load or network issues might be received in exceptional conditions. Although you should not receive these types of errors in normal, well-provisioned deployments, your application must be prepared to handle them and take proper action.

The latter type of errors may be further divided into the following subgroups:

  • Transient. This error type indicates an environment and/or resource limitation either on the server or on the network link between the client and the server. Examples of transient errors include timeout errors or temporary failures such as LCB_ETIMEDOUT (took too long to get a reply) and LCB_ETMPFAIL (server was too busy). Transient errors are typically best handled on the application side by backing off and retrying the operation, with the intent of reducing stress on the exhausted resource. Some examples of transient error causes:

    • Insufficient cache memory on the server
    • Overutilization of the network link between the client and server or between several servers
    • Router or switch failure
    • Failover of a node
    • Overutilization of application-side CPU
  • Fatal. This error type indicates that the client has potentially entered into an irrecoverable failed state, either because of invalid user input (or client configuration), or because an administrator has modified settings on the cluster (for example, a bucket has been removed). Examples of fatal errors include LCB_AUTH_ERROR (authentication failed) and LCB_BUCKET_ENOENT (bucket does not exist). Fatal errors typically require inspection of the client configuration and a restart of the client application or a reversal of the change performed at the cluster. Some examples of fatal error causes:

    • Bucket does not exist
    • Bucket password is wrong

The lcb_errflags_t enumeration defines a set of flags that are associated with each error code. These flags define the type of error. Some examples of error types:

  • LCB_ERRTYPE_INPUT, which is set when a malformed parameter is passed to the library
  • LCB_ERRTYPE_DATAOP which is set when the server is unable to satisfy data constraints such as a missing key or a CAS mismatch.

The LCB_EIF<TYPE> methods, where <TYPE> represents one of the errflags_t flags, can be used to check whether an error is of a specific type.

The following example shows how to check the return codes:

static void get_callback(
    lcb_t instance,
    const void *cookie,
    lcb_error_t err,
    const lcb_get_resp_t *resp)
{
    if (err == LCB_SUCCESS) {
        printf("Successfuly retrieved key!\n");
    } else if (LCB_EIFDATA(err)) {
        switch (err) {
            case LCB_KEY_ENOENT:
                printf("Key not found!\n");
                break;
            default:
                printf("Received other unhandled data error\n");
                break;
        }
    } else if (LCB_EIFTMP(err)) {
        printf("Transient error received. May retry\n");
    }
}

Success and Failure

Success and failure depend on the context. A successful return code for one of the data operation APIs (for example, lcb_store) does not mean the operation itself succeeded and the key was successfully stored. Rather, it means the key was successfully placed inside the library’s internal queue. The actual error code is delivered within the response itself.

Errors Received in Scheduling

Errors might be received when scheduling operations inside the library. If a scheduling API returns anything but LCB_SUCCESS, then that implies the operation itself failed as a whole and no callback will be delivered for that operation. Conversely, if a scheduling API returns an LCB_SUCCESS then the callback will always be invoked.

The library might also mask errors during the scheduling phase and deliver them asynchronously to the user via a callback (for example, when implementation constraints do not easily allow the immediate returning of an error code).

Logging

You can use the library’s logging API to forward messages to your logging framework of choice. Additionally, you can enable logging to the console’s standard error by setting the LCB_LOGLEVEL environment variable.

Setting the logging level via the environment variable allows applications linked against the library that do not offer native support for logging to still employ the use of the diagnostics provided by the library. You do something like this to display logging information:

$ LCB_LOGLEVEL=5 ./my_app

The value of the LCB_LOGLEVEL environment variable is an integer from 1 to 5. The higher the value, the more verbose the details. The value of 0 disables the console logging.

To set up your own logger, you must define a logging callback to be invoked whenever the library emits a logging message

static void logger(
    lcb_logprocs *procs,
    unsigned int iid,
    const char *module,
    int severity,
    const char *srcfile,
    int srcline,
    const char *fmt,
    va_list ap)
{
    char buf[4096];
    vsprintf(buf, ap, buf);
    dispatch_to_my_logging(buf);
}

static void apply_logging(lcb_t instance)
{
    lcb_logprocs procs = { 0 };
    procs.v.v0.callback = logger;
    lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_LOGGER, &procs);
}

The lcb_logprocs pointer must point to valid memory and must not be freed by the user after passing it to the library until the instance associated with it is destroyed.

The arguments to the logging function are:

  • procs—The logging procedure structure passed to the lcb_cntl() operation
  • iid—An integer used to identify the lcb_t. This is useful if you have multiple instances running in your application
  • module—A short string representing the subsystem that emitted the message
  • severity—An integer describing the severity of the message (higher is more severe). Refer to the lcb_log_severity_t enum within <libcouchbase/types.h> for a full listing.
  • srcfile and srcline—File and line where this message was emitted
  • fmt—a format string
  • ap—arguments to the format string

Additional diagnostic information is provided by the error callback. The error callback is a legacy interface and should generally not be used. The error callback, however, does allow programmatic capture of some errors—something that is not easy with the logging interface.

Specifically, the error callback receives error information when a bootstrap or configuration update has failed.

static void error_callback(lcb_t instance, lcb_error_t err, const char *msg)
{
    /** ... */
}

lcb_set_error_callback(instance, error_callback);

Configuring and Tuning

SASL

libcouchbase version 2.2 and later supports the CRAM-MD5 authentication mechanism, which enables you to avoid passing the bucket password as plain text over the wire.

Along with this change, a new setting was introduced: LCB_CNTL_FORCE_SASL_MECH, which forces a specific Simple Authentication and Security Layer (SASL) mechanism to use for authentication. This can allow a user to ensure a certain level of security and have the connection fail if the desired mechanism is not available.

lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_FORCE_SASL_MECH, "CRAM-MD5");

Configuration and Bootstrapping

libcouchbase 2.3 and later supports multiple methods of bootstrapping and configuration retrieval. Normally you should not change these settings because the library determines the best method available.

Configuration Sources

You can obtain the cluster configuration by using either the memcached protocol that uses the cluster configuration carrier protocol (CCCP) or the legacy mode that uses a connection to the REST API. CCCP is available only in cluster version 2.5 and later and libcouchbase version 2.3 and later.

The default behavior of the library is to first attempt bootstrap over CCCP and then fall back to HTTP if CCCP fails. CCCP bootstrap is preferable because it does not require a dedicated configuration socket, does not require the latency and overhead of the HTTP protocol, and is more efficient to the internals of the cluster.

To explicitly define the set of configuration modes to use, specify the modes inside the lcb_create_st structure.

The following snippet disables HTTP bootstrapping:

struct lcb_create_st crparams = { 0 };

lcb_config_transport_t transports[] = {
    LCB_CONFIG_TRANSPORT_CCCP,
    LCB_CONFIG_TRANSPORT_LIST_END
};

/** Requires version 2 */
cparams.version = 2;
crparams.v.v2.transports = transports;
crparams.v.v2.host = "foo.com;bar.org;baz.net";

lcb_t instance;
lcb_create(&instance, &crparams);

The transports field accepts an array of enabled transports followed by the end-of-list element (LCB_CONFIG_TRANSPORT_LIST_END). If this parameter is specified, the library only uses the transports that are specified in the list.

The enabled transports remain valid for the duration of the instance. This means that it is used for the initial bootstrap and any subsequent configuration updates.

Memcached Buckets

Memcached buckets do not support CCCP. You must ensure that HTTP is enabled for them (which is the default)

Configuration Updates

During the lifetime of a client, its cluster configuration might need to change or be retrieved. Reasons for this include:

  • The cluster pushing a new configuration to the client
  • Nodes being added, removed, or failed over to or from the cluster
  • Client proactively fetching configuration due to an error

These changes are collectively known as topology changes or configuration changes. As mentioned in the list, fetching a configuration might not always be the result of an actual topology change. For example, if a certain node becomes unreachable to the client, then the client will attempt to fetch a new configuration under the assumption that the unresponsive node has potentially been ejected from the cluster.

For proactive configuration fetches (such as in response to an error) there is effectively a responsiveness-for-efficiency trade off. If the client does not check for updated configurations often enough, it risks failing operations that could have succeeded if it knew about the updated topology. If the client fetches the configuration too quickly, it can cause a lot of traffic and increase load on the server (this especially holds true with HTTP). Because some server versions allocate quite a few resources for each HTTP connection, maintaining idle HTTP connections might be more expensive than re-establishing them on demand.

Rather than guess your hardware and infrastructure requirements, there are several settings in the library to help you tune it to your needs.

Error Thresholds

Each time a network-like error is encountered in the library (that is, a timeout or connection error), an internal error counter is incremented. When the counter reaches a specified value, the library fetches a new configuration and resets the counter to zero.

The intervals between successive increments are timed. If a call is made to increment the counter and the time between the last call exceeds a certain threshold, then the configuration is refetched again.

You can control both behaviors by using the lcb_cntl() method. To modify the error counter threshold, use the LCB_CNTL_CONFERRTHRESH setting. To modify the error delay threshold, use the LCB_CNTL_CONFDELAY_THRESH setting.

File-Based Configuration Cache

The library allows you to cache the configuration settings in a file and then pass the file to create another instance. The new instance is configured without having to perform network I/O to retrieve the initial map.

The goal of this feature is to limit the number of initial connections for retrieving the cluster map and to reduce latency for performing simple operations. Such a usage is ideal for short lived applications that are spawned as simple scripts (for example, PHP scripts).

To use the configuration cache you must use a special form of initialization:

lcb_error_t err;
lcb_t instance;
struct lcb_create_st cropts;
cropts.v.v0.host = "foo.com";
err = lcb_create(&instance, &cropts);
if (err != LCB_SUCCESS) {
    die("Couldn't create instance", err);
}

/* now set the cache */
const char *cachefile = "some_file";
err = lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_CONFIGCACHE, (void*)cachefile);
if (err != LCB_SUCCESS) {
    die("Couldn't assign the configuration cache!\n");
}

If the cache file does not exist, the client bootstraps from the network and writes the information to the file after it has received the map itself. You can check to see if the client has loaded a configuration from the cache (and does not need to fetch the configuration from the network) by using lcb_cntl()

int is_loaded = 0;
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
if (is_loaded) {
    printf("Configuration cache file loaded OK\n");
} else {
    printf("Configuration cache likely does not exist yet\n");
}
Limitations

You need to be aware of the following limitations:

  • Memcached buckets are not supported with the configuration cache.

  • The file-based provider is recommended only for short-lived clients. It is intended to reduce network I/O and latency during the initial bootstrap process. It is not a recommended solution for reducing the total number of connections to the cluster.

Timeouts

Timeouts prevent a single operation from hanging or waiting indefinitely for an I/O operation to complete. Typically, timeouts are exact, but if they are not, they tend to fire early rather than late.

Internally, timeouts are implemented within an event loop (as opposed to a kernel-based signal timer). This means:

  • The timer can fire only when and if the event loop is active.
  • The timer might be delayed if it is scheduled after other operations that perform blocking timeouts or long busy loops. (libcouchbase does not do this, but your own code might).

As an I/O library, libcouchbase has quite a few timeout settings. The main timeout setting is the operation timeout, which indicates the amount of time to wait from when the operation was scheduled. If a response is not received within this time period, the library fails the operation and invokes the callback, at which point the error will be set to LCB_ETIMEDOUT. The operation to control this via lcb_cntl is LCB_CNTL_OP_TIMEOUT.

The bootstrap timeout, which controls the amount of time the library waits for the initial bootstrap, is another important timeout. The bootstrap timeout comprises two timeout values, the absolute timeout and the per-node bootstrap timeout. The absolute timeout controls the overall amount of time the library should wait until a configuration has been received, while the per-node bootstrap timeout controls the amount of time per-node that the library will wait until the next node is retried.

You can access the absolute timeout via the LCB_CNTL_CONFIGURATION_TIMEOUT. This value places an absolute limit on the amount of time the library waits until the client has been bootstrapped before delivering an error. This timeout is used only during the initial connection and has no effect thereafter.

You can access the bootstrap timeout via the LCB_CNTL_CONFIG_NODE_TIMEOUT operation. When fetching the cluster configuration there are often multiple nodes that might function as configuration sources. Configurations are retrieved once during initialization, but might also be fetched later on, typically when an error condition is detected by the library or when the cluster topology changes. In such situations, the behavior is to start fetching the configuration from each node in the list, in sequence. If the retrieval of the configuration from the first node fails with a timeout (that is, the node is unresponsive) the library will proceed to fetch from the next node and so on until all nodes are exhausted.

Integration with libcouchbase

This tutorial assumes you have installed libcouchbase on your systems, following the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.

The libcouchbase is written in C and can be integrated in various ways with your application. The simplest integration scenario is when your application is written in a scripting language (supported by a Couchbase client library) that already has a libcouchbase wrapper. Couchbase has built and maintains following libcouchbase-based libraries:

Users of these libraries automatically get an API, which looks natural for their platform, so that it doesn’t require knowledge of libcouchbase APIs. This tutorial is mostly about two other use cases:

  • Add libcouchbase into an existing application to implement persistence layer. This section will contain two parts: first we will create a simple echo server written using libev library. Second, we will show how to persist each message going through the server to Couchbase.

    Get complete code here: https://github.com/couchbaselabs/libev-couchbase-example

  • Build applications around libcouchbase. Here we’ll show how to build a proxy server for regular memcached clients. Building this proxy server is just like building a moxi server, but the proxy server will be more limited more limited in abilities.

    Get complete code here: https://github.com/couchbaselabs/libcouchbase-proxy-sample

Add to an Application

Let’s start with a specific existing application for this example. Our application is the asynchronous single-threaded echo server, which leverages libev to solve C10K problem and efficiently serves a huge number of concurrent connections. It should listen to the given port (in this sample 4567) and send back everything the client submitted to it.

The libev library implements reactor pattern which uses the term event loop to represent an entity which encapsulates the register of handlers for various IO events and the timers. To start, we have the application code from the main function:

int main()
{
    struct ev_loop *loop = ev_default_loop(0);
    int sd;
    struct sockaddr_in addr;
    struct ev_io server;

    if ((sd = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
        fail("socket error");
    }

    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(PORT_NO);
    addr.sin_addr.s_addr = INADDR_ANY;

    if (bind(sd, (struct sockaddr *) &addr, sizeof(addr)) != 0) {
        fail("bind error");
    }
    if (listen(sd, 2) < 0) {
        fail("listen error");
    }

    ev_io_init(&server, accept_cb, sd, EV_READ);
    ev_io_start(loop, &server);

    printf("Listen on %d\n", PORT_NO);
    ev_run(loop, 0);

    return 0;
}

Here we are create an event loop instance, and create and bind a new socket to local port PORT_NO (4567). After that we can mark this socket as listening and ask libev to call the function accept_cb when the socket become ready to read, that is when the new client will come to us. The final step is to run ev_run() which does all the server interaction.

The accept_cb is in charge of allocating client structures 60 the buffer for messages. The sample is using ring buffers from the libcouchbase code for easier managing memory for IO buffers.

void accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
    struct client_s *client;
    int flags;

    if (EV_ERROR & revents) {
        fail("got invalid event");
    }
    client = malloc(sizeof(struct client_s));
    if (!client) {
        fail("client watcher alloc");
    }
    if (!ringbuffer_initialize(&client->buf, BUFFER_SIZE)) {
        fail("client buffer alloc");
    }
    client->naddr = sizeof(client->addr);
    client->fd = accept(watcher->fd, (struct sockaddr *)&client->addr,
                        &client->naddr);
    if (client->fd < 0) {
        fail("accept error");
    }
    if ((flags = fcntl(client->fd, F_GETFL, NULL)) == -1) {
        fail("fcntl(F_GETFL)");
    }
    if (fcntl(client->fd, F_SETFL, flags | O_NONBLOCK) == -1) {
        fail("fcntl(F_SETFL, O_NONBLOCK)");
    }

    printf("Successfully connected with client\n");

    ev_io_init((ev_io *)client, client_cb, client->fd, EV_READ);
    ev_io_start(loop, (ev_io *)client);
}

The most interesting part of the function above is in the last two lines. It registers the new handler, client_cb, upon reading an event for the client socket. This handler will be called each time when the kernel receives new data from the socket, so that it could be read with a non-blocking call.

The last snippet of our application depicts the client_cb function, which handles all the application logic. It reads everything from the socket if it is ready, registers itself for write events, and then sends all the contents out to the network. It is also the handling end of a stream situation, which means that the client closed the connection with the Couchbase server.

void client_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
    struct lcb_iovec_st iov[2];
    ssize_t nbytes;
    struct client_s *client = (struct client_s *)watcher;

    if (EV_READ & revents) {
        ringbuffer_ensure_capacity(&client->buf, BUFFER_SIZE);
        ringbuffer_get_iov(&client->buf, RINGBUFFER_WRITE, iov);
        nbytes = io_recvv(watcher->fd, iov);
        if (nbytes < 0) {
            fail("read error");
        } else if (nbytes == 0) {
            ev_io_stop(loop, watcher);
            ringbuffer_destruct(&client->buf);
            free(client);
            printf("Peer disconnected\n");
            return;
        } else {
            ringbuffer_produced(&client->buf, nbytes);
            printf("Received %zu bytes\n", nbytes);
            ev_io_stop(loop, watcher);
            ev_io_set(watcher, watcher->fd, EV_WRITE);
            ev_io_start(loop, watcher);
        }
    } else if (EV_WRITE & revents) {
        ringbuffer_get_iov(&client->buf, RINGBUFFER_READ, iov);
        nbytes = io_sendv(watcher->fd, iov);
        if (nbytes < 0) {
            fail("write error");
        } else if (nbytes == 0) {
            ev_io_stop(loop, watcher);
            ringbuffer_destruct(&client->buf);
            free(client);
            printf("Peer disconnected\n");
            return;
        } else {
            ringbuffer_consumed(&client->buf, nbytes);
            printf("Sent %zu bytes\n", nbytes);
            ev_io_stop(loop, watcher);
            ev_io_set(watcher, watcher->fd, EV_READ);
            ev_io_start(loop, watcher);
        }
    } else  {
        fail("got invalid event");
    }
}

Again, the whole application is stored on github at https://github.com/couchbaselabs/libev-couchbase-example. Let’s see it in action. To clone and build the applications, use the following commands:

~ $ git clone git://github.com/couchbaselabs/libev-couchbase-example.git
~ $ cd libev-couchbase-example/step1
step1 $ ./autogen.sh && ./configure && make

As a client for an echo application, we can build any telnet-like application. The following listing simulates two terminals by splitting them with a vertical line:

step 1 $ ./server                   |  step1 $ telnet localhost 4567
Listen on 4567                      |  Trying 127.0.0.1...
Successfully connected with client  |  Connected to localhost.
Received 14 bytes                   |  Escape character is '^]'.
Sent 14 bytes                       |  Hello world!
Received 9 bytes                    |  Hello world!
Sent 9 bytes                        |  foo bar
Peer disconnected                   |  foo bar
                                    |  ^]
                                    |  telnet> Connection closed.

Now that we described our application, let’s integrate Couchbase as the database using libcouchbase. The server.c file will have some small additions. In the main function, before initializing server socket, we will call storage_init and assign the connection information to the server structure variable.

server.handle = storage_init(loop, "localhost:8091", "default", null);

Also in the client_cb handler, each time libcouchbase receives data from the client, it will send the data to Couchbase server with storage_put where the_key is a string "example".

storage_put(client, the_key, val, nbytes);

Here, the initial application from step one is already using its own custom event loop to handle other IO in the application. The application could be more complex, but libcouchbase has a separate interface to inject your IO implementation into the library. In the simplest case, for example if your application is using one of the IO libraries supported by libcouchbase, you can just pass your event loop instance into the initializer, and libcouchbase won’t create new one. Instead it will register all its events on this external loop. In case your application is using its own implementation of a reactor pattern, such as nginx, or you use linux epoll API, you can easily write your own IO plugin and pass it to the connection initializer.

The echo server is using libev library, and libcouchbase provides a plugin to libev out-of-the-box. In this case we can imagine that it doesn’t already have it. In this case, we can copy the plugin from libcouchbase distribution into the directory step2/lcb-plugin/ and update build files appropriately.

Now we inspect the storage.c, where our storage_init() and storage_put() defined.

lcb_t storage_init(struct ev_loop *loop, const char *host, const char *bucket, const char *password)
{
    struct lcb_create_st opts;
    struct lcb_create_io_ops_st io_opts;
    lcb_t handle;
    lcb_error_t err;

    io_opts.version = 1;
    io_opts.v.v1.sofile = NULL;
    io_opts.v.v1.symbol = "lcb_create_libev_io_opts";
    io_opts.v.v1.cookie = loop;

    opts.version = 0;
    opts.v.v0.host = host;
    opts.v.v0.bucket = bucket;
    opts.v.v0.user = bucket;
    opts.v.v0.passwd = password;

    err = lcb_create_io_ops(&opts.v.v0.io, &io_opts);
    if (err != LCB_SUCCESS) {
        error_callback(NULL, err, "failed to create IO object");
        return NULL;
    }
    err = lcb_create(&handle, &opts);
    if (err != LCB_SUCCESS) {
        error_callback(NULL, err, "failed to create connection object");
        return NULL;
    }

    (void)lcb_set_error_callback(handle, error_callback);
    (void)lcb_set_store_callback(handle, storage_callback);

    err = lcb_connect(handle);
    if (err != LCB_SUCCESS) {
        error_callback(handle, err, "failed to connect to the server");
        return NULL;
    }

    return handle;
}

The end of the function may look familiar, but the io_opts structure is more interesting. It defines how to look up our custom IO plugin. The io_opts.v.v1.sofile is set to NULL to specify that our plugin compiled into the current executable image. In this case, symbol is the string name of the function with the following signature:

lcb_error_t lcb_create_libev_io_opts(int version, lcb_io_opt_t *io, void *loop);

This initializes the IO plugin and then the cookie will be passed to this function as the loop argument. You can refer to the lcb_create_io_ops(3) manpage for other ways to initialize IO subsystem.

The libcouchbase client is purely asynchronous; therefore, each data operation isplits into two parts: a function-scheduler, which validates and copies all arguments to internal buffers for further handing, and a function-callback, which will be called with the results of the operation. It is possible to implement a wrapper which will provide a more synchronous API, but it is difficult to write a library API that is both generic and efficient. Since all communication is done by using function arguments, it is easier to maintain backward compatible APIs by versioning both incoming structures and results. Let’s see how we did it in our sample:

void storage_put(struct client_s *client, const char *key,
                 const void *val, size_t nval)
{
    lcb_error_t err;
    lcb_store_cmd_t cmd;
    const lcb_store_cmd_t *cmds[] = { &cmd };

    memset(&cmd, 0, sizeof(cmd));
    cmd.version = 0;
    cmd.v.v0.key = key;
    cmd.v.v0.nkey = strlen(key);
    cmd.v.v0.bytes = val;
    cmd.v.v0.nbytes = nval;
    cmd.v.v0.operation = LCB_SET;
    err = lcb_store(client->handle, client, 1, cmds);
    if (err != LCB_SUCCESS) {
        error_callback(client->handle, err, "failed to schedule store operation");
    }
}

void storage_callback(lcb_t instance, const void *cookie,
                      lcb_storage_t operation, lcb_error_t error,
                      const lcb_store_resp_t *resp)
{
    struct client_s *client = (struct client_s *)cookie;
    struct ev_io* watcher = (struct ev_io*)client;
    char cas_str[128];
    ssize_t nb;

    nb = snprintf(cas_str, 128, "%"PRIu64"\n", resp->v.v0.cas);
    if (nb < 0) {
        fail("output CAS value");
    }
    ringbuffer_ensure_capacity(&client->out, nb);
    if (ringbuffer_write(&client->out, cas_str, nb) != nb) {
        fail("write CAS into the buffer");
    }
    ev_io_stop(client->loop, watcher);
    ev_io_set(watcher, watcher->fd, EV_WRITE);
    ev_io_start(client->loop, watcher);

    (void)operation;
    (void)resp;
}

Here, the storage_put() function is a wrapper over lcb_store(3). The storage_put() function translates arguments to the versioned structure and runs lcb_store to pass it to the library. The data won’t be sent immediately, but when the library connects to the data socket, it will be ready to accept data. After the server processes the request, the application will be notified asynchronously via storage_callback. This callback will copy CAS value to the output buffer to be sent to the client.

Time to demonstrate an application. This application is built the same as before, except you need to navigate to step2/ directory. As previously noted, the script will be split to demonstrate both server and the client:

step2 $ ./server                    |  step2 $ telnet localhost 4567
Listen on 4567                      |  Trying 127.0.0.1...
Successfully connected with client  |  Connected to localhost.
Received 14 bytes                   |  Escape character is '^]'.
Sent 20 bytes                       |  Hello world!
Peer disconnected                   |  2916447493390860288
                                    |  ^]
                                    |  telnet> Connection closed.

Now you can install cbc tools from package libcouchbase2-bin. If you installed libcouchbase from source, you most likely have it already. Then you can check that it created a key example and that the CAS values are matching:

step2 $ printf "%x\n" 2916447493390860288
28794d8b11a40000
step2 $ cbc cat example
"example" Size:14 Flags:0 CAS:28794d8b11a40000
Hello world!

Build an Application

You can use libcouchbase when you are building Couchbase-enabled application and want to bootstrap quickly. To do so you can build your application using the IO abstraction provided with libcouchbase.

In this example, we create a thin proxy application which will expose the memcached-compatible API for the system. Similar approaches exist for moxi server. Also this application will use less hard-coded logic and more look like real-life service. Full source code can be found at https://github.com/couchbaselabs/libcouchbase-proxy-sample.

For clarity, we skip parsing and initialization of the Couchbase connection from the main function. The proxy code begins in run_proxy, which is similar to the main function in the previous section. The important difference here is that we are using the IO abstraction from libcouchbase, to register events, and drive an event loop.

void run_proxy(lcb_t conn)
{
    lcb_io_opt_t io;
    struct sockaddr_in addr;
    int sock, rv;
    server_t server;

    io = opts.v.v0.io;
    info("starting proxy on port %d", port);
    sock = socket(PF_INET, SOCK_STREAM, 0);
    if (sock == -1) {
        fail("socket()", strerror(errno));
    }
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = INADDR_ANY;
    rv = bind(sock, (struct sockaddr *)&addr, sizeof(addr));
    if (rv == -1) {
        fail("bind()");
    }
    rv = listen(sock, 10);
    if (rv == -1) {
        fail("listen()");
    }
    server.conn = conn;
    server.io = io;
    server.event = io->v.v0.create_event(io);
    if (server.event == NULL) {
        fail("failed to create event for proxy");
    }
    io->v.v0.update_event(io, sock, server.event, LCB_READ_EVENT,
                          &server, proxy_accept_callback);
    lcb_set_error_callback(conn, error_callback);
    lcb_set_get_callback(conn, get_callback);
    lcb_set_store_callback(conn, store_callback);
    info("use ctrl-c to stop");
    io->v.v0.run_event_loop(io);
}

As in the previous example, we create a listening socket, and then we bind a handler to process read events. The handler is proxy_accept_callback, and it will be triggered for each new client. In turn it will create a new client structure, defined as follows:

typedef struct client_st client_t;
struct client_st {
    int id;
    server_t *server;
    int sock;
    ringbuffer_t in;
    ringbuffer_t out;
    void *event;
};

This initializes input and output buffers,and makes the socket descriptors non-blocking. At the end, the code will register the handler function proxy_client_callback for read events, and then wait for incoming data.

The proxy implements a very limited number of protocol commands: GET, SET, and VERSION. But other commands can be easily added. Here’s an example:

void proxy_client_callback(lcb_socket_t sock, short which, void *data)
{
    struct lcb_iovec_st iov[2];
    ssize_t rv;
    lcb_io_opt_t io;
    client_t *cl = data;

    io = cl->server->io;
    if (which & LCB_READ_EVENT) {
        for (;;) {
            /* read in chunks of BUFFER_SIZE */
            ringbuffer_ensure_capacity(&cl->in, BUFFER_SIZE);
            ringbuffer_get_iov(&cl->in, RINGBUFFER_WRITE, iov);
            rv = io->v.v0.recvv(io, cl->sock, iov, 2);
            if (rv == -1) {
                if (io->v.v0.error == EINTR) {
                    /* interrupted by signal */
                    continue;
                } else if (io->v.v0.error == EWOULDBLOCK) {
                    /* nothing to read right now */
                    io->v.v0.update_event(io, cl->sock, cl->event,
                                          LCB_WRITE_EVENT, cl,
                                          proxy_client_callback);
                    break;
                } else {
                    fail("read error");
                }
            } else if (rv == 0) {
                /* end of stream */
                io->v.v0.destroy_event(io, cl->event);
                ringbuffer_destruct(&cl->in);
                ringbuffer_destruct(&cl->out);
                close(cl->sock);
                info("[%d] disconnected", cl->id);
                free(cl);
                return;
            } else {
                ringbuffer_produced(&cl->in, rv);

                for (;;) {
                    protocol_binary_request_header req;
                    lcb_size_t nr, sz;
                    char *buf;

                    /* make sure the buffer is aligned */
                    if (ringbuffer_ensure_alignment(&cl->in) != 0) {
                        fail("cannot align the buffer");
                    }
                    /* take the packet header from the buffer */
                    nr = ringbuffer_peek(&cl->in, req.bytes, sizeof(req));
                    if (nr < sizeof(req)) {
                        break;
                    }
                    /* make sure the buffer has whole the body */
                    sz = ntohl(req.request.bodylen) + sizeof(req);
                    if (cl->in.nbytes < sz) {
                        break;
                    }
                    /* copy packet into intermediate buffer */
                    buf = malloc(sizeof(char) * sz);
                    if (buf == NULL) {
                        fail("cannot allocate buffer for packet");
                    }
                    nr = ringbuffer_read(&cl->in, buf, sz);
                    if (nr < sizeof(req)) {
                        fail("input buffer doesn't contain enough data");
                    }
                    /* handle packet and deallocate the intermediate
                     * buffer */
                    handle_packet(cl, buf);
                    free(buf);
                }
            }
        }
    }
    if (which & LCB_WRITE_EVENT) {
        /* check if we have something to send */
        ringbuffer_get_iov(&cl->out, RINGBUFFER_READ, iov);
        if (iov[0].iov_len + iov[1].iov_len == 0) {
            io->v.v0.delete_event(io, cl->sock, cl->event);
            return;
        }
        rv = io->v.v0.sendv(io, cl->sock, iov, 2);
        if (rv < 0) {
            fail("write error");
        } else if (rv == 0) {
            io->v.v0.destroy_event(io, cl->event);
            ringbuffer_destruct(&cl->in);
            ringbuffer_destruct(&cl->out);
            close(cl->sock);
            info("[%d] disconnected", cl->id);
            free(cl);
            return;
        } else {
            ringbuffer_consumed(&cl->out, rv);
            io->v.v0.update_event(io, cl->sock, cl->event,
                                  LCB_READ_EVENT, cl,
                                  proxy_client_callback);
        }
    }
    (void)sock;
}

The function proxy_client_callback logically divided into two parts.

  • The first part occurs when the event loop notifies us that the socket is ready for non-blocking reading. At this point, the function tries to read all the data available on the socket. It finds the packets and passes aligned byte arrays to handle_packet. Once the code gets a special code from the IO subsystem get from the IO subsystem special code, that it cannot read anything without blocking, it stops processing and registers itself for write events, because in nearest future, responses will appear in the output buffer.

  • The second part of the function reacts to write events and is much simpler. It just checks to see if there is data in the output buffer. If there is data in the output buffer, it will try to send that data to the client. If not, the functionwill unregister itself from write events to save CPU cycles.

The function handle_packet is a good place to start playing with this example if you’d like to add new features there. It decodes protocol packet and translates it into libcouchbase calls.

void handle_packet(client_t *cl, char *buf)
{
    protocol_binary_request_header *req = (void *)buf;
    protocol_binary_response_header res;
    union {
        lcb_get_cmd_t get;
        lcb_store_cmd_t set;
    } cmd;
    union {
        const lcb_get_cmd_t *get[1];
        const lcb_store_cmd_t *set[1];
    } cmds;
    cookie_t *cookie;

    cookie = malloc(sizeof(cookie_t));
    if (cookie == NULL) {
        fail("cannot allocate buffer for command cookie");
    }
    cookie->client = cl;
    cookie->opaque = req->request.opaque;
    cookie->opcode = req->request.opcode;
    memset(&cmd, 0, sizeof(cmd));
    switch (req->request.opcode) {
    case PROTOCOL_BINARY_CMD_GET:
        cmds.get[0] = &cmd.get;
        cmd.get.v.v0.nkey = ntohs(req->request.keylen);
        cmd.get.v.v0.key = buf + sizeof(*req);
        info("[%d] get \"%.*s\"", cl->id,
             (int)cmd.get.v.v0.nkey, (char *)cmd.get.v.v0.key);
        lcb_get(cl->server->conn, (const void*)cookie, 1, cmds.get);
        break;
    case PROTOCOL_BINARY_CMD_SET:
        cmds.set[0] = &cmd.set;
        cmd.set.v.v0.operation = LCB_SET;
        cmd.set.v.v0.nkey = ntohs(req->request.keylen);
        cmd.set.v.v0.key = buf + sizeof(*req) + 8;
        cmd.set.v.v0.nbytes = ntohl(req->request.bodylen) - 8 - ntohs(req->request.keylen);
        cmd.set.v.v0.bytes = buf + sizeof(*req) + 8 + ntohs(req->request.keylen);
        cmd.set.v.v0.cas = req->request.cas;
        cmd.set.v.v0.datatype = req->request.datatype;
        memcpy(&cmd.set.v.v0.flags, buf + sizeof(*req), 4);
        cmd.set.v.v0.flags = ntohl(cmd.set.v.v0.flags);
        memcpy(&cmd.set.v.v0.exptime, buf + sizeof(*req) + 4, 4);
        cmd.set.v.v0.exptime = ntohl(cmd.set.v.v0.exptime);
        info("[%d] set \"%.*s\"", cl->id,
             (int)cmd.set.v.v0.nkey, (char *)cmd.set.v.v0.key);
        lcb_store(cl->server->conn, (const void*)cookie, 1, cmds.set);
        break;
    case PROTOCOL_BINARY_CMD_VERSION:
        free(cookie);
        info("[%d] version", cl->id);
        res.response.magic = PROTOCOL_BINARY_RES;
        res.response.opcode = req->request.opcode;
        res.response.keylen = 0;
        res.response.extlen = 0;
        res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
        res.response.status = PROTOCOL_BINARY_RESPONSE_SUCCESS;
        res.response.bodylen = htonl(sizeof(version_msg));
        res.response.opaque = req->request.opaque;
        res.response.cas = 0;
        ringbuffer_ensure_capacity(&cl->out, sizeof(res));
        ringbuffer_write(&cl->out, res.bytes, sizeof(res));
        ringbuffer_write(&cl->out, version_msg, sizeof(version_msg));
        break;
    default:
        free(cookie);
        info("[%d] unsupported command: 0x%02x", cl->id, req->request.opcode);
        res.response.magic = PROTOCOL_BINARY_RES;
        res.response.opcode = req->request.opcode;
        res.response.keylen = 0;
        res.response.extlen = 0;
        res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
        res.response.status = PROTOCOL_BINARY_RESPONSE_NOT_SUPPORTED;
        res.response.bodylen = htonl(sizeof(notsup_msg));
        res.response.opaque = req->request.opaque;
        res.response.cas = 0;
        ringbuffer_ensure_capacity(&cl->out, sizeof(notsup_msg) + sizeof(res));
        ringbuffer_write(&cl->out, res.bytes, sizeof(res));
        ringbuffer_write(&cl->out, notsup_msg, sizeof(notsup_msg));
    }
}

To maintain a logical link between requests and responses for the clients, the protocol defines a special field opaque, which can be a special tag or sequence number. To preserve this field in the response, we will put it into a cookie_t structure. During the proxy initialization, we defined two callback-functions for libcouchbase:

lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);

In this kind of application, these functions just build a protocol response from libcouchbase return value. They also register the proxy_client_callback on write events to send out data recently written into the buffer.

void get_callback(lcb_t conn, const void *cookie, lcb_error_t err,
                  const lcb_get_resp_t *item)
{
    cookie_t *c = (cookie_t *)cookie;
    client_t *cl = c->client;
    lcb_io_opt_t io = cl->server->io;
    protocol_binary_response_get res;

    res.message.header.response.magic = PROTOCOL_BINARY_RES;
    res.message.header.response.opcode = PROTOCOL_BINARY_CMD_GET;
    res.message.header.response.keylen = 0;
    res.message.header.response.extlen = 4;
    res.message.header.response.datatype = item->v.v0.datatype;
    res.message.header.response.status = htons(map_status(err));
    res.message.header.response.bodylen = htonl(4 + item->v.v0.nbytes);
    res.message.header.response.opaque = c->opaque;
    res.message.header.response.cas = item->v.v0.cas;
    res.message.body.flags = htonl(item->v.v0.flags);
    ringbuffer_ensure_capacity(&cl->out, sizeof(res.bytes) + item->v.v0.nbytes);
    ringbuffer_write(&cl->out, res.bytes, sizeof(res.bytes));
    ringbuffer_write(&cl->out, item->v.v0.bytes, item->v.v0.nbytes);
    io->v.v0.update_event(io, cl->sock, cl->event, LCB_WRITE_EVENT,
                          cl, proxy_client_callback);
    free(c);
    (void)conn;
}

These are viritually all the integration points with libcouchbase. As we demonstrated, libcouchbase more than a protocol parser; it also an abstract and portable IO framework, allowing you to build asynchronous applications around Couchbase Server.

To conclude, let’s run your favourite memcached client and verify our proxy. First you need to clone and build it:

$ git clone git://github.com/couchbaselabs/libcouchbase-proxy-sample.git
$ cd libcouchbase-proxy-sample
libcouchbase-proxy-sample $ make

Run the server and send couple of commands to it. We will use the dalli ruby gem here:

libcouchbase-proxy-sample $ ./proxy                 |  $ irb
connecting to bucket "default" at "localhost:8091"  |  irb:001:0> require 'dalli'
starting proxy on port 1987                         |  true
use ctrl-c to stop                                  |  irb:002:0> m = Dalli::Client.new  "localhost:1987"
[1] connected                                       |  #<Dalli::Client:0x007f716321ce20 @servers=["localhost:1987"], @options={}, @ring=nil>
[1] version                                         |  irb:003:0> m.set("foo", "bar")
[1] set "foo"                                       |  true
[1] get "foo"                                       |  irb:004:0> m.get("foo")
[1] disconnected                                    |  "bar"

Release Notes

The following sections provide release notes for individual release versions of the C Couchbase Client Library. To browse or submit new issues, see Couchbase Client Library C Issues Tracker.

Release Notes for Couchbase Client Library C 2.3.2 GA (01 July 2014)

Fixes in 2.3.1

  • Fixed a bug where randomizing host lists during the initial connect would skip the first node in the list.

    Issues: CCBC-433

  • Assign the cccp_cookie to the provider. This fixes an issue where a configuration request over CCCP to an already-connected node would not be received, potentially causing delays in the client receiving the newest cluster topology.

    Issues: CCBC-414

  • Fixed a bug where stale commands (and their cookies) would be confused with newer responses. This issue caused segmentation faults when requesting a new configuration on a connected node via CCCP because the CCCP cookie format was being overlaid on a different area of memory. This bug also manifested itself during multi-get responses where some items were not found.

    Issues: CCBC-435

  • Don’t attempt to remap keys to nodes without any vBuckets. When a NOT_MY_VBUCKET error is received, the behavior of the library is to attempt to map the key to another node in the cluster. It determines the node by the forward map, and if that doesn’t exist it selects a random node in the cluster. Sometimes a node will exist within the cluster map only because it is in an ‘eject wait’ state where the node has already been ejected from the cluster and is just returning NOT_MY_VBUCKET for all new requests. The issue arises when the library attempts to connect to this node. In this case, the node may either not respond to the new connection or it may respond with an authentication failure (because the node is really no longer part of the cluster). This new version fixes this issue by only remapping items to nodes that actually contain at least one vBucket.

    Issues: CCBC-420

Release Notes for Couchbase Client Library C 2.3.1 GA (09 May 2014)

New Features and Behavior Changes in 2.3.1

  • Add lcb_cntl() interface for configuration cache Configuration cache options can be set after instantiation using lcb_cntl() with the new LCB_CNTL_CONFIGCACHE operation. The old-style lcb_create_compat creation path is deprecated.

    Issues: CCBC-395

  • Compare configuration revision information for memcached cluster bootstrap. Previously we would refresh the configuration upon receipt of any new configuration update from memcached. This is fixed in 2.3.1 where the configuration is applied only if it is deemed to be newer than the current configuration. With memcached bootstrap this is true only if the configuration’s rev field is higher than the current one.

    Issues: CCBC-332 CCBC-364

Fixes in 2.3.1

  • Segfault in connmgr_invoke_request Occasionally a segmentation fault would happen when a connection was being released as a result of a connection failure. This was because of invalid list tracking.

    Issues: CCBC-404

  • Get-with-replica occasionally crashes on Windows and UV during topology changes. This was due to not allocating a buffer if one did not exist.

    Issues CCBC-394

  • Application binary interface (ABI) compatibility broken between 2.x and 2.3 for lcb_create_compat. This has been fixed by symbol aliasing between versions. Use the lcb_cntl() API to set the configuration cache, as specified in CCBC-395.

    Issues: CCBC-392

  • Failed assertion on get-with-replica when connection fails. If a connection fails with a CMD_GET_REPLICA command still in the queue an assertion failure will crash the library. This has been fixed by handling the opcode in the failout_single_request function.

    Issues: CCBC-385

  • Unknown Winsock error codes crash application. This was fixed by providing proper handlers for Winsock codes that were not explicitly converted into their POSIX equivalents.

    Issues: CCBC-384

  • Fix memory leak in configuration parsing. A leak was introduced in version 2.3.0 by not freeing the JSON pool structure. This has been fixed in 2.3.1

    Issues: CCBC-376

  • lcb_get_host and lcb_get_port might return host-port combinations from different servers. If multiple servers are listening on different ports this might result in yielding an invalid endpoint by combining the output from those two functions. This has been fixed in 2.3.1 by returning the host and port from the first host in lieu of a currently-connected REST endpoint.

    Issues: CCBC-370

  • Initial bootstrapping failure may mask LCB_BUCKET_ENOENT calls with LCB_ETIMEDOUT. This has been fixed by not retrying configuration retrieval if an explicit HTTP 404 code is received. Note that when using bootstrap over memcached, a missing bucket might still be manifest as LCB_AUTH_ERROR.

    Issues: CCBC-368

  • Ensure lcb_get_host does not return NULL when the associated lcb_t is of LCB_TYPE_CLUSTER. This would cause crashes in some applications that relied on this function to not return NULL.

    Issues: CCBC-367

  • Fixed spurious timeouts being delivered in asynchronous use cases. In applications that do not use lcb_wait() the library will potentially time out commands internally triggering an erroneous configuration refresh. While this issue would not end up failing operations it will cause unnecessary network traffic for retrieving configurations. Applications using lcb_wait() are not affected because that function resets the timeout handler.

    Issues: CCBC-389

Release Notes for Couchbase Client Library C 2.3.0 GA (07 April 2014)

New Features and Behavior Changes in 2.3.0

  • Cluster Configuration Carrier Publication) (CCCP)

    CCCP is the new and more efficient way to bootstrap from a cluster using the native memcached protocol and is quicker than the previous HTTP bootstrap mechanism, dramatically improving startup times and reducing load on the server. This feature is available in Couchbase Server version 2.5 and later. The existing HTTP configuration is still supported and is employed as a fallback in the event that CCCP is not supported.

    In conjunction with this, a new struct version has been added to the lcb_create_st parameters structure for use with lcb_create. The new struct allows you to get more control over how the client is initialized. The following example configures a client to always use the CCCP protocol and never attempt to fall back on HTTP:

      lcb_t instance;
      struct lcb_create_st options;
      lcb_config_transport_t enabled_transports[] = {
      LCB_CONFIG_TRANSPORT_CCCP,
      LCB_CONFIG_TRANSPORT_LIST_END
      };
    
      memset(&options, 0, sizeof(options));
      options.version = 2;
      options.v.v2.mchosts = "example.com:11210";
      options.v.v2.transports = enabled_transports;
    
      lcb_error_t rc = lcb_create(&instance, &options);
      if (rc != LCB_SUCCESS) {
      fprintf(stderr, "Failed to create instance: %s\n", lcb_strerror(instance, rc));
      }
    

    The CCCP implementation required a significant rewrite in how sockets were created and re-used. Particularly, a connection pooling feature was implemented.

    The cbc command now has an additional -C option that accepts the preferred configuration mechanism to use.

    Issues: CCBC-234

  • Logging hooks

    This improvements adds various levels of diagnostic logging with the library itself. It may be utilized via the environment (by setting the LCB_LOGLEVEL environment variable to a positive integer – the higher the number the more verbose the logging).

    Integrators may also use the logging API specified in <libcouchbase/types.h> to proxy the library’s logging messages into your own application.

    Current events logged include connection initialization, destruction, connection pool management, configuration changes, and timeouts.

    By default the library is silent.

    Issues: CCBC-305

  • Per-node bootstrap/config timeouts

    This change provides fine-grained control over how long to wait per-node to receive updated configuration information. This setting helps adjust the initial and subsequent bootstrap processes to help ensure each node gets a slice of time.

    Issues: CCBC-316

  • Master-only observe option

    This adds a new struct version to the lcb_observe_cmd_t which allows you to select only the master node. You can use this to efficiently check whether the key exists (without retrieving it). It also allows you to get the item’s CAS without fetching it.

    Issues: CCBC-152

Fixes in 2.3.0

  • Handle spurious EWOULDBLOCK on UV/Win32 This issue caused odd errors on Windows when large amounts of data would be received on the socket.

    Issues: CCBC-297

  • Correctly parse "", "0" and "1" for environment variables. In previous versions having the entry set to an empty string or 0 would still be treated by the library as a true value for various environment variables. This has been fixed so that clear “False” values such as the empty string or 0 are treated as such.

    Issues: CCBC-340

  • Fix spurious callback deliveries on failed multi-packet commands. Multi-packet commands will no longer deliver spurious callbacks on failure. Previously these commands would be relocated to the same server during a configuration change, resulting in multiple callbacks for the same command. In this case the client would think all the commands had been completed, and when the next response arrived it would incorrectly map it to a different request.

    Issues: CCBC-150 CCBC-321 CCBC-281

  • Fix assertion upon receiving a slow HTTP streaming update which does not receive a new topology. In such cases a user would receive an error like this: src/bconf_io.c:175: instance_timeout_handler: Assertion instance->confstatus != LCB_CONFSTATE_CONFIGURED' failed..

    Issues: CCBC-320

Known Issues in 2.3.0

  • SASL password stored in plaintext with configuration cache. This should be stripped from the file before it is written to disk

    Issues: CCBC-347

  • Memcached buckets may receive operation failures during rebalance. The library should attempt to rehash the key and forward the item to the appropriate server

    Issues: CCBC-331

  • New rev field in configuration not used. Ideally the library should use this to compare the validity of two configurations.

    Issues: CCBC-332

  • Users of the lcb_compat_st structure will need to recompile their application to work with the new library. This is because the embedded lcb_create_st structure has change. This affects any application using the configuration cache as it is created using said structure.

    Issues: CCBC-392

Release Notes for Couchbase Client Library C 2.2.0 GA (06 November 2013)

New Features and Behavior Changes in 2.2.0

  • Handle 302 redirects in HTTP (views and administrative requests). By default the library follows up to three redirects. After the limit is reached, the request is terminated with code LCB_TOO_MANY_REDIRECTS. The limit is configurable through LCB_CNTL_MAX_REDIRECTS. If it is set to -1, it disables the redirect limit. The following example shows how to set the limit:

      int new_value = 5;
      lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_MAX_REDIRECTS, &new_value);
    

    Issues: CCBC-169

  • Replace isasl with cbsasl. cbsasl implements both PLAIN and CRAM-MD5 authentication mechanisms.

  • LCB_CNTL_MEMDNODE_INFO command updated to include effective SASL mechanism:

      cb_cntl_server_t node;
      node.version = 1;
      node.v.v1.index = 0; /* first node */
      lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_MEMDNODE_INFO, &node);
      if (node.v.v1.sasl_mech) {
          printf("authenticated via SASL '%s'\n",
                 node.v.v1.sasl_mech);
      }
    
  • You can force a specific authentication mechanism for the connection handle by using the LCB_CNTL_FORCE_SASL_MECH command:

      lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_FORCE_SASL_MECH, "PLAIN");
    

    Issues: CCBC-243

  • Stricter, more inspectable behavior for the configuration cache. This provides a test and an additional lcb_cntl operation to check the status of the configuration cache. Also, it switches off the configuration cache with memcached buckets.

      int is_loaded;
      lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
      if (is_loaded) {
          printf("Configuration cache saved us a trip to the config server\n");
      } else {
          printf("We had to contact the configuration server for some reason\n");
      }
    

    Issues: CCBC-204 CCBC-205

Fixes in 2.2.0

  • libuv plugin: use the same CRT for free and malloc.

    Issues : CCBC-286

  • Fail NOT_MY_VBUCKET responses on time-out.

    Issues : CCBC-288

  • Do a full purge when negotiation times out. In this case we must purge the server from all commands and not simply pop individual items.

    Issues : CCBC-275

  • Reset the server’s buffers upon reconnection. This fixes a crash experienced when requesting a new read with the previous buffer still intact. This was exposed by calling lcb_failout_server on a time-out error while maintaining the same server struct.

    Issues : CCBC-275

  • Make server buffers reentrant-safe. When purging implicit commands, we invoke callbacks that might in turn cause other LCB entry points to be invoked, which can shift the contents or positions of the ring buffers that we’re reading from.

    Issues : CCBC-282

  • Use common config retry mechanism for bad configuration cache. This uses the same error handling mechanism as when a bad configuration has been received from the network. New LCB_CONFIG_CACHE_INVALID error code to notify the user of such a situation

    Issues : CCBC-278

  • Handle getl and unl when purging the server (thanks Robert Groenenberg).

    Issues : CCBC-274

  • Don’t fail out all commands on a time-out. Only fail those commands that are old enough to have timed out already.

  • Don’t record and use TTP/TTR from observe. Just poll at a fixed interval, because the responses from the server side can be unreliable.

    Issues : CCBC-269

  • Allow hooks for mapping server codes to errors. This also helps handle sane behavior if a new error code is introduced, or allow user-defined logging when a specific error code is received.

      lcb_errmap_callback default_callback;
    
      lcb_error_t user_map_error(lcb_t instance, lcb_uint16_t in)
      {
        if (in == PROTOCOL_BINARY_RESPONSE_ETMPFAIL) {
          fprintf(stderr, "temporary failure on server\n");
        }
        return default_callback(instance, in);
      }
    
      ...
    
      default_callback = lcb_set_errmap_callback(conn, user_map_error);
    
  • Add an example of a connection pool. See example/instancepool directory

  • Force lcb_wait return result of wait operation instead of lcb_get_last_error. It returns last_error if and only if the handle is not yet configured

    Issues : CCBC-279

  • cbc-pillowfight: compute item size correctly during set If minSize and maxSize are set to the same value it can sometimes crash since it may try to read out of memory bounds from the allocated data buffer.

    Issues : CCBC-284

  • Apply key prefix CLI option in cbc-pillowfight

    Issues : CCBC-283

  • Add --enable-maintainer-mode. Maintainer mode enables --enable-werror --enable-warnings --enable-debug, forces all plugins to be installed and forces all tests, tools, and examples to be built

  • Expose LCB_MAX_ERROR to allow user-defined codes

    Issues : CCBC-255

Release Notes for Couchbase Client Library C 2.1.3 GA (10 September 2013)

New Features and Behavior Changes in 2.1.3

  • Use cluster type connection for cbc-bucket-flush. Although flush command is accessible for bucket type connections, cbc-bucket-flush doesn’t use provided bucket name to connect to, therefore it will fail if the bucket name isn’t “default”.

  • Allow to make connect order deterministic. It allows the user to toggle between deterministic and random connect order for the supplied nodes list. By default it will randomize the list.

Fixes in 2.1.3

  • Updated gtest to version 1.7.0. Fixes issue with building test suite with new XCode 5.0 version being released later this month.

  • Do not try to parse config for LCB_TYPE_CLUSTER handles. It fixes timouts for management operations (like ‘cbc bucket-create’, ‘cbc bucket-flush’, ‘cbc bucket-delete’ and ‘cbc admin’)

    Issues : CCBC-265

  • Skip unfinished SASL commands on rebalance. During rebalance, it is possible that the newly added server doesn’t have chance to finish SASL auth before the cluster will push config update, in this case packet relocator messing cookies. Also the patch makes sure that SASL command/cookie isn’t mixing with other commands

    Issues : CCBC-263

  • Do not allow to use Administrator account for LCB_TYPE_BUCKET

  • Fig segmentation faults during tests load of node.js. Sets inside_handler on socket_connected. Previously we were always using SASL auth, and as such, we wouldn’t flush packets from the cmd_log using server_send_packets (which calls apply_want). apply_want shouldn’t be called more than once per event loop entry – so this sets and unsets the inside_handler flag.

    Issues : CCBC-258

  • Added support of libuv 0.8

  • Close config connection before trying next node. It will fix asserts in case of the config node becomes unresponsive, and the threshold controlled by LCB_CNTL_CONFERRTHRESH and lcb_cntl(3)

Release Notes for Couchbase Client Library C 2.1.2 GA (27 August 2013)

Fixes in 2.1.2

  • Use bucket name in SASL if username omitted. Without this fix, you can may encounter a segmentation faults for buckets, which are not protected by a password.

    Issues : CCBC-253 CCBC-254

  • Preserve IO cookie in options_from_info when using v0 plugins with user-provided IO loop instance. This issue was introduced in 2.1.0.

  • Display the effective IO backend in cbc-version. This is helpful to quickly detect what is the effective IO plugin on a given system.

Release Notes for Couchbase Client Library C 2.1.1 GA (22 August 2013)

New Features and Behavior Changes in 2.1.1

  • Fallback to ‘select’ IO plugin if default plugin cannot be loaded. On UNIX-like systems, default IO backend is ‘libevent’, which uses third-party library might be not available at the run-time. Read in lcb_cntl(3couchbase) man page in section LCB_CNTL_IOPS_DEFAULT_TYPES about how to determine effective IO plugin, when your code chose to use LCB_IO_OPS_DEFAULT during connection instantiation. The fallback mode doesn’t affect application which specify IO backend explicitly.

    Issues : CCBC-246

  • Skip misconfigured nodes in the list. New lcb_cntl(3couchbase) added to control whether the library will skip nodes in initial node list, which listen on configuration port (8091 usually) but doesn’t meet required parameters (invalid authentication or missing bucket). By default report this issue and stop trying nodes from the list, like all previous release. Read more at man page lcb_cntl(3couchbase) in section LCB_CNTL_SKIP_CONFIGURATION_ERRORS_ON_CONNECT

    Issues : CCBC-192

  • Distribute debug information with release binaries on Windows

    Issues : CCBC-245

Fixes in 2.1.1

  • Do not use socket after failout. Fixes segmentation faults during rebalance.

    Issues : CCBC-239

  • Use provided credentials for authenticating to the data nodes. With this fix, it is no longer possible to use Administrator credentials with a bucket. If your configuration does so, you must change the credentials you use before applying this update. No documentation guides use of Administrator credentials, so this change is not expected to affect few, if any deployments.

  • Do not disable config.h on UNIX-like platforms. It fixes build issue, when application is trying to include plugins from the tarball.

    Issues : CCBC-248

Release Notes for Couchbase Client Library C 2.1.0 GA (18 August 2013)

New Features and Behavior Changes in 2.1.0

  • New backend select. This backend is based on the select(2) system call and its Windows version. It could be considered the most portable solution and is available with the libcouchbase core.

  • API for durability operations. This new API is based on lcb_observe(3) and allows you to monitor keys more easily. See the man pages lcb_durability_poll(3) and lcb_set_durability_callback(3) for more info.

    Issues : CCBC-145

  • New backend libuv. This backend previously was part of the couchnode project and is now available as a plugin. Because libuv doesn’t ship binary packages there is no binary package libcouchbase2-libuv. You can build plugin from the source distribution, or through the libcouchbase-dev or libcouchbase-devel package on UNIX like systems.

    Issues : CCBC-236

  • New backend iocp. This is a Windows specific backend, which uses “I/O Completion Ports”. As a part of the change, a new version of plugin API was introduced which is more optimized to this model of asynchronous IO.

  • New configuration interface lcb_cntl(3) along with new tunable options of the library and connection instances. In this release the following settings are available. See the man page for more information and examples.:

    • LCB_CNTL_OP_TIMEOUT operation timeout (default 2.5 seconds)

    • LCB_CNTL_CONFIGURATION_TIMEOUT time to fetch cluster configuration. This is similar to a connection timeout (default 5 seconds)

    • LCB_CNTL_VIEW_TIMEOUT timeout for couchbase views (default 75 seconds)

    • LCB_CNTL_HTTP_TIMEOUT timeout for other HTTP operations like RESTful flush, bucket creating etc. (default 75 seconds)

    • LCB_CNTL_RBUFSIZE size of the internal read buffer (default 32768 bytes)

    • LCB_CNTL_WBUFSIZE size of the internal write buffer (default 32768 bytes)

    • LCB_CNTL_HANDLETYPE type of the lcb\_t handler (readonly)

    • LCB_CNTL_VBCONFIG returns pointer to VBUCKET_CONFIG_HANDLE (readonly)

    • LCB_CNTL_IOPS get the implementation of IO (lcb_io_opt_t)

    • LCB_CNTL_VBMAP get vBucket ID for a given key

    • LCB_CNTL_MEMDNODE_INFO get memcached node info

    • LCB_CNTL_CONFIGNODE_INFO get config node info

    • LCB_CNTL_SYNCMODE control synchronous behavior (default LCB_ASYNCHRONOUS)

    • LCB_CNTL_IP6POLICY specify IPv4/IPv6 policy (default LCB_IPV6_DISABLED)

    • LCB_CNTL_CONFERRTHRESH control configuration error threshold (default 100)

    • LCB_CNTL_DURABILITY_TIMEOUT durability timeout (default 5 seconds)

    • LCB_CNTL_DURABILITY_INTERVAL durability polling interval (default 100 milliseconds)

    • LCB_CNTL_IOPS_DEFAULT_TYPES get the default IO types

    • LCB_CNTL_IOPS_DLOPEN_DEBUG control verbose printing of dynamic loading of IO plugins.

Fixes in 2.1.0

  • Fixed bug when REPLICA_SELECT didn’t invoke callbacks for negative error codes

    Issues : CCBC-228

  • Fixed bug when LCB_REPLICA_FIRST fails if first try does not return key

    Issues : CCBC-229

Known Issues in 2.1.0

  • From the release the 2.1.0 package libcouchbase2 will not install an IO backend automatically. If you are upgrading, there are no changes because you have already libcouchbase2-libev or libcouchbase2-libevent packages installed. For new installations, a backend must be selected for the client library to work correctly.

    If for example you are using the PHP SDK, the old way, which works for pre-2.1.0 versions is:

    # DEB-based systems
     shell> sudo apt-get install libcouchbase2 libcouchbase-dev
     # RPM-based systems
     shell> sudo yum install libcouchbase2 libcouchbase-devel
    

    But a more explicit way to do this, which works for all versions (including 2.1.0) is:

    # DEB-based systems
      shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
      # RPM-based systems
      shell> sudo yum install libcouchbase2-libevent libcouchbase-devel
    

Release Notes for Couchbase Client Library C 2.0.7 GA (10 July 2013)

New Features and Behavior Changes in 2.0.7

  • Improve lcb\_get\_replica(). Now it is possible to choose between three strategies:

    1. LCB_REPLICA_FIRST : Previously accessible and now the default, the caller will get a reply from the first replica to successfully reply within the timeout for the operation or will receive an error.

    2. LCB_REPLICA_ALL : Ask all replicas to send documents/items back.

    3. LCB_REPLICA_SELECT : Select one replica by the index in the configuration starting from zero. This approach can more quickly receive all possible replies for a given topology, but it can also generate false negatives.

    Note that applications should not assume the order of the replicas indicates more recent data is at a lower index number. It is up to the application to determine which version of a document/item it may wish to use in the case of retrieving data from a replica.

    Issues : CCBC-183

Release Notes for Couchbase Client Library C 2.0.6 GA (07 May 2013)

New Features and Behavior Changes in 2.0.6

Fixes in 2.0.6

  • Fix segfault when rebalancing. When a (!connected) server is reconnected, the tasks in its “pending” buffer will be moved into “output” buffer. If its connection is broken again immediately, relocate_packets() will go to wrong path.

    Issues : CCBC-188

  • Don’t try to switch to backup nodes when timeout is reached

    Issues : CCBC-202

  • Fix compile error with sun studio. "src/event.c", line 172: error: statement not reached (E_STATEMENT_NOT_REACHED)

  • Don’t invoke HTTP callbacks after cancellation, because user code might assume a previously-freed resource is still valid

  • Check if SASL struct is valid before disposing

    Issues : CCBC-188

  • example/yajl/couchview.c: pass cookie to the command Fixes coredump when executing./examples/yajl/couchview

Release Notes for Couchbase Client Library C 2.0.5 GA (05 April 2013)

New Features and Behavior Changes in 2.0.5

  • pillowfight example updated to optionally use threads

Fixes in 2.0.5

Release Notes for Couchbase Client Library C 2.0.4 GA (06 March 2013)

Fixes in 2.0.4

  • Build error on solaris/sparc: -Werror=cast-align

    Issues : CCBC-178

  • Fixed illegal memory access in win32 plugin

    Issues : CCBC-147

  • Work properly on systems where EWOULDBLOCK != EAGAIN

    Issues : CCBC-175

  • The library stops iterating backup nodes list if the next one isn’t accessible.

    Issues : CCBC-182

  • The bootstrap URI is not parsed correctly

    Issues : CCBC-185

  • Segmentation fault when the hostname resolved into several addresses and first of them reject couchbase connections.

    Issues : CCBC-180

Release Notes for Couchbase Client Library C 2.0.3 GA (06 February 2013)

New Features and Behavior Changes in 2.0.3

  • Add a new library: libcouchbase_debug.so (see include/libcouchbase/debug.h) which is a new library that contains new debug functionality.

  • Added manual pages for the library.

Fixes in 2.0.3

  • Observe malfunctions in the case of multiple keys and server failure.

    Issues : CCBC-155

  • Reset internal state on lcb_connect(). Allow caller to use lcb_connect() multiple times to implement reconnecting using the same lcb_t instance. Also it sets up the initial-connection timer for users who don’t use lcb_wait() and drive IO loop manually.

    Issues : CCBC-153

  • Invalid read in libevent plugin, when the plugin compiled in 1.x mode

    Issues : CCBC-171

  • Shrink internal lookup tables (and reduce the size of lcb_t)

  • Issues : CCBC-156

Release Notes for Couchbase Client Library C 2.0.2 GA (04 January 2013)

Fixes in 2.0.2

  • Document LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH. Enhance the lcb_strerror test to detect undocumented error codes.

  • Commands sent to multiple servers fail to detect the respose if mixed with other commands.

    Issues : CCBC-150

  • Under high load the library could generate LCB_ETIMEDOUT errors without reason owing to internal limitations.

    Issues : CCBC-153

  • Cancellation of the HTTP request might lead to memory leaks or to segfaults (2e3875c2).

    Issues : CCBC-151

Release Notes for Couchbase Client Library C 2.0.1 GA (11 December 2012)

New Features and Behavior Changes in 2.0.1

  • SystemTap and DTrace integration

Fixes in 2.0.1

  • Fix a memory leak on the use of http headers

    Issues : CCBC-130

  • libev-plugin: delay all timers while the loop isn’t active. It will fix LCB_ETIMEOUT in the following scenario:

    • connect the instance

    • sleep for time greater than default timeout (e.g. 3 seconds)

    • schedule and execute a command (it will be timed out immediately)

  • Do not abort when purging SASL commands

    Issues : CCBC-136

  • Fix possible SEGFAULT. Not-periodic timers are destroyed after calling user’s callback, after that library performed read from freed pointer.

  • Compensate for cluster nodes lacking couchApiBase

    Issues : CCBC-131

  • Ensure HTTP works even when the network may be unreliable. This changeset encompasses several issues which had been found with HTTP requests during network errors and configuration changes. Specifically some duplicate code paths were removed, and the process for delivering an HTTP response back to the user is more streamlined.

    Issues : CCBC-132, CCBC-133

  • libev-plugin: reset IO event on delete. We need to reset it, because it might be re-used later

  • Make library C89 friendly again

Release Notes for Couchbase Client Library C 2.0.0 GA (27 November 2012)

New Features and Behavior Changes in 2.0.0

  • Add the CAS to the delete callback

Fixes in 2.0.0

  • Minor update of the packaging layout:

    • libcouchbase-all package comes without version

    • extract debug symbols from libcouchbase-{bin,core} to libcouchbase-dbg package

  • Install unlock callback in synchronous mode

Release Notes for Couchbase Client Library C 2.0.0beta3 Beta (21 November 2012)

New Features and Behavior Changes in 2.0.0beta3

  • Try all known plugins for LCB_IO_OPS_DEFAULT in run time

  • Allow to use ‘cbc-hash’ with files

  • Create man pages for cbc and cbcrc

  • Use dynamic versioning for plugins

  • Lookup the plugin symbol also in the current executable image.

    Issues : CCBC-114

  • Allow the user to specify a different hash key. All of the data operations contains a hashkey and nhashkey field. This allows you to “group” items together in your cluster. A typical use case for this is if you’re storing lets say data for a single user in multiple objects. If you want to ensure that either all or none of the objects are available if a server goes down, it could be a good idea to locate them on the same server. Do bear in mind that if you do try to decide where objects is located, you may end up with an uneven distribution of the number of items on each node. This will again result in some nodes being more busy than others etc. This is why some clients doesn’t allow you to do this, so bear in mind that by doing so you might not be able to get your objects from other clients.

    Issues : CCBC-119

  • Add documentation about the error codes

    Issues : CCBC-87

  • Add lcb_verify_compiler_setup(). This function allows the “user” of the library to verify that the compiler use a compatible struct packing scheme.

Fixes in 2.0.0beta3

  • lcb_error_t member in the http callbacks shouldn’t reflect the HTTP response code. So the error code will be always LCB_SUCCESS if the library managed to receive the data successfully.

    Issues : CCBC-118

  • Fix cbc-bucket-create. `sasl-password' is misspelled, and it fails to parse the command line option.

  • Remove libtool version from the plugins

  • Do not allow admin operations without authentication

  • check for ewouldblock/eintr on failed send

  • Purge stale OBSERVE packets

    Issues : CCBC-120

  • Allow to use gethrtime() from C++

  • Remove unauthorized asserion (d344037). The lcb_server_send_packets() function later check if the server object connected and establish connection if not (with raising possible errors)

    Issues : CCBC-113

  • Don’t use the time_t for win32. When compiling from php it turns out that it gets another size of the time_t type, causing the struct offsets to differ.

  • Reformat and refactor lcb_server_purge_implicit_responses:

    • move packet allocation out of GET handler

    • dropping NOOP command shouldn’t return error code

    Issues : CCBC-120

  • Try to switch another server from backup list on timeout

    Issues : CCBC-122

  • Timer in libev uses double for interval. Ref: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#code_ev_timer_code_relative_and_opti

  • Fix illegal memory access. Reconnect config listener if the config connection was gone without proper shutdown.

    Issues : CCBC-104

  • Fix using freed memory (was introduced in 4397181)

  • Return zero from do_read_data() if operations_per_call reached. The `operations_per_call' limit was introduced to prevent from freezing event loop. But in the function variable rv could store two different results and in case of reaching this limit it is returning number of the processed records, which is wrong. The function should return either zero (success) or non-zero (failure).

    Issues : CCBC-115

Release Notes for Couchbase Client Library C 2.0.0beta2 Beta (12 October 2012)

New Features and Behavior Changes in 2.0.0beta2

  • Implement a new libev plugin. It is compatible with both libev3 and libev4.

  • Allow libcouchbase to connect to an instance without specifying bucket. It is useful when the bucket not needed, e.g. when performing administration tasks.

  • Allow users to build the library without dependencies. For example, without plugins at all. This may be useful if the plugin is implemented by or built into the host application.

  • Allow users to use environment variables to pick the event plugin

  • Add a new interface version for creating IO objects via plugins

  • Allow to disable CXX targets

  • Allow users to install both libraries (2.x and 1.x) on the same system.

  • Cleanup HTTP callbacks. Use the same callbacks both for Management and View commands, and rename them to lcb_http_complete_callback and lcb_http_data_callback.

  • Add support for raw http requests. libcouchase already contains all the bits to execute a raw http request, except for the possibility to specify a host:port, username and password.

  • Make the content type optional for lcb_make_http_request()

Fixes in 2.0.0beta2

  • Fix invalid memory access in cbc tool. Affected command is cbc-bucket-create

  • Search ev.h also in ${includedir}/libev

  • Fix password memory leak in http.c (7e71493)

  • lcb_create: replace assert() with error code

  • Breakout event loop in default error_callback. This provides better default behavior for users who haven’t defined global error callback.

    Issues : CCBC-105

  • Fix linked event/timer lists for win32

    Issues : CCBC-103

  • Fix SEGFAULT if IO struct is allocated not by the lcb_create()

  • Fix memory leak after an unsuccessful connection

  • lcb_connect() should honor the syncmode setting. Automatically call lcb_wait() when in synchronous mode

Release Notes for Couchbase Client Library C 2.0.0beta Beta (13 September 2012)

New Features and Behavior Changes in 2.0.0beta

  • Bundle Windows packages as zip archives

  • Refactor the API. This is a full redesign of the current libcouchbase API that’ll allow us to extend parts of the API without breaking binary compatibility. Also it renames all functions to have lcb prefix instead of libcouchbase and LCB/LIBCOUCHBASE in macros.

  • Implement getter for number of nodes in the cluster: lcb_get_num_nodes()

  • Add lcb_get_server_list() to get current server list

  • Deliver HTTP headers via callbacks

  • Implement RESTful flush in the cbc toolset

  • Merge lcb_get_locked into lcb_get function

  • Bundle libvbucket

Fixes in 2.0.0beta

  • Fix a problem with allocating too few slots in the backup_nodes. Fixes illegal memory access.

  • Include sys/uio.h. Needed by OpenBSD

  • Added --enable-fat-binary. Helps to solve issues when linking with fat binaries on MacOS.

  • Differentiate between TMPFAILs. This allows a developer to know if the temporary condition where the request cannot be handled is due to a constraint on the client or the server.

    Issues : CCBC-98

  • Correct buffer length for POST/PUT headers

    Issues : CCBC-96

  • Fix switching to backup node in case of server outage

    Issues : CCBC-91

  • Fix locking keys in multi-get mode

  • Release the memory allocated by the http parser

    Issues : CCBC-89

  • Fix initialization of backup nodes array. The code switching nodes relies on NULL terminator rather than nbackup_nodes variable. Fixes illegal memory access.

    Issues : CCBC-90

  • Default to IPv4 only

    Issues : CCBC-80

  • Reset timer for commands with NOT_MY_VBUCKET response

    Issues : CCBC-91

  • Sync memcached/protocol_binary.h. Pull extra protocol_binary_datatypes declarations.

  • Fix bug where HTTP method is not set

  • Don’t try to put the current node last in the backup list. This may cause “duplicates” in the list if the REST server returns another name for the server than you used. Ex: you specify “localhost” and the REST response contains 127.0.0.1

  • Release ringbuffer in lcb_purge_single_server

    Issues : CCBC-92

Release Notes for Couchbase Client Library C 1.1.0dp9 Developer Preview (27 July 2012)

Fixes in 1.1.0dp9

  • Render auth credentials for View requests. libcouchbase_make_http_request() won’t accept credentials anymore. It will pick them bucket configuration.

Release Notes for Couchbase Client Library C 1.1.0dp8 Developer Preview (27 July 2012)

New Features and Behavior Changes in 1.1.0dp8

  • Allow the user to get the number of replicas using libcouchbase_get_num_replicas()

  • Separate HTTP callbacks for couch and management requests

  • Implement read replica

    Issues : CCBC-82

  • Let users detect if the event loop running already using libcouchbase_is_waiting() function.

  • Add OBSERVE command

    Issues : CCBC-15

  • Allow users to specify content type for HTTP request.

  • Allow a user to breakout from the event loop in callbacks using libcouchbase_breakout()

  • New cbc commands and options:

    • cbc-view (remove couchview example)

    • cbc-verbosity

    • cbc-admin

    • cbc-bucket-delete

    • cbc-bucket-create

    • Add -p and -r options to cbc-cp to control persistence (uses OBSERVE internally)

  • Allow the client to specify the verbosity level on the servers using lcb_set_verbosity() function.

  • Implement general purpose timers. It is possible for users to define their own timers using libcouchbase_timer_create() function. (See headers for more info). Implement multiple timers for windows

    Issues : CCBC-85

Fixes in 1.1.0dp8

  • Claim that server has data in buffers if there are HTTP requests pending. Without this patch the event loop can be stopped prematurely.

  • Make libcouchbase_wait() re-entrable

  • Fix to handle the case when View base doesn’t have URI schema.

  • Bind timeouts to server sockets instead of commands. This means that from this point timeout interval will be started from the latest IO activity on the socket. This is a behavior change from the 1.0 series.

  • Use separate error code for ENOMEM on the client

    Issues : CCBC-77

Release Notes for Couchbase Client Library C 1.1.0dp7 Developer Preview (19 June 2012)

New Features and Behavior Changes in 1.1.0dp7

  • Implement function to execution management requests. Using libcouchbase_make_management_request() function you can configure the cluster, add/remove buckets, rebalance etc. It behaves like libcouchbase_make_couch_request() but works with another endpoint.

  • Add support for notification callbacks for configuration changes. Now it is possible to install a hook using function libcouchbase_set_configuration_callback(), and be notified about all configuration changes.

Fixes in 1.1.0dp7

  • Extract HTTP client. Backward incompatible change in Couchbase View subsystem

Release Notes for Couchbase Client Library C 1.1.0dp6 Developer Preview (13 June 2012)

New Features and Behavior Changes in 1.1.0dp6

  • Implement ‘help’ command for cbc tool

    Issues : RCBC-71

Fixes in 1.1.0dp6

  • Undefine NDEBUG to avoid asserts to be optimized out

  • Fix compilation on macosx with gtest from homebrew

    Issues : RCBC-72

  • Close dynamic libraries. Fixes small memory leak

    Issues : RCBC-70

  • Include types definitions for POSIX systems. Fixes C++ builds on some systems.

    Issues : RCBC-63

  • Fix win32 builds:

    • Add suffix to cbc command implementations;

    • Fix guards for socket errno macros;

    • Define size_t types to fix MSVC 9 build;

    • MSVC 9 isn’t C99, but has stddef.h, so just include it.

Release Notes for Couchbase Client Library C 1.1.0dp5 Developer Preview (06 June 2012)

New Features and Behavior Changes in 1.1.0dp5

  • Implement ‘cbc-hash’ to get server/vbucket for given key

Fixes in 1.1.0dp5

  • The library doesn’t depend on pthreads (eliminates package lint warnings)

Release Notes for Couchbase Client Library C 1.1.0dp4 Developer Preview (05 June 2012)

Fixes in 1.1.0dp4

  • cbc: strtoull doesn’t exist on win32, therefore use C++ equivalent.

Release Notes for Couchbase Client Library C 1.1.0dp3 Developer Preview (03 June 2012)

New Features and Behavior Changes in 1.1.0dp3

  • Implement GET_LOCKED (GETL) command

    Issues : CCBC-68

  • Implement UNLOCK_KEY (UNL) command

    Issues : CCBC-68

Fixes in 1.1.0dp3

  • Timeouts can occur during topology changes, rather than be correctly retried. Send the retry-packet to new server

    Issues : CCBC-64

  • Fix ringbuffer_memcpy() (36afdb2). Fix ringbuffer_is_continous().

  • A hang could occur in libcouchbase_wait() after the timeout period. Check for breakout condition after purging servers

    Issues : CCBC-62

  • Destroy view requests items when server get destroyed. Fixes memory leaks.

  • A fix for a buffer overflow with the supplied password as has been integrated. While it is a buffer overflow issue, this is not considered to be a possible security issue because the password to the bucket is not commonly supplied by an untrusted source

    Issues : RCBC-33

  • A small memory leak can occur with frequent calls to libcouchbase_create() and libcouchbase_destroy()

    Issues : CCBC-65

  • Use vbucket_found_incorrect_master() to get correct server index during relocating buffers. Pick up cookies from pending buffer unless node connected.

  • Do not call View callbacks for cancelled requests

  • hashset.c: iterate over whole set on rehashing. Fixes memory leaks related to hash collisions (905ef95)

Release Notes for Couchbase Client Library C 1.1.0dp2 Developer Preview (10 April 2012)

Fixes in 1.1.0dp2

  • Don’t wait for empty buffers. If called with no operations queued, libcouchbase_wait() will block forever. This means that a single threaded application that calls libcouchbase_wait() at different times to make sure operations are sent to the server runs the risk of stalling indefinitely. This is a very likely scenario.

    Issues : CCBC-59

  • Don’t define size_t and ssize_t for VS2008

  • Fix segfault while authorizing on protected buckets (211bb04)

Release Notes for Couchbase Client Library C 1.1.0dp Developer Preview (05 April 2012)

New Features and Behavior Changes in 1.1.0dp

  • This release adds new functionality to directly access Couchbase Server views using the libcouchbase_make_couch_request() function. See the associated documentation and header files for more details.

Fixes in 1.1.0dp

  • Request the tap bytes in a known byte order (adf2b30)

    Issues : MB-4834

  • Check for newer libvbucket