Introduction

A newer version of this software is available

You are viewing the documentation for an older version of this software. To find the documentation for the current version, visit the Couchbase documentation home page.

This guide provides information for developers who want to use the Couchbase C SDK to build applications that use Couchbase Server.

Getting Started

Now that you’ve installed Couchbase and have created a cluster of servers, you need a client library to read and write data from the cluster. The Couchbase C client library, also known as libcouchbase, can be used from your application to access Couchbase Server.

Here’s a quick outline of what you’ll learn in this chapter:

  • Get the client library.

  • Build the client library from source (optional).

  • Write a simple program to connecting to Couchbase and save some data.

Downloading the Couchbase Client Library

Download the latest client library from the Couchbase download site. It is available as a source archive in both .zip and .tar.gz

Installing on Linux using Packages

Packages are provided for RedHat and Ubuntu Linux systems by providing your package manager with the Couchbase repository information. The method for installation is dependent on your platform:

RedHat/CentOS

You must populate RPM with a new source, which is dependent on your RedHat version:

Then to install libcouchbase with libevent backend, run:

shell> sudo yum check-update
shell sudo yum install -y  libcouchbase2-libevent libcouchbase-devel

Ubuntu

You must update the apt-get repository to install the client library:

Also make sure you have the GPG key installed:

shell> wget -O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add -

Then to install libcouchbase with libevent backend, run:

shell> sudo apt-get update
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev

Installing using packages on Mac OS X

This client library is available via a homebrew recipe. After you install homebrew, install libcouchbase:

shell> brew install \
          https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb

Installing from Source: Linux and Mac OS X

Installation is like many common libraries, following the ./configure, make, make install conventions.

For libvbucket, extract the archive then cd into the directory and run:

shell> ./configure
shell> make install

Standard configure options such as --prefix can be passed to the configure command. For additional information on configuration options, run./configure with –help.

For libcouchbase, extract the archive, then cd into the directory and run:

shell> ./configure CPPFLAGS="-I/opt/couchbase/include" --disable-couchbasemock
shell> make install

The --disable-couchbasemock simply disables some tests which are common during the development of libcouchbase, but not required when installing a release.

Installing from Source: Microsoft Windows

Building and installing on Microsoft Windows requires nmake and tools in Microsoft Visual Studio 2010.

Open the Visual Studio Command Prompt and navigate to the directory for the extracted archive for libcouchbase. The NMakefile defines an INSTALL variable as C:\local. Edit the NMakefile if you want to change the installation location. Then build and install libcouchbase:

shell> nmake -f NMakefile install

From libcouchbase version 2.1.0, you can also use CMake system to generate correct MS Visual Studio project, for example:

shell> cmake -G "Visual Studio 10"

Hello C Couchbase

The C client library, libcouchbase, is a callback-oriented client which makes it very easy to write high performance programs. There are a few ways you can drive IO with the library. The simplest approach is to use the synchronous interface over the asynch internals of the library. More advanced programs will either call the libcouchbase_wait() function after generating some operations or drive the event loop themselves.

To connect, you first configure the connection options and then create an instance of the connection to the cluster:

struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;

memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";

err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
    fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);

/*
 * Initiate the connect sequence in libcouchbase
 */
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
    fprintf(stderr, "Failed to initiate connect: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Run the event loop and wait until we've connected */
lcb_wait(instance);

Callbacks are used by the library and are simple functions which handle the result of operations. For example:

struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;

memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";

err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
    fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);

/*
 * Initiate the connect sequence in libcouchbase
 */
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
    fprintf(stderr, "Failed to initiate connect: %s\n",
            lcb_strerror(NULL, err));
    return 1;
}

/* Run the event loop and wait until we've connected */
lcb_wait(instance);

Callbacks can be set up for all of your operations called in libcouchbase. In the API, you’ll note the use of a cookie. This is metadata from your application which is associated with the request. The libcouchbase library will not inspect any cookie or send the cookie to the server.

When you put the connect logic and the get callback together and plug them into a complete program with the include headers, you get:

#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

static void error_callback(lcb_t instance,
                           lcb_error_t err,
                           const char *errinfo)
{
    fprintf(stderr, "Error %s: %s", lcb_strerror(instance, err),
            errinfo ? errinfo : "");
    exit(EXIT_FAILURE);
}

/* the callback invoked by the library when receiving a get response */
static void get_callback(lcb_t instance,
                         const void *cookie,
                         lcb_error_t error,
                         const lcb_get_resp_t *resp)
{
    if (error != LCB_SUCCESS) {
        fprintf(stderr, "Failed to retrieve \"");
        fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
        fprintf(stderr, "\": %s\n", lcb_strerror(instance, error));
    } else {
        fprintf(stderr, "Data for key: \"");
        fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
        fprintf(stderr, "\" is : ");
        fwrite(resp->v.v0.bytes, 1, resp->v.v0.nbytes, stderr);
    }
    (void)cookie; /* ignore */
}

int main(void)
{
    struct lcb_create_st create_options;
    lcb_t instance;
    lcb_error_t err;

    memset(&create_options, 0, sizeof(create_options));
    create_options.v.v0.host = "myserver:8091";
    create_options.v.v0.user = "mybucket";
    create_options.v.v0.passwd = "secret";
    create_options.v.v0.bucket = "mybucket";

    err = lcb_create(&instance, &create_options);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }

    /* set up the handler to catch all errors */
    lcb_set_error_callback(instance, error_callback);

    /* initiate the connect sequence in libcouchbase */
    err = lcb_connect(instance);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to initiate connect: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }

    /* run the event loop and wait until we've connected */
    lcb_wait(instance);

    /* set up a callback for our get requests  */
    lcb_set_get_callback(instance, get_callback);

    {
        lcb_get_cmd_t cmd;
        const lcb_get_cmd_t *commands[1];

        commands[0] = &cmd;
        memset(&cmd, 0, sizeof(cmd));
        cmd.v.v0.key = "foo";
        cmd.v.v0.nkey = 3;

        err = lcb_get(instance, NULL, 1, commands);
        if (err != LCB_SUCCESS) {
            fprintf(stderr, "Failed to get: %s\n",
                    lcb_strerror(NULL, err));
            return 1;
        }
    }

    lcb_wait(instance);

    lcb_destroy(instance);
    exit(EXIT_SUCCESS);
}

To compile this sample program, you must link to libcouchbase :

shell> gcc -o hellocb -lcouchbase hellocb.c

Tutorial

This tutorial assumes you have installed libcouchbase on your systems per the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.

If you need to set up a server node or data bucket, you can do so with Couchbase Administrative Console, Couchbase Command-Line Interface (CLI), or Couchbase REST API. For information and instructions, see:

After you’ve set up your Couchbase Server and installed the needed client libraries, you can compile and run the following basic program.

Simple Example

Before getting into a more complex example of the programming model to this library, we will walk through a straightforward example of a program which builds with libcouchbase, connects to a server, and sets and gets a value:

/* -*- Mode: C; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- */
/*
 *     Copyright 2012 Couchbase, Inc.
 *
 *   Licensed under the Apache License, Version 2.0 (the "License");
 *   you may not use this file except in compliance with the License.
 *   You may obtain a copy of the License at
 *
 *       http://www.apache.org/licenses/LICENSE-2.0
 *
 *   Unless required by applicable law or agreed to in writing, software
 *   distributed under the License is distributed on an "AS IS" BASIS,
 *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 *   See the License for the specific language governing permissions and
 *   limitations under the License.
 */

/*
 * BUILD:
 *
 *      cc -o minimal minimal.c -lcouchbase
 *      cl /DWIN32 /Iinclude minimal.c lib\libcouchbase.lib
 *
 * RUN:
 *
 *      valgrind -v --tool=memcheck  --leak-check=full --show-reachable=yes ./minimal
 *      ./minimal <host:port> <bucket> <passwd>
 *      mininal.exe <host:port> <bucket> <passwd>
 */
#include <stdio.h>
#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#ifdef _WIN32
#define PRIu64 "I64u"
#else
#include <inttypes.h>
#endif

static void error_callback(lcb_t instance, lcb_error_t error, const char *errinfo)
{
    fprintf(stderr, "ERROR: %s (0x%x), %s\n",
            lcb_strerror(instance, error), error, errinfo);
    exit(EXIT_FAILURE);
}

static void store_callback(lcb_t instance, const void *cookie,
                           lcb_storage_t operation,
                           lcb_error_t error,
                           const lcb_store_resp_t *item)
{
    if (error == LCB_SUCCESS) {
        fprintf(stderr, "STORED \"");
        fwrite(item->v.v0.key, sizeof(char), item->v.v0.nkey, stderr);
        fprintf(stderr, "\" CAS: %"PRIu64"\n", item->v.v0.cas);
    } else {
        fprintf(stderr, "STORE ERROR: %s (0x%x)\n",
                lcb_strerror(instance, error), error);
        exit(EXIT_FAILURE);
    }
    (void)cookie;
    (void)operation;
}

static void get_callback(lcb_t instance, const void *cookie, lcb_error_t error,
                         const lcb_get_resp_t *item)
{
    if (error == LCB_SUCCESS) {
        fprintf(stderr, "GOT \"");
        fwrite(item->v.v0.key, sizeof(char), item->v.v0.nkey, stderr);
        fprintf(stderr, "\" CAS: %"PRIu64" FLAGS:0x%x SIZE:%lu\n",
                item->v.v0.cas, item->v.v0.flags, (unsigned long)item->v.v0.nbytes);
        fwrite(item->v.v0.bytes, sizeof(char), item->v.v0.nbytes, stderr);
        fprintf(stderr, "\n");
    } else {
        fprintf(stderr, "GET ERROR: %s (0x%x)\n",
                lcb_strerror(instance, error), error);
    }
    (void)cookie;
}

int main(int argc, char *argv[])
{
    lcb_error_t err;
    lcb_t instance;
    struct lcb_create_st create_options;
    struct lcb_create_io_ops_st io_opts;

    io_opts.version = 0;
    io_opts.v.v0.type = LCB_IO_OPS_DEFAULT;
    io_opts.v.v0.cookie = NULL;

    memset(&create_options, 0, sizeof(create_options));
    err = lcb_create_io_ops(&create_options.v.v0.io, &io_opts);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create IO instance: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }

    if (argc > 1) {
        create_options.v.v0.host = argv[1];
    }
    if (argc > 2) {
        create_options.v.v0.user = argv[2];
        create_options.v.v0.bucket = argv[2];
    }
    if (argc > 3) {
        create_options.v.v0.passwd = argv[3];
    }
    err = lcb_create(&instance, &create_options);
    if (err != LCB_SUCCESS) {
        fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
                lcb_strerror(NULL, err));
        return 1;
    }
    (void)lcb_set_error_callback(instance, error_callback);
    /* Initiate the connect sequence in libcouchbase */
    if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
        fprintf(stderr, "Failed to initiate connect: %s\n",
                lcb_strerror(NULL, err));
        lcb_destroy(instance);
        return 1;
    }
    (void)lcb_set_get_callback(instance, get_callback);
    (void)lcb_set_store_callback(instance, store_callback);
    /* Run the event loop and wait until we've connected */
    lcb_wait(instance);
    {
        lcb_store_cmd_t cmd;
        const lcb_store_cmd_t *commands[1];

        commands[0] = &cmd;
        memset(&cmd, 0, sizeof(cmd));
        cmd.v.v0.operation = LCB_SET;
        cmd.v.v0.key = "foo";
        cmd.v.v0.nkey = 3;
        cmd.v.v0.bytes = "bar";
        cmd.v.v0.nbytes = 3;
        err = lcb_store(instance, NULL, 1, commands);
        if (err != LCB_SUCCESS) {
            fprintf(stderr, "Failed to set: %s\n", lcb_strerror(NULL, err));
            return 1;
        }
    }
    lcb_wait(instance);
    {
        lcb_get_cmd_t cmd;
        const lcb_get_cmd_t *commands[1];
        commands[0] = &cmd;
        memset(&cmd, 0, sizeof(cmd));
        cmd.v.v0.key = "foo";
        cmd.v.v0.nkey = 3;
        err = lcb_get(instance, NULL, 1, commands);
        if (err != LCB_SUCCESS) {
            fprintf(stderr, "Failed to get: %s\n", lcb_strerror(NULL, err));
            return 1;
        }
    }
    lcb_wait(instance);
    lcb_destroy(instance);

    return 0;
}

Additional Resources

In addition to the other sections of this manual, such as the Getting Started guide and the API reference, the libcouchbase package includes an examples directory and a tools directory. Each of these show simple Couchbase tools and an example libcouchbase programs.

On Linux you may find these examples in TODO…

Common Patterns

SASL

Starting from the version 2.2 libcouchbase supports CRAM-MD5 authentication mechanism. Which allows to avoid passing bucket password as a plain text over the wires.

Along with this change, new setting was introduced LCB_CNTL_FORCE_SASL_MECH. It forces a specific SASL mechanism to use for authentication. This can allow a user to ensure a certain level of security and have the connection fail if the desired mechanism is not available.

lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_FORCE_SASL_MECH, "CRAM-MD5");

Integration with libcouchbase

This tutorial assumes you have installed libcouchbase on your systems, following the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.

The libcouchbase is written in C and can be integrated in various ways with your application. The simplest integration scenario is when your application is written in a scripting language (supported by a Couchbase client library) that already has a libcouchbase wrapper. Couchbase has built and maintains following libcouchbase-based libraries:

Users of these libraries automatically get an API, which looks natural for their platform, so that it doesn’t require knowledge of libcouchbase APIs. This tutorial is mostly about two other use cases:

  • Add libcouchbase into an existing application to implement persistence layer. This section will contain two parts: first we will create a simple echo server written using libev library. Second, we will show how to persist each message going through the server to Couchbase.

    Get complete code here: https://github.com/couchbaselabs/libev-couchbase-example

  • Build applications around libcouchbase. Here we’ll show how to build a proxy server for regular memcached clients. Building this proxy server is just like building a moxi server, but the proxy server will be more limited more limited in abilities.

    Get complete code here: https://github.com/couchbaselabs/libcouchbase-proxy-sample

Add to an Application

Let’s start with a specific existing application for this example. Our application is the asynchronous single-threaded echo server, which leverages libev to solve C10K problem and efficiently serves a huge number of concurrent connections. It should listen to the given port (in this sample 4567) and send back everything the client submitted to it.

The libev library implements reactor pattern which uses the term event loop to represent an entity which encapsulates the register of handlers for various IO events and the timers. To start, we have the application code from the main function:

int main()
{
    struct ev_loop *loop = ev_default_loop(0);
    int sd;
    struct sockaddr_in addr;
    struct ev_io server;

    if ((sd = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
        fail("socket error");
    }

    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(PORT_NO);
    addr.sin_addr.s_addr = INADDR_ANY;

    if (bind(sd, (struct sockaddr *) &addr, sizeof(addr)) != 0) {
        fail("bind error");
    }
    if (listen(sd, 2) < 0) {
        fail("listen error");
    }

    ev_io_init(&server, accept_cb, sd, EV_READ);
    ev_io_start(loop, &server);

    printf("Listen on %d\n", PORT_NO);
    ev_run(loop, 0);

    return 0;
}

Here we are create an event loop instance, and create and bind a new socket to local port PORT_NO (4567). After that we can mark this socket as listening and ask libev to call the function accept_cb when the socket become ready to read, that is when the new client will come to us. The final step is to run ev_run() which does all the server interaction.

The accept_cb is in charge of allocating client structures 60 the buffer for messages. The sample is using ring buffers from the libcouchbase code for easier managing memory for IO buffers.

void accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
    struct client_s *client;
    int flags;

    if (EV_ERROR & revents) {
        fail("got invalid event");
    }
    client = malloc(sizeof(struct client_s));
    if (!client) {
        fail("client watcher alloc");
    }
    if (!ringbuffer_initialize(&client->buf, BUFFER_SIZE)) {
        fail("client buffer alloc");
    }
    client->naddr = sizeof(client->addr);
    client->fd = accept(watcher->fd, (struct sockaddr *)&client->addr,
                        &client->naddr);
    if (client->fd < 0) {
        fail("accept error");
    }
    if ((flags = fcntl(client->fd, F_GETFL, NULL)) == -1) {
        fail("fcntl(F_GETFL)");
    }
    if (fcntl(client->fd, F_SETFL, flags | O_NONBLOCK) == -1) {
        fail("fcntl(F_SETFL, O_NONBLOCK)");
    }

    printf("Successfully connected with client\n");

    ev_io_init((ev_io *)client, client_cb, client->fd, EV_READ);
    ev_io_start(loop, (ev_io *)client);
}

The most interesting part of the function above is in the last two lines. It registers the new handler, client_cb, upon reading an event for the client socket. This handler will be called each time when the kernel receives new data from the socket, so that it could be read with a non-blocking call.

The last snippet of our application depicts the client_cb function, which handles all the application logic. It reads everything from the socket if it is ready, registers itself for write events, and then sends all the contents out to the network. It is also the handling end of a stream situation, which means that the client closed the connection with the Couchbase server.

void client_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
    struct lcb_iovec_st iov[2];
    ssize_t nbytes;
    struct client_s *client = (struct client_s *)watcher;

    if (EV_READ & revents) {
        ringbuffer_ensure_capacity(&client->buf, BUFFER_SIZE);
        ringbuffer_get_iov(&client->buf, RINGBUFFER_WRITE, iov);
        nbytes = io_recvv(watcher->fd, iov);
        if (nbytes < 0) {
            fail("read error");
        } else if (nbytes == 0) {
            ev_io_stop(loop, watcher);
            ringbuffer_destruct(&client->buf);
            free(client);
            printf("Peer disconnected\n");
            return;
        } else {
            ringbuffer_produced(&client->buf, nbytes);
            printf("Received %zu bytes\n", nbytes);
            ev_io_stop(loop, watcher);
            ev_io_set(watcher, watcher->fd, EV_WRITE);
            ev_io_start(loop, watcher);
        }
    } else if (EV_WRITE & revents) {
        ringbuffer_get_iov(&client->buf, RINGBUFFER_READ, iov);
        nbytes = io_sendv(watcher->fd, iov);
        if (nbytes < 0) {
            fail("write error");
        } else if (nbytes == 0) {
            ev_io_stop(loop, watcher);
            ringbuffer_destruct(&client->buf);
            free(client);
            printf("Peer disconnected\n");
            return;
        } else {
            ringbuffer_consumed(&client->buf, nbytes);
            printf("Sent %zu bytes\n", nbytes);
            ev_io_stop(loop, watcher);
            ev_io_set(watcher, watcher->fd, EV_READ);
            ev_io_start(loop, watcher);
        }
    } else  {
        fail("got invalid event");
    }
}

Again, the whole application is stored on github at https://github.com/couchbaselabs/libev-couchbase-example. Let’s see it in action. To clone and build the applications, use the following commands:

~ $ git clone git://github.com/couchbaselabs/libev-couchbase-example.git
~ $ cd libev-couchbase-example/step1
step1 $ ./autogen.sh && ./configure && make

As a client for an echo application, we can build any telnet-like application. The following listing simulates two terminals by splitting them with a vertical line:

step 1 $ ./server                   |  step1 $ telnet localhost 4567
Listen on 4567                      |  Trying 127.0.0.1...
Successfully connected with client  |  Connected to localhost.
Received 14 bytes                   |  Escape character is '^]'.
Sent 14 bytes                       |  Hello world!
Received 9 bytes                    |  Hello world!
Sent 9 bytes                        |  foo bar
Peer disconnected                   |  foo bar
                                    |  ^]
                                    |  telnet> Connection closed.

Now that we described our application, let’s integrate Couchbase as the database using libcouchbase. The server.c file will have some small additions. In the main function, before initializing server socket, we will call storage_init and assign the connection information to the server structure variable.

server.handle = storage_init(loop, "localhost:8091", "default", null);

Also in the client_cb handler, each time libcouchbase receives data from the client, it will send the data to Couchbase server with storage_put where the_key is a string "example".

storage_put(client, the_key, val, nbytes);

Here, the initial application from step one is already using its own custom event loop to handle other IO in the application. The application could be more complex, but libcouchbase has a separate interface to inject your IO implementation into the library. In the simplest case, for example if your application is using one of the IO libraries supported by libcouchbase, you can just pass your event loop instance into the initializer, and libcouchbase won’t create new one. Instead it will register all its events on this external loop. In case your application is using its own implementation of a reactor pattern, such as nginx, or you use linux epoll API, you can easily write your own IO plugin and pass it to the connection initializer.

The echo server is using libev library, and libcouchbase provides a plugin to libev out-of-the-box. In this case we can imagine that it doesn’t already have it. In this case, we can copy the plugin from libcouchbase distribution into the directory step2/lcb-plugin/ and update build files appropriately.

Now we inspect the storage.c, where our storage_init() and storage_put() defined.

lcb_t storage_init(struct ev_loop *loop, const char *host, const char *bucket, const char *password)
{
    struct lcb_create_st opts;
    struct lcb_create_io_ops_st io_opts;
    lcb_t handle;
    lcb_error_t err;

    io_opts.version = 1;
    io_opts.v.v1.sofile = NULL;
    io_opts.v.v1.symbol = "lcb_create_libev_io_opts";
    io_opts.v.v1.cookie = loop;

    opts.version = 0;
    opts.v.v0.host = host;
    opts.v.v0.bucket = bucket;
    opts.v.v0.user = bucket;
    opts.v.v0.passwd = password;

    err = lcb_create_io_ops(&opts.v.v0.io, &io_opts);
    if (err != LCB_SUCCESS) {
        error_callback(NULL, err, "failed to create IO object");
        return NULL;
    }
    err = lcb_create(&handle, &opts);
    if (err != LCB_SUCCESS) {
        error_callback(NULL, err, "failed to create connection object");
        return NULL;
    }

    (void)lcb_set_error_callback(handle, error_callback);
    (void)lcb_set_store_callback(handle, storage_callback);

    err = lcb_connect(handle);
    if (err != LCB_SUCCESS) {
        error_callback(handle, err, "failed to connect to the server");
        return NULL;
    }

    return handle;
}

The end of the function may look familiar, but the io_opts structure is more interesting. It defines how to look up our custom IO plugin. The io_opts.v.v1.sofile is set to NULL to specify that our plugin compiled into the current executable image. In this case, symbol is the string name of the function with the following signature:

lcb_error_t lcb_create_libev_io_opts(int version, lcb_io_opt_t *io, void *loop);

This initializes the IO plugin and then the cookie will be passed to this function as the loop argument. You can refer to the lcb_create_io_ops(3) manpage for other ways to initialize IO subsystem.

The libcouchbase client is purely asynchronous; therefore, each data operation isplits into two parts: a function-scheduler, which validates and copies all arguments to internal buffers for further handing, and a function-callback, which will be called with the results of the operation. It is possible to implement a wrapper which will provide a more synchronous API, but it is difficult to write a library API that is both generic and efficient. Since all communication is done by using function arguments, it is easier to maintain backward compatible APIs by versioning both incoming structures and results. Let’s see how we did it in our sample:

void storage_put(struct client_s *client, const char *key,
                 const void *val, size_t nval)
{
    lcb_error_t err;
    lcb_store_cmd_t cmd;
    const lcb_store_cmd_t *cmds[] = { &cmd };

    memset(&cmd, 0, sizeof(cmd));
    cmd.version = 0;
    cmd.v.v0.key = key;
    cmd.v.v0.nkey = strlen(key);
    cmd.v.v0.bytes = val;
    cmd.v.v0.nbytes = nval;
    cmd.v.v0.operation = LCB_SET;
    err = lcb_store(client->handle, client, 1, cmds);
    if (err != LCB_SUCCESS) {
        error_callback(client->handle, err, "failed to schedule store operation");
    }
}

void storage_callback(lcb_t instance, const void *cookie,
                      lcb_storage_t operation, lcb_error_t error,
                      const lcb_store_resp_t *resp)
{
    struct client_s *client = (struct client_s *)cookie;
    struct ev_io* watcher = (struct ev_io*)client;
    char cas_str[128];
    ssize_t nb;

    nb = snprintf(cas_str, 128, "%"PRIu64"\n", resp->v.v0.cas);
    if (nb < 0) {
        fail("output CAS value");
    }
    ringbuffer_ensure_capacity(&client->out, nb);
    if (ringbuffer_write(&client->out, cas_str, nb) != nb) {
        fail("write CAS into the buffer");
    }
    ev_io_stop(client->loop, watcher);
    ev_io_set(watcher, watcher->fd, EV_WRITE);
    ev_io_start(client->loop, watcher);

    (void)operation;
    (void)resp;
}

Here, the storage_put() function is a wrapper over lcb_store(3). The storage_put() function translates arguments to the versioned structure and runs lcb_store to pass it to the library. The data won’t be sent immediately, but when the library connects to the data socket, it will be ready to accept data. After the server processes the request, the application will be notified asynchronously via storage_callback. This callback will copy CAS value to the output buffer to be sent to the client.

Time to demonstrate an application. This application is built the same as before, except you need to navigate to step2/ directory. As previously noted, the script will be split to demonstrate both server and the client:

step2 $ ./server                    |  step2 $ telnet localhost 4567
Listen on 4567                      |  Trying 127.0.0.1...
Successfully connected with client  |  Connected to localhost.
Received 14 bytes                   |  Escape character is '^]'.
Sent 20 bytes                       |  Hello world!
Peer disconnected                   |  2916447493390860288
                                    |  ^]
                                    |  telnet> Connection closed.

Now you can install cbc tools from package libcouchbase2-bin. If you installed libcouchbase from source, you most likely have it already. Then you can check that it created a key example and that the CAS values are matching:

step2 $ printf "%x\n" 2916447493390860288
28794d8b11a40000
step2 $ cbc cat example
"example" Size:14 Flags:0 CAS:28794d8b11a40000
Hello world!

Build an Application

You can use libcouchbase when you are building Couchbase-enabled application and want to bootstrap quickly. To do so you can build your application using the IO abstraction provided with libcouchbase.

In this example, we create a thin proxy application which will expose the memcached-compatible API for the system. Similar approaches exist for moxi server. Also this application will use less hard-coded logic and more look like real-life service. Full source code can be found at https://github.com/couchbaselabs/libcouchbase-proxy-sample.

For clarity, we skip parsing and initialization of the Couchbase connection from the main function. The proxy code begins in run_proxy, which is similar to the main function in the previous section. The important difference here is that we are using the IO abstraction from libcouchbase, to register events, and drive an event loop.

void run_proxy(lcb_t conn)
{
    lcb_io_opt_t io;
    struct sockaddr_in addr;
    int sock, rv;
    server_t server;

    io = opts.v.v0.io;
    info("starting proxy on port %d", port);
    sock = socket(PF_INET, SOCK_STREAM, 0);
    if (sock == -1) {
        fail("socket()", strerror(errno));
    }
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = INADDR_ANY;
    rv = bind(sock, (struct sockaddr *)&addr, sizeof(addr));
    if (rv == -1) {
        fail("bind()");
    }
    rv = listen(sock, 10);
    if (rv == -1) {
        fail("listen()");
    }
    server.conn = conn;
    server.io = io;
    server.event = io->v.v0.create_event(io);
    if (server.event == NULL) {
        fail("failed to create event for proxy");
    }
    io->v.v0.update_event(io, sock, server.event, LCB_READ_EVENT,
                          &server, proxy_accept_callback);
    lcb_set_error_callback(conn, error_callback);
    lcb_set_get_callback(conn, get_callback);
    lcb_set_store_callback(conn, store_callback);
    info("use ctrl-c to stop");
    io->v.v0.run_event_loop(io);
}

As in the previous example, we create a listening socket, and then we bind a handler to process read events. The handler is proxy_accept_callback, and it will be triggered for each new client. In turn it will create a new client structure, defined as follows:

typedef struct client_st client_t;
struct client_st {
    int id;
    server_t *server;
    int sock;
    ringbuffer_t in;
    ringbuffer_t out;
    void *event;
};

This initializes input and output buffers,and makes the socket descriptors non-blocking. At the end, the code will register the handler function proxy_client_callback for read events, and then wait for incoming data.

The proxy implements a very limited number of protocol commands: GET, SET, and VERSION. But other commands can be easily added. Here’s an example:

void proxy_client_callback(lcb_socket_t sock, short which, void *data)
{
    struct lcb_iovec_st iov[2];
    ssize_t rv;
    lcb_io_opt_t io;
    client_t *cl = data;

    io = cl->server->io;
    if (which & LCB_READ_EVENT) {
        for (;;) {
            /* read in chunks of BUFFER_SIZE */
            ringbuffer_ensure_capacity(&cl->in, BUFFER_SIZE);
            ringbuffer_get_iov(&cl->in, RINGBUFFER_WRITE, iov);
            rv = io->v.v0.recvv(io, cl->sock, iov, 2);
            if (rv == -1) {
                if (io->v.v0.error == EINTR) {
                    /* interrupted by signal */
                    continue;
                } else if (io->v.v0.error == EWOULDBLOCK) {
                    /* nothing to read right now */
                    io->v.v0.update_event(io, cl->sock, cl->event,
                                          LCB_WRITE_EVENT, cl,
                                          proxy_client_callback);
                    break;
                } else {
                    fail("read error");
                }
            } else if (rv == 0) {
                /* end of stream */
                io->v.v0.destroy_event(io, cl->event);
                ringbuffer_destruct(&cl->in);
                ringbuffer_destruct(&cl->out);
                close(cl->sock);
                info("[%d] disconnected", cl->id);
                free(cl);
                return;
            } else {
                ringbuffer_produced(&cl->in, rv);

                for (;;) {
                    protocol_binary_request_header req;
                    lcb_size_t nr, sz;
                    char *buf;

                    /* make sure the buffer is aligned */
                    if (ringbuffer_ensure_alignment(&cl->in) != 0) {
                        fail("cannot align the buffer");
                    }
                    /* take the packet header from the buffer */
                    nr = ringbuffer_peek(&cl->in, req.bytes, sizeof(req));
                    if (nr < sizeof(req)) {
                        break;
                    }
                    /* make sure the buffer has whole the body */
                    sz = ntohl(req.request.bodylen) + sizeof(req);
                    if (cl->in.nbytes < sz) {
                        break;
                    }
                    /* copy packet into intermediate buffer */
                    buf = malloc(sizeof(char) * sz);
                    if (buf == NULL) {
                        fail("cannot allocate buffer for packet");
                    }
                    nr = ringbuffer_read(&cl->in, buf, sz);
                    if (nr < sizeof(req)) {
                        fail("input buffer doesn't contain enough data");
                    }
                    /* handle packet and deallocate the intermediate
                     * buffer */
                    handle_packet(cl, buf);
                    free(buf);
                }
            }
        }
    }
    if (which & LCB_WRITE_EVENT) {
        /* check if we have something to send */
        ringbuffer_get_iov(&cl->out, RINGBUFFER_READ, iov);
        if (iov[0].iov_len + iov[1].iov_len == 0) {
            io->v.v0.delete_event(io, cl->sock, cl->event);
            return;
        }
        rv = io->v.v0.sendv(io, cl->sock, iov, 2);
        if (rv < 0) {
            fail("write error");
        } else if (rv == 0) {
            io->v.v0.destroy_event(io, cl->event);
            ringbuffer_destruct(&cl->in);
            ringbuffer_destruct(&cl->out);
            close(cl->sock);
            info("[%d] disconnected", cl->id);
            free(cl);
            return;
        } else {
            ringbuffer_consumed(&cl->out, rv);
            io->v.v0.update_event(io, cl->sock, cl->event,
                                  LCB_READ_EVENT, cl,
                                  proxy_client_callback);
        }
    }
    (void)sock;
}

The function proxy_client_callback logically divided into two parts.

  • The first part occurs when the event loop notifies us that the socket is ready for non-blocking reading. At this point, the function tries to read all the data available on the socket. It finds the packets and passes aligned byte arrays to handle_packet. Once the code gets a special code from the IO subsystem get from the IO subsystem special code, that it cannot read anything without blocking, it stops processing and registers itself for write events, because in nearest future, responses will appear in the output buffer.

  • The second part of the function reacts to write events and is much simpler. It just checks to see if there is data in the output buffer. If there is data in the output buffer, it will try to send that data to the client. If not, the functionwill unregister itself from write events to save CPU cycles.

The function handle_packet is a good place to start playing with this example if you’d like to add new features there. It decodes protocol packet and translates it into libcouchbase calls.

void handle_packet(client_t *cl, char *buf)
{
    protocol_binary_request_header *req = (void *)buf;
    protocol_binary_response_header res;
    union {
        lcb_get_cmd_t get;
        lcb_store_cmd_t set;
    } cmd;
    union {
        const lcb_get_cmd_t *get[1];
        const lcb_store_cmd_t *set[1];
    } cmds;
    cookie_t *cookie;

    cookie = malloc(sizeof(cookie_t));
    if (cookie == NULL) {
        fail("cannot allocate buffer for command cookie");
    }
    cookie->client = cl;
    cookie->opaque = req->request.opaque;
    cookie->opcode = req->request.opcode;
    memset(&cmd, 0, sizeof(cmd));
    switch (req->request.opcode) {
    case PROTOCOL_BINARY_CMD_GET:
        cmds.get[0] = &cmd.get;
        cmd.get.v.v0.nkey = ntohs(req->request.keylen);
        cmd.get.v.v0.key = buf + sizeof(*req);
        info("[%d] get \"%.*s\"", cl->id,
             (int)cmd.get.v.v0.nkey, (char *)cmd.get.v.v0.key);
        lcb_get(cl->server->conn, (const void*)cookie, 1, cmds.get);
        break;
    case PROTOCOL_BINARY_CMD_SET:
        cmds.set[0] = &cmd.set;
        cmd.set.v.v0.operation = LCB_SET;
        cmd.set.v.v0.nkey = ntohs(req->request.keylen);
        cmd.set.v.v0.key = buf + sizeof(*req) + 8;
        cmd.set.v.v0.nbytes = ntohl(req->request.bodylen) - 8 - ntohs(req->request.keylen);
        cmd.set.v.v0.bytes = buf + sizeof(*req) + 8 + ntohs(req->request.keylen);
        cmd.set.v.v0.cas = req->request.cas;
        cmd.set.v.v0.datatype = req->request.datatype;
        memcpy(&cmd.set.v.v0.flags, buf + sizeof(*req), 4);
        cmd.set.v.v0.flags = ntohl(cmd.set.v.v0.flags);
        memcpy(&cmd.set.v.v0.exptime, buf + sizeof(*req) + 4, 4);
        cmd.set.v.v0.exptime = ntohl(cmd.set.v.v0.exptime);
        info("[%d] set \"%.*s\"", cl->id,
             (int)cmd.set.v.v0.nkey, (char *)cmd.set.v.v0.key);
        lcb_store(cl->server->conn, (const void*)cookie, 1, cmds.set);
        break;
    case PROTOCOL_BINARY_CMD_VERSION:
        free(cookie);
        info("[%d] version", cl->id);
        res.response.magic = PROTOCOL_BINARY_RES;
        res.response.opcode = req->request.opcode;
        res.response.keylen = 0;
        res.response.extlen = 0;
        res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
        res.response.status = PROTOCOL_BINARY_RESPONSE_SUCCESS;
        res.response.bodylen = htonl(sizeof(version_msg));
        res.response.opaque = req->request.opaque;
        res.response.cas = 0;
        ringbuffer_ensure_capacity(&cl->out, sizeof(res));
        ringbuffer_write(&cl->out, res.bytes, sizeof(res));
        ringbuffer_write(&cl->out, version_msg, sizeof(version_msg));
        break;
    default:
        free(cookie);
        info("[%d] unsupported command: 0x%02x", cl->id, req->request.opcode);
        res.response.magic = PROTOCOL_BINARY_RES;
        res.response.opcode = req->request.opcode;
        res.response.keylen = 0;
        res.response.extlen = 0;
        res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
        res.response.status = PROTOCOL_BINARY_RESPONSE_NOT_SUPPORTED;
        res.response.bodylen = htonl(sizeof(notsup_msg));
        res.response.opaque = req->request.opaque;
        res.response.cas = 0;
        ringbuffer_ensure_capacity(&cl->out, sizeof(notsup_msg) + sizeof(res));
        ringbuffer_write(&cl->out, res.bytes, sizeof(res));
        ringbuffer_write(&cl->out, notsup_msg, sizeof(notsup_msg));
    }
}

To maintain a logical link between requests and responses for the clients, the protocol defines a special field opaque, which can be a special tag or sequence number. To preserve this field in the response, we will put it into a cookie_t structure. During the proxy initialization, we defined two callback-functions for libcouchbase:

lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);

In this kind of application, these functions just build a protocol response from libcouchbase return value. They also register the proxy_client_callback on write events to send out data recently written into the buffer.

void get_callback(lcb_t conn, const void *cookie, lcb_error_t err,
                  const lcb_get_resp_t *item)
{
    cookie_t *c = (cookie_t *)cookie;
    client_t *cl = c->client;
    lcb_io_opt_t io = cl->server->io;
    protocol_binary_response_get res;

    res.message.header.response.magic = PROTOCOL_BINARY_RES;
    res.message.header.response.opcode = PROTOCOL_BINARY_CMD_GET;
    res.message.header.response.keylen = 0;
    res.message.header.response.extlen = 4;
    res.message.header.response.datatype = item->v.v0.datatype;
    res.message.header.response.status = htons(map_status(err));
    res.message.header.response.bodylen = htonl(4 + item->v.v0.nbytes);
    res.message.header.response.opaque = c->opaque;
    res.message.header.response.cas = item->v.v0.cas;
    res.message.body.flags = htonl(item->v.v0.flags);
    ringbuffer_ensure_capacity(&cl->out, sizeof(res.bytes) + item->v.v0.nbytes);
    ringbuffer_write(&cl->out, res.bytes, sizeof(res.bytes));
    ringbuffer_write(&cl->out, item->v.v0.bytes, item->v.v0.nbytes);
    io->v.v0.update_event(io, cl->sock, cl->event, LCB_WRITE_EVENT,
                          cl, proxy_client_callback);
    free(c);
    (void)conn;
}

These are viritually all the integration points with libcouchbase. As we demonstrated, libcouchbase more than a protocol parser; it also an abstract and portable IO framework, allowing you to build asynchronous applications around Couchbase Server.

To conclude, let’s run your favourite memcached client and verify our proxy. First you need to clone and build it:

$ git clone git://github.com/couchbaselabs/libcouchbase-proxy-sample.git
$ cd libcouchbase-proxy-sample
libcouchbase-proxy-sample $ make

Run the server and send couple of commands to it. We will use the dalli ruby gem here:

libcouchbase-proxy-sample $ ./proxy                 |  $ irb
connecting to bucket "default" at "localhost:8091"  |  irb:001:0> require 'dalli'
starting proxy on port 1987                         |  true
use ctrl-c to stop                                  |  irb:002:0> m = Dalli::Client.new  "localhost:1987"
[1] connected                                       |  #<Dalli::Client:0x007f716321ce20 @servers=["localhost:1987"], @options={}, @ring=nil>
[1] version                                         |  irb:003:0> m.set("foo", "bar")
[1] set "foo"                                       |  true
[1] get "foo"                                       |  irb:004:0> m.get("foo")
[1] disconnected                                    |  "bar"

Appendix: Release Notes

The following sections provide release notes for individual release versions of Couchbase Client Library C. To browse or submit new issues, see Couchbase Client Library C Issues Tracker.

Release Notes for Couchbase Client Library C 2.2.0 GA (06 November 2013)

New Features and Behavior Changes in 2.2.0

  • Handle 302 redirects in HTTP (views and administrative requests). By default the library follows up to three redirects. After the limit is reached, the request is terminated with code LCB_TOO_MANY_REDIRECTS. The limit is configurable through LCB_CNTL_MAX_REDIRECTS. If it is set to -1, it disables the redirect limit. The following example shows how to set the limit:

      int new_value = 5;
  lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_MAX_REDIRECTS, &new_value);

    Issues: CCBC-169

  • Replace isasl with cbsasl. cbsasl implements both PLAIN and CRAM-MD5 authentication mechanisms.

  • LCB_CNTL_MEMDNODE_INFO command updated to include effective SASL mechanism:

      cb_cntl_server_t node;
  node.version = 1;
  node.v.v1.index = 0; /* first node */
  lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_MEMDNODE_INFO, &node);
  if (node.v.v1.sasl_mech) {
      printf("authenticated via SASL '%s'\n",
             node.v.v1.sasl_mech);
  }

  • You can force a specific authentication mechanism for the connection handle by using the LCB_CNTL_FORCE_SASL_MECH command:

      lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_FORCE_SASL_MECH, "PLAIN");

    Issues: CCBC-243

  • Stricter, more inspectable behavior for the configuration cache. This provides a test and an additional lcb_cntl operation to check the status of the configuration cache. Also, it switches off the configuration cache with memcached buckets.

      int is_loaded;
  lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
  if (is_loaded) {
      printf("Configuration cache saved us a trip to the config server\n");
  } else {
      printf("We had to contact the configuration server for some reason\n");
  }

    Issues: CCBC-204 CCBC-205

Fixes in 2.2.0

  • libuv plugin: use the same CRT for free and malloc.

    Issues : CCBC-286

  • Fail NOT_MY_VBUCKET responses on time-out.

    Issues : CCBC-288

  • Do a full purge when negotiation times out. In this case we must purge the server from all commands and not simply pop individual items.

    Issues : CCBC-275

  • Reset the server’s buffers upon reconnection. This fixes a crash experienced when requesting a new read with the previous buffer still intact. This was exposed by calling lcb_failout_server on a time-out error while maintaining the same server struct.

    Issues : CCBC-275

  • Make server buffers reentrant-safe. When purging implicit commands, we invoke callbacks that might in turn cause other LCB entry points to be invoked, which can shift the contents or positions of the ring buffers that we’re reading from.

    Issues : CCBC-282

  • Use common config retry mechanism for bad configuration cache. This uses the same error handling mechanism as when a bad configuration has been received from the network. New LCB_CONFIG_CACHE_INVALID error code to notify the user of such a situation

    Issues : CCBC-278

  • Handle getl and unl when purging the server (thanks Robert Groenenberg).

    Issues : CCBC-274

  • Don’t fail out all commands on a time-out. Only fail those commands that are old enough to have timed out already.

  • Don’t record and use TTP/TTR from observe. Just poll at a fixed interval, because the responses from the server side can be unreliable.

    Issues : CCBC-269

  • Allow hooks for mapping server codes to errors. This also helps handle sane behavior if a new error code is introduced, or allow user-defined logging when a specific error code is received.

      lcb_errmap_callback default_callback;

  lcb_error_t user_map_error(lcb_t instance, lcb_uint16_t in)
  {
    if (in == PROTOCOL_BINARY_RESPONSE_ETMPFAIL) {
      fprintf(stderr, "temporary failure on server\n");
    }
    return default_callback(instance, in);
  }

  ...

  default_callback = lcb_set_errmap_callback(conn, user_map_error);

  • Add an example of a connection pool. See example/instancepool directory

  • Force lcb_wait return result of wait operation instead of lcb_get_last_error. It returns last_error if and only if the handle is not yet configured

    Issues : CCBC-279

  • cbc-pillowfight: compute item size correctly during set If minSize and maxSize are set to the same value it can sometimes crash since it may try to read out of memory bounds from the allocated data buffer.

    Issues : CCBC-284

  • Apply key prefix CLI option in cbc-pillowfight

    Issues : CCBC-283

  • Add --enable-maintainer-mode. Maintainer mode enables --enable-werror --enable-warnings --enable-debug, forces all plugins to be installed and forces all tests, tools, and examples to be built

  • Expose LCB_MAX_ERROR to allow user-defined codes

    Issues : CCBC-255

Release Notes for Couchbase Client Library C 2.1.3 GA (10 September 2013)

New Features and Behavior Changes in 2.1.3

  • Use cluster type connection for cbc-bucket-flush. Although flush command is accessible for bucket type connections, cbc-bucket-flush doesn’t use provided bucket name to connect to, therefore it will fail if the bucket name isn’t “default”.

  • Allow to make connect order deterministic. It allows the user to toggle between deterministic and random connect order for the supplied nodes list. By default it will randomize the list.

Fixes in 2.1.3

  • Updated gtest to version 1.7.0. Fixes issue with building test suite with new XCode 5.0 version being released later this month.

  • Do not try to parse config for LCB_TYPE_CLUSTER handles. It fixes timouts for management operations (like ‘cbc bucket-create’, ‘cbc bucket-flush’, ‘cbc bucket-delete’ and ‘cbc admin’)

    Issues : CCBC-265

  • Skip unfinished SASL commands on rebalance. During rebalance, it is possible that the newly added server doesn’t have chance to finish SASL auth before the cluster will push config update, in this case packet relocator messing cookies. Also the patch makes sure that SASL command/cookie isn’t mixing with other commands

    Issues : CCBC-263

  • Do not allow to use Administrator account for LCB_TYPE_BUCKET

  • Fig segmentation faults during tests load of node.js. Sets inside_handler on socket_connected. Previously we were always using SASL auth, and as such, we wouldn’t flush packets from the cmd_log using server_send_packets (which calls apply_want). apply_want shouldn’t be called more than once per event loop entry – so this sets and unsets the inside_handler flag.

    Issues : CCBC-258

  • Added support of libuv 0.8

  • Close config connection before trying next node. It will fix asserts in case of the config node becomes unresponsive, and the threshold controlled by LCB_CNTL_CONFERRTHRESH and lcb_cntl(3)

Release Notes for Couchbase Client Library C 2.1.2 GA (27 August 2013)

Fixes in 2.1.2

  • Use bucket name in SASL if username omitted. Without this fix, you can may encounter a segmentation faults for buckets, which are not protected by a password.

    Issues : CCBC-253 CCBC-254

  • Preserve IO cookie in options_from_info when using v0 plugins with user-provided IO loop instance. This issue was introduced in 2.1.0.

  • Display the effective IO backend in cbc-version. This is helpful to quickly detect what is the effective IO plugin on a given system.

Release Notes for Couchbase Client Library C 2.1.1 GA (22 August 2013)

New Features and Behavior Changes in 2.1.1

  • Fallback to ‘select’ IO plugin if default plugin cannot be loaded. On UNIX-like systems, default IO backend is ‘libevent’, which uses third-party library might be not available at the run-time. Read in lcb_cntl(3couchbase) man page in section LCB_CNTL_IOPS_DEFAULT_TYPES about how to determine effective IO plugin, when your code chose to use LCB_IO_OPS_DEFAULT during connection instantiation. The fallback mode doesn’t affect application which specify IO backend explicitly.

    Issues : CCBC-246

  • Skip misconfigured nodes in the list. New lcb_cntl(3couchbase) added to control whether the library will skip nodes in initial node list, which listen on configuration port (8091 usually) but doesn’t meet required parameters (invalid authentication or missing bucket). By default report this issue and stop trying nodes from the list, like all previous release. Read more at man page lcb_cntl(3couchbase) in section LCB_CNTL_SKIP_CONFIGURATION_ERRORS_ON_CONNECT

    Issues : CCBC-192

  • Distribute debug information with release binaries on Windows

    Issues : CCBC-245

Fixes in 2.1.1

  • Do not use socket after failout. Fixes segmentation faults during rebalance.

    Issues : CCBC-239

  • Use provided credentials for authenticating to the data nodes. With this fix, it is no longer possible to use Administrator credentials with a bucket. If your configuration does so, you must change the credentials you use before applying this update. No documentation guides use of Administrator credentials, so this change is not expected to affect few, if any deployments.

  • Do not disable config.h on UNIX-like platforms. It fixes build issue, when application is trying to include plugins from the tarball.

    Issues : CCBC-248

Release Notes for Couchbase Client Library C 2.1.0 GA (18 August 2013)

New Features and Behavior Changes in 2.1.0

  • New backend select. This backend is based on the select(2) system call and its Windows version. It could be considered the most portable solution and is available with the libcouchbase core.

  • API for durability operations. This new API is based on lcb_observe(3) and allows you to monitor keys more easily. See the man pages lcb_durability_poll(3) and lcb_set_durability_callback(3) for more info.

    Issues : CCBC-145

  • New backend libuv. This backend previously was part of the couchnode project and is now available as a plugin. Because libuv doesn’t ship binary packages there is no binary package libcouchbase2-libuv. You can build plugin from the source distribution, or through the libcouchbase-dev or libcouchbase-devel package on UNIX like systems.

    Issues : CCBC-236

  • New backend iocp. This is a Windows specific backend, which uses “I/O Completion Ports”. As a part of the change, a new version of plugin API was introduced which is more optimized to this model of asynchronous IO.

  • New configuration interface lcb_cntl(3) along with new tunable options of the library and connection instances. In this release the following settings are available. See the man page for more information and examples.:

    • LCB_CNTL_OP_TIMEOUT operation timeout (default 2.5 seconds)

    • LCB_CNTL_CONFIGURATION_TIMEOUT time to fetch cluster configuration. This is similar to a connection timeout (default 5 seconds)

    • LCB_CNTL_VIEW_TIMEOUT timeout for couchbase views (default 75 seconds)

    • LCB_CNTL_HTTP_TIMEOUT timeout for other HTTP operations like RESTful flush, bucket creating etc. (default 75 seconds)

    • LCB_CNTL_RBUFSIZE size of the internal read buffer (default 32768 bytes)

    • LCB_CNTL_WBUFSIZE size of the internal write buffer (default 32768 bytes)

    • LCB_CNTL_HANDLETYPE type of the lcb\_t handler (readonly)

    • LCB_CNTL_VBCONFIG returns pointer to VBUCKET_CONFIG_HANDLE (readonly)

    • LCB_CNTL_IOPS get the implementation of IO (lcb_io_opt_t)

    • LCB_CNTL_VBMAP get vBucket ID for a given key

    • LCB_CNTL_MEMDNODE_INFO get memcached node info

    • LCB_CNTL_CONFIGNODE_INFO get config node info

    • LCB_CNTL_SYNCMODE control synchronous behavior (default LCB_ASYNCHRONOUS)

    • LCB_CNTL_IP6POLICY specify IPv4/IPv6 policy (default LCB_IPV6_DISABLED)

    • LCB_CNTL_CONFERRTHRESH control configuration error threshold (default 100)

    • LCB_CNTL_DURABILITY_TIMEOUT durability timeout (default 5 seconds)

    • LCB_CNTL_DURABILITY_INTERVAL durability polling interval (default 100 milliseconds)

    • LCB_CNTL_IOPS_DEFAULT_TYPES get the default IO types

    • LCB_CNTL_IOPS_DLOPEN_DEBUG control verbose printing of dynamic loading of IO plugins.

Fixes in 2.1.0

  • Fixed bug when REPLICA_SELECT didn’t invoke callbacks for negative error codes

    Issues : CCBC-228

  • Fixed bug when LCB_REPLICA_FIRST fails if first try does not return key

    Issues : CCBC-229

Known Issues in 2.1.0

  • From the release the 2.1.0 package libcouchbase2 will not install an IO backend automatically. If you are upgrading, there are no changes because you have already libcouchbase2-libev or libcouchbase2-libevent packages installed. For new installations, a backend must be selected for the client library to work correctly.

    If for example you are using the PHP SDK, the old way, which works for pre-2.1.0 versions is:

    # DEB-based systems
 shell> sudo apt-get install libcouchbase2 libcouchbase-dev
 # RPM-based systems
 shell> sudo yum install libcouchbase2 libcouchbase-devel

    But a more explicit way to do this, which works for all versions (including 2.1.0) is:

    # DEB-based systems
  shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
  # RPM-based systems
  shell> sudo yum install libcouchbase2-libevent libcouchbase-devel

Release Notes for Couchbase Client Library C 2.0.7 GA (10 July 2013)

New Features and Behavior Changes in 2.0.7

  • Improve lcb\_get\_replica(). Now it is possible to choose between three strategies:

    1. LCB_REPLICA_FIRST : Previously accessible and now the default, the caller will get a reply from the first replica to successfully reply within the timeout for the operation or will receive an error.

    2. LCB_REPLICA_ALL : Ask all replicas to send documents/items back.

    3. LCB_REPLICA_SELECT : Select one replica by the index in the configuration starting from zero. This approach can more quickly receive all possible replies for a given topology, but it can also generate false negatives.

    Note that applications should not assume the order of the replicas indicates more recent data is at a lower index number. It is up to the application to determine which version of a document/item it may wish to use in the case of retrieving data from a replica.

    Issues : CCBC-183

Release Notes for Couchbase Client Library C 2.0.6 GA (07 May 2013)

New Features and Behavior Changes in 2.0.6

Fixes in 2.0.6

  • Fix segfault when rebalancing. When a (!connected) server is reconnected, the tasks in its “pending” buffer will be moved into “output” buffer. If its connection is broken again immediately, relocate_packets() will go to wrong path.

    Issues : CCBC-188

  • Don’t try to switch to backup nodes when timeout is reached

    Issues : CCBC-202

  • Fix compile error with sun studio. "src/event.c", line 172: error: statement not reached (E_STATEMENT_NOT_REACHED)

  • Don’t invoke HTTP callbacks after cancellation, because user code might assume a previously-freed resource is still valid

  • Check if SASL struct is valid before disposing

    Issues : CCBC-188

  • example/yajl/couchview.c: pass cookie to the command Fixes coredump when executing./examples/yajl/couchview

Release Notes for Couchbase Client Library C 2.0.5 GA (05 April 2013)

New Features and Behavior Changes in 2.0.5

  • pillowfight example updated to optionally use threads

Fixes in 2.0.5

Release Notes for Couchbase Client Library C 2.0.4 GA (06 March 2013)

Fixes in 2.0.4

  • Build error on solaris/sparc: -Werror=cast-align

    Issues : CCBC-178

  • Fixed illegal memory access in win32 plugin

    Issues : CCBC-147

  • Work properly on systems where EWOULDBLOCK != EAGAIN

    Issues : CCBC-175

  • The library stops iterating backup nodes list if the next one isn’t accessible.

    Issues : CCBC-182

  • The bootstrap URI is not parsed correctly

    Issues : CCBC-185

  • Segmentation fault when the hostname resolved into several addresses and first of them reject couchbase connections.

    Issues : CCBC-180

Release Notes for Couchbase Client Library C 2.0.3 GA (06 February 2013)

New Features and Behavior Changes in 2.0.3

  • Add a new library: libcouchbase_debug.so (see include/libcouchbase/debug.h) which is a new library that contains new debug functionality.

  • Added manual pages for the library.

Fixes in 2.0.3

  • Observe malfunctions in the case of multiple keys and server failure.

    Issues : CCBC-155

  • Reset internal state on lcb_connect(). Allow caller to use lcb_connect() multiple times to implement reconnecting using the same lcb_t instance. Also it sets up the initial-connection timer for users who don’t use lcb_wait() and drive IO loop manually.

    Issues : CCBC-153

  • Invalid read in libevent plugin, when the plugin compiled in 1.x mode

    Issues : CCBC-171

  • Shrink internal lookup tables (and reduce the size of lcb_t)

  • Issues : CCBC-156

Release Notes for Couchbase Client Library C 2.0.2 GA (04 January 2013)

Fixes in 2.0.2

  • Document LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH. Enhance the lcb_strerror test to detect undocumented error codes.

  • Commands sent to multiple servers fail to detect the respose if mixed with other commands.

    Issues : CCBC-150

  • Under high load the library could generate LCB_ETIMEDOUT errors without reason owing to internal limitations.

    Issues : CCBC-153

  • Cancellation of the HTTP request might lead to memory leaks or to segfaults (2e3875c2).

    Issues : CCBC-151

Release Notes for Couchbase Client Library C 2.0.1 GA (11 December 2012)

New Features and Behavior Changes in 2.0.1

  • SystemTap and DTrace integration

Fixes in 2.0.1

  • Fix a memory leak on the use of http headers

    Issues : CCBC-130

  • libev-plugin: delay all timers while the loop isn’t active. It will fix LCB_ETIMEOUT in the following scenario:

    • connect the instance

    • sleep for time greater than default timeout (e.g. 3 seconds)

    • schedule and execute a command (it will be timed out immediately)

  • Do not abort when purging SASL commands

    Issues : CCBC-136

  • Fix possible SEGFAULT. Not-periodic timers are destroyed after calling user’s callback, after that library performed read from freed pointer.

  • Compensate for cluster nodes lacking couchApiBase

    Issues : CCBC-131

  • Ensure HTTP works even when the network may be unreliable. This changeset encompasses several issues which had been found with HTTP requests during network errors and configuration changes. Specifically some duplicate code paths were removed, and the process for delivering an HTTP response back to the user is more streamlined.

    Issues : CCBC-132, CCBC-133

  • libev-plugin: reset IO event on delete. We need to reset it, because it might be re-used later

  • Make library C89 friendly again

Release Notes for Couchbase Client Library C 2.0.0 GA (27 November 2012)

New Features and Behavior Changes in 2.0.0

  • Add the CAS to the delete callback

Fixes in 2.0.0

  • Minor update of the packaging layout:

    • libcouchbase-all package comes without version

    • extract debug symbols from libcouchbase-{bin,core} to libcouchbase-dbg package

  • Install unlock callback in synchronous mode

Release Notes for Couchbase Client Library C 2.0.0beta3 Beta (21 November 2012)

New Features and Behavior Changes in 2.0.0beta3

  • Try all known plugins for LCB_IO_OPS_DEFAULT in run time

  • Allow to use ‘cbc-hash’ with files

  • Create man pages for cbc and cbcrc

  • Use dynamic versioning for plugins

  • Lookup the plugin symbol also in the current executable image.

    Issues : CCBC-114

  • Allow the user to specify a different hash key. All of the data operations contains a hashkey and nhashkey field. This allows you to “group” items together in your cluster. A typical use case for this is if you’re storing lets say data for a single user in multiple objects. If you want to ensure that either all or none of the objects are available if a server goes down, it could be a good idea to locate them on the same server. Do bear in mind that if you do try to decide where objects is located, you may end up with an uneven distribution of the number of items on each node. This will again result in some nodes being more busy than others etc. This is why some clients doesn’t allow you to do this, so bear in mind that by doing so you might not be able to get your objects from other clients.

    Issues : CCBC-119

  • Add documentation about the error codes

    Issues : CCBC-87

  • Add lcb_verify_compiler_setup(). This function allows the “user” of the library to verify that the compiler use a compatible struct packing scheme.

Fixes in 2.0.0beta3

  • lcb_error_t member in the http callbacks shouldn’t reflect the HTTP response code. So the error code will be always LCB_SUCCESS if the library managed to receive the data successfully.

    Issues : CCBC-118

  • Fix cbc-bucket-create. `sasl-password' is misspelled, and it fails to parse the command line option.

  • Remove libtool version from the plugins

  • Do not allow admin operations without authentication

  • check for ewouldblock/eintr on failed send

  • Purge stale OBSERVE packets

    Issues : CCBC-120

  • Allow to use gethrtime() from C++

  • Remove unauthorized asserion (d344037). The lcb_server_send_packets() function later check if the server object connected and establish connection if not (with raising possible errors)

    Issues : CCBC-113

  • Don’t use the time_t for win32. When compiling from php it turns out that it gets another size of the time_t type, causing the struct offsets to differ.

  • Reformat and refactor lcb_server_purge_implicit_responses:

    • move packet allocation out of GET handler

    • dropping NOOP command shouldn’t return error code

    Issues : CCBC-120

  • Try to switch another server from backup list on timeout

    Issues : CCBC-122

  • Timer in libev uses double for interval. Ref: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#code_ev_timer_code_relative_and_opti

  • Fix illegal memory access. Reconnect config listener if the config connection was gone without proper shutdown.

    Issues : CCBC-104

  • Fix using freed memory (was introduced in 4397181)

  • Return zero from do_read_data() if operations_per_call reached. The `operations_per_call' limit was introduced to prevent from freezing event loop. But in the function variable rv could store two different results and in case of reaching this limit it is returning number of the processed records, which is wrong. The function should return either zero (success) or non-zero (failure).

    Issues : CCBC-115

Release Notes for Couchbase Client Library C 2.0.0beta2 Beta (12 October 2012)

New Features and Behavior Changes in 2.0.0beta2

  • Implement a new libev plugin. It is compatible with both libev3 and libev4.

  • Allow libcouchbase to connect to an instance without specifying bucket. It is useful when the bucket not needed, e.g. when performing administration tasks.

  • Allow users to build the library without dependencies. For example, without plugins at all. This may be useful if the plugin is implemented by or built into the host application.

  • Allow users to use environment variables to pick the event plugin

  • Add a new interface version for creating IO objects via plugins

  • Allow to disable CXX targets

  • Allow users to install both libraries (2.x and 1.x) on the same system.

  • Cleanup HTTP callbacks. Use the same callbacks both for Management and View commands, and rename them to lcb_http_complete_callback and lcb_http_data_callback.

  • Add support for raw http requests. libcouchase already contains all the bits to execute a raw http request, except for the possibility to specify a host:port, username and password.

  • Make the content type optional for lcb_make_http_request()

Fixes in 2.0.0beta2

  • Fix invalid memory access in cbc tool. Affected command is cbc-bucket-create

  • Search ev.h also in ${includedir}/libev

  • Fix password memory leak in http.c (7e71493)

  • lcb_create: replace assert() with error code

  • Breakout event loop in default error_callback. This provides better default behavior for users who haven’t defined global error callback.

    Issues : CCBC-105

  • Fix linked event/timer lists for win32

    Issues : CCBC-103

  • Fix SEGFAULT if IO struct is allocated not by the lcb_create()

  • Fix memory leak after an unsuccessful connection

  • lcb_connect() should honor the syncmode setting. Automatically call lcb_wait() when in synchronous mode

Release Notes for Couchbase Client Library C 2.0.0beta Beta (13 September 2012)

New Features and Behavior Changes in 2.0.0beta

  • Bundle Windows packages as zip archives

  • Refactor the API. This is a full redesign of the current libcouchbase API that’ll allow us to extend parts of the API without breaking binary compatibility. Also it renames all functions to have lcb prefix instead of libcouchbase and LCB/LIBCOUCHBASE in macros.

  • Implement getter for number of nodes in the cluster: lcb_get_num_nodes()

  • Add lcb_get_server_list() to get current server list

  • Deliver HTTP headers via callbacks

  • Implement RESTful flush in the cbc toolset

  • Merge lcb_get_locked into lcb_get function

  • Bundle libvbucket

Fixes in 2.0.0beta

  • Fix a problem with allocating too few slots in the backup_nodes. Fixes illegal memory access.

  • Include sys/uio.h. Needed by OpenBSD

  • Added --enable-fat-binary. Helps to solve issues when linking with fat binaries on MacOS.

  • Differentiate between TMPFAILs. This allows a developer to know if the temporary condition where the request cannot be handled is due to a constraint on the client or the server.

    Issues : CCBC-98

  • Correct buffer length for POST/PUT headers

    Issues : CCBC-96

  • Fix switching to backup node in case of server outage

    Issues : CCBC-91

  • Fix locking keys in multi-get mode

  • Release the memory allocated by the http parser

    Issues : CCBC-89

  • Fix initialization of backup nodes array. The code switching nodes relies on NULL terminator rather than nbackup_nodes variable. Fixes illegal memory access.

    Issues : CCBC-90

  • Default to IPv4 only

    Issues : CCBC-80

  • Reset timer for commands with NOT_MY_VBUCKET response

    Issues : CCBC-91

  • Sync memcached/protocol_binary.h. Pull extra protocol_binary_datatypes declarations.

  • Fix bug where HTTP method is not set

  • Don’t try to put the current node last in the backup list. This may cause “duplicates” in the list if the REST server returns another name for the server than you used. Ex: you specify “localhost” and the REST response contains 127.0.0.1

  • Release ringbuffer in lcb_purge_single_server

    Issues : CCBC-92

Release Notes for Couchbase Client Library C 1.1.0dp9 Developer Preview (27 July 2012)

Fixes in 1.1.0dp9

  • Render auth credentials for View requests. libcouchbase_make_http_request() won’t accept credentials anymore. It will pick them bucket configuration.

Release Notes for Couchbase Client Library C 1.1.0dp8 Developer Preview (27 July 2012)

New Features and Behavior Changes in 1.1.0dp8

  • Allow the user to get the number of replicas using libcouchbase_get_num_replicas()

  • Separate HTTP callbacks for couch and management requests

  • Implement read replica

    Issues : CCBC-82

  • Let users detect if the event loop running already using libcouchbase_is_waiting() function.

  • Add OBSERVE command

    Issues : CCBC-15

  • Allow users to specify content type for HTTP request.

  • Allow a user to breakout from the event loop in callbacks using libcouchbase_breakout()

  • New cbc commands and options:

    • cbc-view (remove couchview example)

    • cbc-verbosity

    • cbc-admin

    • cbc-bucket-delete

    • cbc-bucket-create

    • Add -p and -r options to cbc-cp to control persistence (uses OBSERVE internally)

  • Allow the client to specify the verbosity level on the servers using lcb_set_verbosity() function.

  • Implement general purpose timers. It is possible for users to define their own timers using libcouchbase_timer_create() function. (See headers for more info). Implement multiple timers for windows

    Issues : CCBC-85

Fixes in 1.1.0dp8

  • Claim that server has data in buffers if there are HTTP requests pending. Without this patch the event loop can be stopped prematurely.

  • Make libcouchbase_wait() re-entrable

  • Fix to handle the case when View base doesn’t have URI schema.

  • Bind timeouts to server sockets instead of commands. This means that from this point timeout interval will be started from the latest IO activity on the socket. This is a behavior change from the 1.0 series.

  • Use separate error code for ENOMEM on the client

    Issues : CCBC-77

Release Notes for Couchbase Client Library C 1.1.0dp7 Developer Preview (19 June 2012)

New Features and Behavior Changes in 1.1.0dp7

  • Implement function to execution management requests. Using libcouchbase_make_management_request() function you can configure the cluster, add/remove buckets, rebalance etc. It behaves like libcouchbase_make_couch_request() but works with another endpoint.

  • Add support for notification callbacks for configuration changes. Now it is possible to install a hook using function libcouchbase_set_configuration_callback(), and be notified about all configuration changes.

Fixes in 1.1.0dp7

  • Extract HTTP client. Backward incompatible change in Couchbase View subsystem

Release Notes for Couchbase Client Library C 1.1.0dp6 Developer Preview (13 June 2012)

New Features and Behavior Changes in 1.1.0dp6

  • Implement ‘help’ command for cbc tool

    Issues : RCBC-71

Fixes in 1.1.0dp6

  • Undefine NDEBUG to avoid asserts to be optimized out

  • Fix compilation on macosx with gtest from homebrew

    Issues : RCBC-72

  • Close dynamic libraries. Fixes small memory leak

    Issues : RCBC-70

  • Include types definitions for POSIX systems. Fixes C++ builds on some systems.

    Issues : RCBC-63

  • Fix win32 builds:

    • Add suffix to cbc command implementations;

    • Fix guards for socket errno macros;

    • Define size_t types to fix MSVC 9 build;

    • MSVC 9 isn’t C99, but has stddef.h, so just include it.

Release Notes for Couchbase Client Library C 1.1.0dp5 Developer Preview (06 June 2012)

New Features and Behavior Changes in 1.1.0dp5

  • Implement ‘cbc-hash’ to get server/vbucket for given key

Fixes in 1.1.0dp5

  • The library doesn’t depend on pthreads (eliminates package lint warnings)

Release Notes for Couchbase Client Library C 1.1.0dp4 Developer Preview (05 June 2012)

Fixes in 1.1.0dp4

  • cbc: strtoull doesn’t exist on win32, therefore use C++ equivalent.

Release Notes for Couchbase Client Library C 1.1.0dp3 Developer Preview (03 June 2012)

New Features and Behavior Changes in 1.1.0dp3

  • Implement GET_LOCKED (GETL) command

    Issues : CCBC-68

  • Implement UNLOCK_KEY (UNL) command

    Issues : CCBC-68

Fixes in 1.1.0dp3

  • Timeouts can occur during topology changes, rather than be correctly retried. Send the retry-packet to new server

    Issues : CCBC-64

  • Fix ringbuffer_memcpy() (36afdb2). Fix ringbuffer_is_continous().

  • A hang could occur in libcouchbase_wait() after the timeout period. Check for breakout condition after purging servers

    Issues : CCBC-62

  • Destroy view requests items when server get destroyed. Fixes memory leaks.

  • A fix for a buffer overflow with the supplied password as has been integrated. While it is a buffer overflow issue, this is not considered to be a possible security issue because the password to the bucket is not commonly supplied by an untrusted source

    Issues : RCBC-33

  • A small memory leak can occur with frequent calls to libcouchbase_create() and libcouchbase_destroy()

    Issues : CCBC-65

  • Use vbucket_found_incorrect_master() to get correct server index during relocating buffers. Pick up cookies from pending buffer unless node connected.

  • Do not call View callbacks for cancelled requests

  • hashset.c: iterate over whole set on rehashing. Fixes memory leaks related to hash collisions (905ef95)

Release Notes for Couchbase Client Library C 1.1.0dp2 Developer Preview (10 April 2012)

Fixes in 1.1.0dp2

  • Don’t wait for empty buffers. If called with no operations queued, libcouchbase_wait() will block forever. This means that a single threaded application that calls libcouchbase_wait() at different times to make sure operations are sent to the server runs the risk of stalling indefinitely. This is a very likely scenario.

    Issues : CCBC-59

  • Don’t define size_t and ssize_t for VS2008

  • Fix segfault while authorizing on protected buckets (211bb04)

Release Notes for Couchbase Client Library C 1.1.0dp Developer Preview (05 April 2012)

New Features and Behavior Changes in 1.1.0dp

  • This release adds new functionality to directly access Couchbase Server views using the libcouchbase_make_couch_request() function. See the associated documentation and header files for more details.

Fixes in 1.1.0dp

  • Request the tap bytes in a known byte order (adf2b30)

    Issues : MB-4834

  • Check for newer libvbucket