A newer version of this software is available
You are viewing the documentation for an older version of this software. To find the documentation for the current version, visit the Couchbase documentation home page.
This guide provides information for developers who want to use the Couchbase C SDK to build applications that use Couchbase Server.
Now that you’ve installed Couchbase and have created a cluster of servers, you need a client library to read and write data from the cluster. The Couchbase C client library, also known as libcouchbase, can be used from your application to access Couchbase Server.
Here’s a quick outline of what you’ll learn in this chapter:
Get the client library.
Build the client library from source (optional).
Write a simple program to connecting to Couchbase and save some data.
Download the latest client library from the Couchbase download
site. It is available as a source
archive in both .zip
and .tar.gz
Packages are provided for RedHat and Ubuntu Linux systems by providing your package manager with the Couchbase repository information. The method for installation is dependent on your platform:
You must populate RPM with a new source, which is dependent on your RedHat version:
RHEL/CentOS 5.5
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos55-i386.repo
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos55-x86_64.repo
RHEL/CentOS 6.2
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos62-i686.repo
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos62-x86_64.repo
Then to install libcouchbase with libevent backend, run:
shell> sudo yum check-update
shell sudo yum install -y libcouchbase2-libevent libcouchbase-devel
You must update the apt-get
repository to install the client library:
Ubuntu 12.04 Precise Pangolin (Debian unstable)
Also compatible with recent versions, which have libevent2
.
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1204.list
Ubuntu 11.10 Oneiric Ocelot (Debian unstable)
Also compatible with recent versions, which have libevent2
.
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1110.list
Ubuntu 10.04 Lucid Lynx (Debian stable or testing)
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1004.list
Also make sure you have the GPG key installed:
shell> wget -O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add -
Then to install libcouchbase with libevent backend, run:
shell> sudo apt-get update
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
This client library is available via a homebrew recipe. After you install homebrew, install libcouchbase:
shell> brew install \
https://github.com/couchbase/homebrew/raw/stable/Library/Formula/libcouchbase.rb
Installation is like many common libraries, following the ./configure
, make
,
make install
conventions.
For libvbucket, extract the archive then cd
into the directory and run:
shell> ./configure
shell> make install
Standard configure options such as --prefix
can be passed to the configure
command. For additional information on configuration options, run./configure
with –help.
For libcouchbase, extract the archive, then cd into the directory and run:
shell> ./configure CPPFLAGS="-I/opt/couchbase/include" --disable-couchbasemock
shell> make install
The --disable-couchbasemock
simply disables some tests which are common during
the development of libcouchbase, but not required when installing a release.
Building and installing on Microsoft Windows requires nmake
and tools in
Microsoft Visual Studio 2010.
Open the Visual Studio Command Prompt and navigate to the directory for the
extracted archive for libcouchbase. The NMakefile defines an INSTALL
variable
as C:\local
. Edit the NMakefile if you want to change the installation
location. Then build and install libcouchbase:
shell> nmake -f NMakefile install
From libcouchbase version 2.1.0, you can also use CMake system to generate correct MS Visual Studio project, for example:
shell> cmake -G "Visual Studio 10"
The C client library, libcouchbase
, is a callback-oriented client which makes
it very easy to write high performance programs. There are a few ways you can
drive IO with the library. The simplest approach is to use the synchronous
interface over the asynch internals of the library. More advanced programs will
either call the libcouchbase_wait()
function after generating some operations
or drive the event loop themselves.
To connect, you first configure the connection options and then create an instance of the connection to the cluster:
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);
/*
* Initiate the connect sequence in libcouchbase
*/
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Run the event loop and wait until we've connected */
lcb_wait(instance);
Callbacks are used by the library and are simple functions which handle the result of operations. For example:
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);
/*
* Initiate the connect sequence in libcouchbase
*/
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Run the event loop and wait until we've connected */
lcb_wait(instance);
Callbacks can be set up for all of your operations called in libcouchbase. In the API, you’ll note the use of a cookie. This is metadata from your application which is associated with the request. The libcouchbase library will not inspect any cookie or send the cookie to the server.
When you put the connect logic and the get callback together and plug them into a complete program with the include headers, you get:
#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
static void error_callback(lcb_t instance,
lcb_error_t err,
const char *errinfo)
{
fprintf(stderr, "Error %s: %s", lcb_strerror(instance, err),
errinfo ? errinfo : "");
exit(EXIT_FAILURE);
}
/* the callback invoked by the library when receiving a get response */
static void get_callback(lcb_t instance,
const void *cookie,
lcb_error_t error,
const lcb_get_resp_t *resp)
{
if (error != LCB_SUCCESS) {
fprintf(stderr, "Failed to retrieve \"");
fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
fprintf(stderr, "\": %s\n", lcb_strerror(instance, error));
} else {
fprintf(stderr, "Data for key: \"");
fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
fprintf(stderr, "\" is : ");
fwrite(resp->v.v0.bytes, 1, resp->v.v0.nbytes, stderr);
}
(void)cookie; /* ignore */
}
int main(void)
{
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* set up the handler to catch all errors */
lcb_set_error_callback(instance, error_callback);
/* initiate the connect sequence in libcouchbase */
err = lcb_connect(instance);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* run the event loop and wait until we've connected */
lcb_wait(instance);
/* set up a callback for our get requests */
lcb_set_get_callback(instance, get_callback);
{
lcb_get_cmd_t cmd;
const lcb_get_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
err = lcb_get(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to get: %s\n",
lcb_strerror(NULL, err));
return 1;
}
}
lcb_wait(instance);
lcb_destroy(instance);
exit(EXIT_SUCCESS);
}
To compile this sample program, you must link to libcouchbase
:
shell> gcc -o hellocb -lcouchbase hellocb.c
This tutorial assumes you have installed libcouchbase on your systems per the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.
If you need to set up a server node or data bucket, you can do so with Couchbase Administrative Console, Couchbase Command-Line Interface (CLI), or Couchbase REST API. For information and instructions, see:
Using the Couchbase Web Console, for information on using the Couchbase Administrative Console,
Couchbase CLI, for the command line interface,
Couchbase REST API, for creating and managing Couchbase resources.
After you’ve set up your Couchbase Server and installed the needed client libraries, you can compile and run the following basic program.
Before getting into a more complex example of the programming model to this library, we will walk through a straightforward example of a program which builds with libcouchbase, connects to a server, and sets and gets a value:
/* -*- Mode: C; tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- */
/*
* Copyright 2012 Couchbase, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
* BUILD:
*
* cc -o minimal minimal.c -lcouchbase
* cl /DWIN32 /Iinclude minimal.c lib\libcouchbase.lib
*
* RUN:
*
* valgrind -v --tool=memcheck --leak-check=full --show-reachable=yes ./minimal
* ./minimal <host:port> <bucket> <passwd>
* mininal.exe <host:port> <bucket> <passwd>
*/
#include <stdio.h>
#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#ifdef _WIN32
#define PRIu64 "I64u"
#else
#include <inttypes.h>
#endif
static void error_callback(lcb_t instance, lcb_error_t error, const char *errinfo)
{
fprintf(stderr, "ERROR: %s (0x%x), %s\n",
lcb_strerror(instance, error), error, errinfo);
exit(EXIT_FAILURE);
}
static void store_callback(lcb_t instance, const void *cookie,
lcb_storage_t operation,
lcb_error_t error,
const lcb_store_resp_t *item)
{
if (error == LCB_SUCCESS) {
fprintf(stderr, "STORED \"");
fwrite(item->v.v0.key, sizeof(char), item->v.v0.nkey, stderr);
fprintf(stderr, "\" CAS: %"PRIu64"\n", item->v.v0.cas);
} else {
fprintf(stderr, "STORE ERROR: %s (0x%x)\n",
lcb_strerror(instance, error), error);
exit(EXIT_FAILURE);
}
(void)cookie;
(void)operation;
}
static void get_callback(lcb_t instance, const void *cookie, lcb_error_t error,
const lcb_get_resp_t *item)
{
if (error == LCB_SUCCESS) {
fprintf(stderr, "GOT \"");
fwrite(item->v.v0.key, sizeof(char), item->v.v0.nkey, stderr);
fprintf(stderr, "\" CAS: %"PRIu64" FLAGS:0x%x SIZE:%lu\n",
item->v.v0.cas, item->v.v0.flags, (unsigned long)item->v.v0.nbytes);
fwrite(item->v.v0.bytes, sizeof(char), item->v.v0.nbytes, stderr);
fprintf(stderr, "\n");
} else {
fprintf(stderr, "GET ERROR: %s (0x%x)\n",
lcb_strerror(instance, error), error);
}
(void)cookie;
}
int main(int argc, char *argv[])
{
lcb_error_t err;
lcb_t instance;
struct lcb_create_st create_options;
struct lcb_create_io_ops_st io_opts;
io_opts.version = 0;
io_opts.v.v0.type = LCB_IO_OPS_DEFAULT;
io_opts.v.v0.cookie = NULL;
memset(&create_options, 0, sizeof(create_options));
err = lcb_create_io_ops(&create_options.v.v0.io, &io_opts);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create IO instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
if (argc > 1) {
create_options.v.v0.host = argv[1];
}
if (argc > 2) {
create_options.v.v0.user = argv[2];
create_options.v.v0.bucket = argv[2];
}
if (argc > 3) {
create_options.v.v0.passwd = argv[3];
}
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
(void)lcb_set_error_callback(instance, error_callback);
/* Initiate the connect sequence in libcouchbase */
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
lcb_destroy(instance);
return 1;
}
(void)lcb_set_get_callback(instance, get_callback);
(void)lcb_set_store_callback(instance, store_callback);
/* Run the event loop and wait until we've connected */
lcb_wait(instance);
{
lcb_store_cmd_t cmd;
const lcb_store_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.operation = LCB_SET;
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
cmd.v.v0.bytes = "bar";
cmd.v.v0.nbytes = 3;
err = lcb_store(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to set: %s\n", lcb_strerror(NULL, err));
return 1;
}
}
lcb_wait(instance);
{
lcb_get_cmd_t cmd;
const lcb_get_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
err = lcb_get(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to get: %s\n", lcb_strerror(NULL, err));
return 1;
}
}
lcb_wait(instance);
lcb_destroy(instance);
return 0;
}
In addition to the other sections of this manual, such as the Getting Started guide and the API reference, the libcouchbase package includes an examples directory and a tools directory. Each of these show simple Couchbase tools and an example libcouchbase programs.
On Linux you may find these examples in TODO…
Starting from the version 2.2 libcouchbase supports CRAM-MD5 authentication mechanism. Which allows to avoid passing bucket password as a plain text over the wires.
Along with this change, new setting was introduced
LCB_CNTL_FORCE_SASL_MECH
. It forces a specific SASL mechanism to use
for authentication. This can allow a user to ensure a certain level of
security and have the connection fail if the desired mechanism is not available.
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_FORCE_SASL_MECH, "CRAM-MD5");
This tutorial assumes you have installed libcouchbase on your systems, following the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.
The libcouchbase is written in C and can be integrated in various ways with your application. The simplest integration scenario is when your application is written in a scripting language (supported by a Couchbase client library) that already has a libcouchbase wrapper. Couchbase has built and maintains following libcouchbase-based libraries:
Users of these libraries automatically get an API, which looks natural for their platform, so that it doesn’t require knowledge of libcouchbase APIs. This tutorial is mostly about two other use cases:
Add libcouchbase into an existing application to implement persistence layer. This section will contain two parts: first we will create a simple echo server written using libev library. Second, we will show how to persist each message going through the server to Couchbase.
Get complete code here: https://github.com/couchbaselabs/libev-couchbase-example
Build applications around libcouchbase. Here we’ll show how to build a proxy server for regular memcached clients. Building this proxy server is just like building a moxi server, but the proxy server will be more limited more limited in abilities.
Get complete code here: https://github.com/couchbaselabs/libcouchbase-proxy-sample
Let’s start with a specific existing application for this example. Our application is the asynchronous single-threaded echo server, which leverages libev to solve C10K problem and efficiently serves a huge number of concurrent connections. It should listen to the given port (in this sample 4567) and send back everything the client submitted to it.
The libev library implements reactor
pattern which uses the term
event loop
to represent an entity which encapsulates the register of handlers
for various IO events and the timers. To start, we have the application code
from the main function:
int main()
{
struct ev_loop *loop = ev_default_loop(0);
int sd;
struct sockaddr_in addr;
struct ev_io server;
if ((sd = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
fail("socket error");
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(PORT_NO);
addr.sin_addr.s_addr = INADDR_ANY;
if (bind(sd, (struct sockaddr *) &addr, sizeof(addr)) != 0) {
fail("bind error");
}
if (listen(sd, 2) < 0) {
fail("listen error");
}
ev_io_init(&server, accept_cb, sd, EV_READ);
ev_io_start(loop, &server);
printf("Listen on %d\n", PORT_NO);
ev_run(loop, 0);
return 0;
}
Here we are create an event loop instance, and create and bind a new socket to
local port PORT_NO (4567)
. After that we can mark this socket as listening and
ask libev to call the function accept_cb
when the socket become ready to read,
that is when the new client will come to us. The final step is to run ev_run()
which does all the server interaction.
The accept_cb
is in charge of allocating client structures 60 the buffer for
messages. The sample is using ring buffers from the libcouchbase code for easier
managing memory for IO buffers.
void accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
struct client_s *client;
int flags;
if (EV_ERROR & revents) {
fail("got invalid event");
}
client = malloc(sizeof(struct client_s));
if (!client) {
fail("client watcher alloc");
}
if (!ringbuffer_initialize(&client->buf, BUFFER_SIZE)) {
fail("client buffer alloc");
}
client->naddr = sizeof(client->addr);
client->fd = accept(watcher->fd, (struct sockaddr *)&client->addr,
&client->naddr);
if (client->fd < 0) {
fail("accept error");
}
if ((flags = fcntl(client->fd, F_GETFL, NULL)) == -1) {
fail("fcntl(F_GETFL)");
}
if (fcntl(client->fd, F_SETFL, flags | O_NONBLOCK) == -1) {
fail("fcntl(F_SETFL, O_NONBLOCK)");
}
printf("Successfully connected with client\n");
ev_io_init((ev_io *)client, client_cb, client->fd, EV_READ);
ev_io_start(loop, (ev_io *)client);
}
The most interesting part of the function above is in the last two lines. It
registers the new handler, client_cb
, upon reading an event for the client
socket. This handler will be called each time when the kernel receives new data
from the socket, so that it could be read with a non-blocking call.
The last snippet of our application depicts the client_cb
function, which
handles all the application logic. It reads everything from the socket if it is
ready, registers itself for write events, and then sends all the contents out to
the network. It is also the handling end of a stream situation, which means that
the client closed the connection with the Couchbase server.
void client_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
struct lcb_iovec_st iov[2];
ssize_t nbytes;
struct client_s *client = (struct client_s *)watcher;
if (EV_READ & revents) {
ringbuffer_ensure_capacity(&client->buf, BUFFER_SIZE);
ringbuffer_get_iov(&client->buf, RINGBUFFER_WRITE, iov);
nbytes = io_recvv(watcher->fd, iov);
if (nbytes < 0) {
fail("read error");
} else if (nbytes == 0) {
ev_io_stop(loop, watcher);
ringbuffer_destruct(&client->buf);
free(client);
printf("Peer disconnected\n");
return;
} else {
ringbuffer_produced(&client->buf, nbytes);
printf("Received %zu bytes\n", nbytes);
ev_io_stop(loop, watcher);
ev_io_set(watcher, watcher->fd, EV_WRITE);
ev_io_start(loop, watcher);
}
} else if (EV_WRITE & revents) {
ringbuffer_get_iov(&client->buf, RINGBUFFER_READ, iov);
nbytes = io_sendv(watcher->fd, iov);
if (nbytes < 0) {
fail("write error");
} else if (nbytes == 0) {
ev_io_stop(loop, watcher);
ringbuffer_destruct(&client->buf);
free(client);
printf("Peer disconnected\n");
return;
} else {
ringbuffer_consumed(&client->buf, nbytes);
printf("Sent %zu bytes\n", nbytes);
ev_io_stop(loop, watcher);
ev_io_set(watcher, watcher->fd, EV_READ);
ev_io_start(loop, watcher);
}
} else {
fail("got invalid event");
}
}
Again, the whole application is stored on github at https://github.com/couchbaselabs/libev-couchbase-example. Let’s see it in action. To clone and build the applications, use the following commands:
~ $ git clone git://github.com/couchbaselabs/libev-couchbase-example.git
~ $ cd libev-couchbase-example/step1
step1 $ ./autogen.sh && ./configure && make
As a client for an echo application, we can build any telnet-like application. The following listing simulates two terminals by splitting them with a vertical line:
step 1 $ ./server | step1 $ telnet localhost 4567
Listen on 4567 | Trying 127.0.0.1...
Successfully connected with client | Connected to localhost.
Received 14 bytes | Escape character is '^]'.
Sent 14 bytes | Hello world!
Received 9 bytes | Hello world!
Sent 9 bytes | foo bar
Peer disconnected | foo bar
| ^]
| telnet> Connection closed.
Now that we described our application, let’s integrate Couchbase as the database
using libcouchbase. The server.c
file will have some small additions. In the
main
function, before initializing server socket, we will call storage_init
and assign the connection information to the server structure variable.
server.handle = storage_init(loop, "localhost:8091", "default", null);
Also in the client_cb
handler, each time libcouchbase receives data from the
client, it will send the data to Couchbase server with storage_put
where
the_key
is a string "example"
.
storage_put(client, the_key, val, nbytes);
Here, the initial application from step one is already using its own custom event loop to handle other IO in the application. The application could be more complex, but libcouchbase has a separate interface to inject your IO implementation into the library. In the simplest case, for example if your application is using one of the IO libraries supported by libcouchbase, you can just pass your event loop instance into the initializer, and libcouchbase won’t create new one. Instead it will register all its events on this external loop. In case your application is using its own implementation of a reactor pattern, such as nginx, or you use linux epoll API, you can easily write your own IO plugin and pass it to the connection initializer.
The echo server is using libev library, and libcouchbase provides a plugin to
libev out-of-the-box. In this case we can imagine that it doesn’t already have
it. In this case, we can copy the plugin from libcouchbase distribution into the
directory step2/lcb-plugin/
and update build files appropriately.
Now we inspect the storage.c
, where our storage_init()
and storage_put()
defined.
lcb_t storage_init(struct ev_loop *loop, const char *host, const char *bucket, const char *password)
{
struct lcb_create_st opts;
struct lcb_create_io_ops_st io_opts;
lcb_t handle;
lcb_error_t err;
io_opts.version = 1;
io_opts.v.v1.sofile = NULL;
io_opts.v.v1.symbol = "lcb_create_libev_io_opts";
io_opts.v.v1.cookie = loop;
opts.version = 0;
opts.v.v0.host = host;
opts.v.v0.bucket = bucket;
opts.v.v0.user = bucket;
opts.v.v0.passwd = password;
err = lcb_create_io_ops(&opts.v.v0.io, &io_opts);
if (err != LCB_SUCCESS) {
error_callback(NULL, err, "failed to create IO object");
return NULL;
}
err = lcb_create(&handle, &opts);
if (err != LCB_SUCCESS) {
error_callback(NULL, err, "failed to create connection object");
return NULL;
}
(void)lcb_set_error_callback(handle, error_callback);
(void)lcb_set_store_callback(handle, storage_callback);
err = lcb_connect(handle);
if (err != LCB_SUCCESS) {
error_callback(handle, err, "failed to connect to the server");
return NULL;
}
return handle;
}
The end of the function may look familiar, but the io_opts
structure is more
interesting. It defines how to look up our custom IO plugin. The
io_opts.v.v1.sofile
is set to NULL
to specify that our plugin compiled into
the current executable image. In this case, symbol
is the string name of the
function with the following signature:
lcb_error_t lcb_create_libev_io_opts(int version, lcb_io_opt_t *io, void *loop);
This initializes the IO plugin and then the cookie
will be passed to this
function as the loop
argument. You can refer to the lcb_create_io_ops(3)
manpage for other ways to initialize IO subsystem.
The libcouchbase client is purely asynchronous; therefore, each data operation isplits into two parts: a function-scheduler, which validates and copies all arguments to internal buffers for further handing, and a function-callback, which will be called with the results of the operation. It is possible to implement a wrapper which will provide a more synchronous API, but it is difficult to write a library API that is both generic and efficient. Since all communication is done by using function arguments, it is easier to maintain backward compatible APIs by versioning both incoming structures and results. Let’s see how we did it in our sample:
void storage_put(struct client_s *client, const char *key,
const void *val, size_t nval)
{
lcb_error_t err;
lcb_store_cmd_t cmd;
const lcb_store_cmd_t *cmds[] = { &cmd };
memset(&cmd, 0, sizeof(cmd));
cmd.version = 0;
cmd.v.v0.key = key;
cmd.v.v0.nkey = strlen(key);
cmd.v.v0.bytes = val;
cmd.v.v0.nbytes = nval;
cmd.v.v0.operation = LCB_SET;
err = lcb_store(client->handle, client, 1, cmds);
if (err != LCB_SUCCESS) {
error_callback(client->handle, err, "failed to schedule store operation");
}
}
void storage_callback(lcb_t instance, const void *cookie,
lcb_storage_t operation, lcb_error_t error,
const lcb_store_resp_t *resp)
{
struct client_s *client = (struct client_s *)cookie;
struct ev_io* watcher = (struct ev_io*)client;
char cas_str[128];
ssize_t nb;
nb = snprintf(cas_str, 128, "%"PRIu64"\n", resp->v.v0.cas);
if (nb < 0) {
fail("output CAS value");
}
ringbuffer_ensure_capacity(&client->out, nb);
if (ringbuffer_write(&client->out, cas_str, nb) != nb) {
fail("write CAS into the buffer");
}
ev_io_stop(client->loop, watcher);
ev_io_set(watcher, watcher->fd, EV_WRITE);
ev_io_start(client->loop, watcher);
(void)operation;
(void)resp;
}
Here, the storage_put()
function is a wrapper over lcb_store(3)
. The
storage_put()
function translates arguments to the versioned structure and
runs lcb_store
to pass it to the library. The data won’t be sent immediately,
but when the library connects to the data socket, it will be ready to accept
data. After the server processes the request, the application will be notified
asynchronously via storage_callback
. This callback will copy CAS value to the
output buffer to be sent to the client.
Time to demonstrate an application. This application is built the same as
before, except you need to navigate to step2/
directory. As previously noted,
the script will be split to demonstrate both server and the client:
step2 $ ./server | step2 $ telnet localhost 4567
Listen on 4567 | Trying 127.0.0.1...
Successfully connected with client | Connected to localhost.
Received 14 bytes | Escape character is '^]'.
Sent 20 bytes | Hello world!
Peer disconnected | 2916447493390860288
| ^]
| telnet> Connection closed.
Now you can install cbc
tools from package libcouchbase2-bin
. If you
installed libcouchbase from source, you most likely have it already. Then you
can check that it created a key example and that the CAS values are matching:
step2 $ printf "%x\n" 2916447493390860288
28794d8b11a40000
step2 $ cbc cat example
"example" Size:14 Flags:0 CAS:28794d8b11a40000
Hello world!
You can use libcouchbase when you are building Couchbase-enabled application and want to bootstrap quickly. To do so you can build your application using the IO abstraction provided with libcouchbase.
In this example, we create a thin proxy application which will expose the memcached-compatible API for the system. Similar approaches exist for moxi server. Also this application will use less hard-coded logic and more look like real-life service. Full source code can be found at https://github.com/couchbaselabs/libcouchbase-proxy-sample.
For clarity, we skip parsing and initialization of the Couchbase connection from
the main
function. The proxy code begins in run_proxy
, which is similar to
the main function in the previous section. The important difference here is that
we are using the IO abstraction from libcouchbase, to register events, and drive
an event loop.
void run_proxy(lcb_t conn)
{
lcb_io_opt_t io;
struct sockaddr_in addr;
int sock, rv;
server_t server;
io = opts.v.v0.io;
info("starting proxy on port %d", port);
sock = socket(PF_INET, SOCK_STREAM, 0);
if (sock == -1) {
fail("socket()", strerror(errno));
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
addr.sin_addr.s_addr = INADDR_ANY;
rv = bind(sock, (struct sockaddr *)&addr, sizeof(addr));
if (rv == -1) {
fail("bind()");
}
rv = listen(sock, 10);
if (rv == -1) {
fail("listen()");
}
server.conn = conn;
server.io = io;
server.event = io->v.v0.create_event(io);
if (server.event == NULL) {
fail("failed to create event for proxy");
}
io->v.v0.update_event(io, sock, server.event, LCB_READ_EVENT,
&server, proxy_accept_callback);
lcb_set_error_callback(conn, error_callback);
lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);
info("use ctrl-c to stop");
io->v.v0.run_event_loop(io);
}
As in the previous example, we create a listening socket, and then we bind a
handler to process read events. The handler is proxy_accept_callback
, and it
will be triggered for each new client. In turn it will create a new client
structure, defined as follows:
typedef struct client_st client_t;
struct client_st {
int id;
server_t *server;
int sock;
ringbuffer_t in;
ringbuffer_t out;
void *event;
};
This initializes input and output buffers,and makes the socket descriptors
non-blocking. At the end, the code will register the handler function
proxy_client_callback
for read events, and then wait for incoming data.
The proxy implements a very limited number of protocol commands: GET, SET, and VERSION. But other commands can be easily added. Here’s an example:
void proxy_client_callback(lcb_socket_t sock, short which, void *data)
{
struct lcb_iovec_st iov[2];
ssize_t rv;
lcb_io_opt_t io;
client_t *cl = data;
io = cl->server->io;
if (which & LCB_READ_EVENT) {
for (;;) {
/* read in chunks of BUFFER_SIZE */
ringbuffer_ensure_capacity(&cl->in, BUFFER_SIZE);
ringbuffer_get_iov(&cl->in, RINGBUFFER_WRITE, iov);
rv = io->v.v0.recvv(io, cl->sock, iov, 2);
if (rv == -1) {
if (io->v.v0.error == EINTR) {
/* interrupted by signal */
continue;
} else if (io->v.v0.error == EWOULDBLOCK) {
/* nothing to read right now */
io->v.v0.update_event(io, cl->sock, cl->event,
LCB_WRITE_EVENT, cl,
proxy_client_callback);
break;
} else {
fail("read error");
}
} else if (rv == 0) {
/* end of stream */
io->v.v0.destroy_event(io, cl->event);
ringbuffer_destruct(&cl->in);
ringbuffer_destruct(&cl->out);
close(cl->sock);
info("[%d] disconnected", cl->id);
free(cl);
return;
} else {
ringbuffer_produced(&cl->in, rv);
for (;;) {
protocol_binary_request_header req;
lcb_size_t nr, sz;
char *buf;
/* make sure the buffer is aligned */
if (ringbuffer_ensure_alignment(&cl->in) != 0) {
fail("cannot align the buffer");
}
/* take the packet header from the buffer */
nr = ringbuffer_peek(&cl->in, req.bytes, sizeof(req));
if (nr < sizeof(req)) {
break;
}
/* make sure the buffer has whole the body */
sz = ntohl(req.request.bodylen) + sizeof(req);
if (cl->in.nbytes < sz) {
break;
}
/* copy packet into intermediate buffer */
buf = malloc(sizeof(char) * sz);
if (buf == NULL) {
fail("cannot allocate buffer for packet");
}
nr = ringbuffer_read(&cl->in, buf, sz);
if (nr < sizeof(req)) {
fail("input buffer doesn't contain enough data");
}
/* handle packet and deallocate the intermediate
* buffer */
handle_packet(cl, buf);
free(buf);
}
}
}
}
if (which & LCB_WRITE_EVENT) {
/* check if we have something to send */
ringbuffer_get_iov(&cl->out, RINGBUFFER_READ, iov);
if (iov[0].iov_len + iov[1].iov_len == 0) {
io->v.v0.delete_event(io, cl->sock, cl->event);
return;
}
rv = io->v.v0.sendv(io, cl->sock, iov, 2);
if (rv < 0) {
fail("write error");
} else if (rv == 0) {
io->v.v0.destroy_event(io, cl->event);
ringbuffer_destruct(&cl->in);
ringbuffer_destruct(&cl->out);
close(cl->sock);
info("[%d] disconnected", cl->id);
free(cl);
return;
} else {
ringbuffer_consumed(&cl->out, rv);
io->v.v0.update_event(io, cl->sock, cl->event,
LCB_READ_EVENT, cl,
proxy_client_callback);
}
}
(void)sock;
}
The function proxy_client_callback
logically divided into two parts.
The first part occurs when the event loop notifies us that the socket is ready
for non-blocking reading. At this point, the function tries to read all the data
available on the socket. It finds the packets and passes aligned byte arrays to
handle_packet
. Once the code gets a special code from the IO subsystem get
from the IO subsystem special code, that it cannot read anything without
blocking, it stops processing and registers itself for write events, because in
nearest future, responses will appear in the output buffer.
The second part of the function reacts to write events and is much simpler. It just checks to see if there is data in the output buffer. If there is data in the output buffer, it will try to send that data to the client. If not, the functionwill unregister itself from write events to save CPU cycles.
The function handle_packet
is a good place to start playing with this example
if you’d like to add new features there. It decodes protocol packet and
translates it into libcouchbase calls.
void handle_packet(client_t *cl, char *buf)
{
protocol_binary_request_header *req = (void *)buf;
protocol_binary_response_header res;
union {
lcb_get_cmd_t get;
lcb_store_cmd_t set;
} cmd;
union {
const lcb_get_cmd_t *get[1];
const lcb_store_cmd_t *set[1];
} cmds;
cookie_t *cookie;
cookie = malloc(sizeof(cookie_t));
if (cookie == NULL) {
fail("cannot allocate buffer for command cookie");
}
cookie->client = cl;
cookie->opaque = req->request.opaque;
cookie->opcode = req->request.opcode;
memset(&cmd, 0, sizeof(cmd));
switch (req->request.opcode) {
case PROTOCOL_BINARY_CMD_GET:
cmds.get[0] = &cmd.get;
cmd.get.v.v0.nkey = ntohs(req->request.keylen);
cmd.get.v.v0.key = buf + sizeof(*req);
info("[%d] get \"%.*s\"", cl->id,
(int)cmd.get.v.v0.nkey, (char *)cmd.get.v.v0.key);
lcb_get(cl->server->conn, (const void*)cookie, 1, cmds.get);
break;
case PROTOCOL_BINARY_CMD_SET:
cmds.set[0] = &cmd.set;
cmd.set.v.v0.operation = LCB_SET;
cmd.set.v.v0.nkey = ntohs(req->request.keylen);
cmd.set.v.v0.key = buf + sizeof(*req) + 8;
cmd.set.v.v0.nbytes = ntohl(req->request.bodylen) - 8 - ntohs(req->request.keylen);
cmd.set.v.v0.bytes = buf + sizeof(*req) + 8 + ntohs(req->request.keylen);
cmd.set.v.v0.cas = req->request.cas;
cmd.set.v.v0.datatype = req->request.datatype;
memcpy(&cmd.set.v.v0.flags, buf + sizeof(*req), 4);
cmd.set.v.v0.flags = ntohl(cmd.set.v.v0.flags);
memcpy(&cmd.set.v.v0.exptime, buf + sizeof(*req) + 4, 4);
cmd.set.v.v0.exptime = ntohl(cmd.set.v.v0.exptime);
info("[%d] set \"%.*s\"", cl->id,
(int)cmd.set.v.v0.nkey, (char *)cmd.set.v.v0.key);
lcb_store(cl->server->conn, (const void*)cookie, 1, cmds.set);
break;
case PROTOCOL_BINARY_CMD_VERSION:
free(cookie);
info("[%d] version", cl->id);
res.response.magic = PROTOCOL_BINARY_RES;
res.response.opcode = req->request.opcode;
res.response.keylen = 0;
res.response.extlen = 0;
res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
res.response.status = PROTOCOL_BINARY_RESPONSE_SUCCESS;
res.response.bodylen = htonl(sizeof(version_msg));
res.response.opaque = req->request.opaque;
res.response.cas = 0;
ringbuffer_ensure_capacity(&cl->out, sizeof(res));
ringbuffer_write(&cl->out, res.bytes, sizeof(res));
ringbuffer_write(&cl->out, version_msg, sizeof(version_msg));
break;
default:
free(cookie);
info("[%d] unsupported command: 0x%02x", cl->id, req->request.opcode);
res.response.magic = PROTOCOL_BINARY_RES;
res.response.opcode = req->request.opcode;
res.response.keylen = 0;
res.response.extlen = 0;
res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
res.response.status = PROTOCOL_BINARY_RESPONSE_NOT_SUPPORTED;
res.response.bodylen = htonl(sizeof(notsup_msg));
res.response.opaque = req->request.opaque;
res.response.cas = 0;
ringbuffer_ensure_capacity(&cl->out, sizeof(notsup_msg) + sizeof(res));
ringbuffer_write(&cl->out, res.bytes, sizeof(res));
ringbuffer_write(&cl->out, notsup_msg, sizeof(notsup_msg));
}
}
To maintain a logical link between requests and responses for the clients, the
protocol defines a special field opaque
, which can be a special tag or
sequence number. To preserve this field in the response, we will put it into a
cookie_t
structure. During the proxy initialization, we defined two
callback-functions for libcouchbase:
lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);
In this kind of application, these functions just build a protocol response from
libcouchbase return value. They also register the proxy_client_callback
on
write events to send out data recently written into the buffer.
void get_callback(lcb_t conn, const void *cookie, lcb_error_t err,
const lcb_get_resp_t *item)
{
cookie_t *c = (cookie_t *)cookie;
client_t *cl = c->client;
lcb_io_opt_t io = cl->server->io;
protocol_binary_response_get res;
res.message.header.response.magic = PROTOCOL_BINARY_RES;
res.message.header.response.opcode = PROTOCOL_BINARY_CMD_GET;
res.message.header.response.keylen = 0;
res.message.header.response.extlen = 4;
res.message.header.response.datatype = item->v.v0.datatype;
res.message.header.response.status = htons(map_status(err));
res.message.header.response.bodylen = htonl(4 + item->v.v0.nbytes);
res.message.header.response.opaque = c->opaque;
res.message.header.response.cas = item->v.v0.cas;
res.message.body.flags = htonl(item->v.v0.flags);
ringbuffer_ensure_capacity(&cl->out, sizeof(res.bytes) + item->v.v0.nbytes);
ringbuffer_write(&cl->out, res.bytes, sizeof(res.bytes));
ringbuffer_write(&cl->out, item->v.v0.bytes, item->v.v0.nbytes);
io->v.v0.update_event(io, cl->sock, cl->event, LCB_WRITE_EVENT,
cl, proxy_client_callback);
free(c);
(void)conn;
}
These are viritually all the integration points with libcouchbase. As we demonstrated, libcouchbase more than a protocol parser; it also an abstract and portable IO framework, allowing you to build asynchronous applications around Couchbase Server.
To conclude, let’s run your favourite memcached client and verify our proxy. First you need to clone and build it:
$ git clone git://github.com/couchbaselabs/libcouchbase-proxy-sample.git
$ cd libcouchbase-proxy-sample
libcouchbase-proxy-sample $ make
Run the server and send couple of commands to it. We will use the dalli ruby gem here:
libcouchbase-proxy-sample $ ./proxy | $ irb
connecting to bucket "default" at "localhost:8091" | irb:001:0> require 'dalli'
starting proxy on port 1987 | true
use ctrl-c to stop | irb:002:0> m = Dalli::Client.new "localhost:1987"
[1] connected | #<Dalli::Client:0x007f716321ce20 @servers=["localhost:1987"], @options={}, @ring=nil>
[1] version | irb:003:0> m.set("foo", "bar")
[1] set "foo" | true
[1] get "foo" | irb:004:0> m.get("foo")
[1] disconnected | "bar"
The following sections provide release notes for individual release versions of Couchbase Client Library C. To browse or submit new issues, see Couchbase Client Library C Issues Tracker.
New Features and Behavior Changes in 2.2.0
Handle 302 redirects in HTTP (views and administrative requests). By
default the library follows up to three redirects. After the
limit is reached, the request is terminated with code
LCB_TOO_MANY_REDIRECTS
. The limit is configurable through
LCB_CNTL_MAX_REDIRECTS
. If it is set to -1, it disables the redirect
limit. The following example shows how to set the limit:
int new_value = 5;
lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_MAX_REDIRECTS, &new_value);
Issues: CCBC-169
Replace isasl with cbsasl. cbsasl implements both PLAIN and CRAM-MD5 authentication mechanisms.
LCB_CNTL_MEMDNODE_INFO
command updated to include effective
SASL mechanism:
cb_cntl_server_t node;
node.version = 1;
node.v.v1.index = 0; /* first node */
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_MEMDNODE_INFO, &node);
if (node.v.v1.sasl_mech) {
printf("authenticated via SASL '%s'\n",
node.v.v1.sasl_mech);
}
You can force a specific authentication mechanism for
the connection handle by using the LCB_CNTL_FORCE_SASL_MECH
command:
lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_FORCE_SASL_MECH, "PLAIN");
Issues: CCBC-243
Stricter, more inspectable behavior for the configuration cache. This provides a
test and an additional lcb_cntl
operation to check the status of
the configuration cache. Also, it switches off the configuration cache with
memcached buckets.
int is_loaded;
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
if (is_loaded) {
printf("Configuration cache saved us a trip to the config server\n");
} else {
printf("We had to contact the configuration server for some reason\n");
}
Fixes in 2.2.0
libuv plugin: use the same CRT for free and malloc.
Issues : CCBC-286
Fail NOT_MY_VBUCKET
responses on time-out.
Issues : CCBC-288
Do a full purge when negotiation times out. In this case we must purge the server from all commands and not simply pop individual items.
Issues : CCBC-275
Reset the server’s buffers upon reconnection. This fixes a crash
experienced when requesting a new read with the previous buffer
still intact. This was exposed by calling lcb_failout_server
on a
time-out error while maintaining the same server struct.
Issues : CCBC-275
Make server buffers reentrant-safe. When purging implicit commands, we invoke callbacks that might in turn cause other LCB entry points to be invoked, which can shift the contents or positions of the ring buffers that we’re reading from.
Issues : CCBC-282
Use common config retry mechanism for bad configuration cache. This uses the
same error handling mechanism as when a bad configuration has been
received from the network. New LCB_CONFIG_CACHE_INVALID
error code
to notify the user of such a situation
Issues : CCBC-278
Handle getl and unl when purging the server (thanks Robert Groenenberg).
Issues : CCBC-274
Don’t fail out all commands on a time-out. Only fail those commands that are old enough to have timed out already.
Don’t record and use TTP/TTR from observe. Just poll at a fixed interval, because the responses from the server side can be unreliable.
Issues : CCBC-269
Allow hooks for mapping server codes to errors. This also helps handle sane behavior if a new error code is introduced, or allow user-defined logging when a specific error code is received.
lcb_errmap_callback default_callback;
lcb_error_t user_map_error(lcb_t instance, lcb_uint16_t in)
{
if (in == PROTOCOL_BINARY_RESPONSE_ETMPFAIL) {
fprintf(stderr, "temporary failure on server\n");
}
return default_callback(instance, in);
}
...
default_callback = lcb_set_errmap_callback(conn, user_map_error);
Add an example of a connection pool. See example/instancepool
directory
Force lcb_wait
return result of wait operation instead of
lcb_get_last_error
. It returns last_error
if and only if the
handle is not yet configured
Issues : CCBC-279
cbc-pillowfight
: compute item size correctly during set If
minSize
and maxSize
are set to the same value it can sometimes
crash since it may try to read out of memory bounds from the
allocated data buffer.
Issues : CCBC-284
Apply key prefix CLI option in cbc-pillowfight
Issues : CCBC-283
Add --enable-maintainer-mode
. Maintainer mode enables
--enable-werror --enable-warnings --enable-debug
, forces all
plugins to be installed and forces all tests, tools, and examples to
be built
Expose LCB_MAX_ERROR
to allow user-defined codes
Issues : CCBC-255
New Features and Behavior Changes in 2.1.3
Use cluster type connection for cbc-bucket-flush. Although flush command is accessible for bucket type connections, cbc-bucket-flush doesn’t use provided bucket name to connect to, therefore it will fail if the bucket name isn’t “default”.
Allow to make connect order deterministic. It allows the user to toggle between deterministic and random connect order for the supplied nodes list. By default it will randomize the list.
Fixes in 2.1.3
Updated gtest to version 1.7.0. Fixes issue with building test suite with new XCode 5.0 version being released later this month.
Do not try to parse config for LCB_TYPE_CLUSTER
handles. It fixes timouts for management operations (like ‘cbc
bucket-create’, ‘cbc bucket-flush’, ‘cbc bucket-delete’ and ‘cbc
admin’)
Issues : CCBC-265
Skip unfinished SASL commands on rebalance. During rebalance, it is possible that the newly added server doesn’t have chance to finish SASL auth before the cluster will push config update, in this case packet relocator messing cookies. Also the patch makes sure that SASL command/cookie isn’t mixing with other commands
Issues : CCBC-263
Do not allow to use Administrator account for
LCB_TYPE_BUCKET
Fig segmentation faults during tests load of
node.js. Sets inside_handler
on socket_connected
. Previously we
were always using SASL auth, and as such, we wouldn’t flush packets
from the cmd_log
using server_send_packets
(which calls
apply_want
). apply_want
shouldn’t be called more than once per
event loop entry – so this sets and unsets the inside_handler
flag.
Issues : CCBC-258
Added support of libuv 0.8
Close config connection before trying next node. It will fix
asserts in case of the config node becomes unresponsive, and the
threshold controlled by LCB_CNTL_CONFERRTHRESH
and lcb_cntl(3)
Fixes in 2.1.2
Use bucket name in SASL if username omitted. Without this fix, you can may encounter a segmentation faults for buckets, which are not protected by a password.
Preserve IO cookie in options_from_info
when using v0 plugins
with user-provided IO loop instance. This issue was introduced in
2.1.0.
Display the effective IO backend in cbc-version
. This is helpful
to quickly detect what is the effective IO plugin on a given
system.
New Features and Behavior Changes in 2.1.1
Fallback to ‘select’ IO plugin if default plugin cannot be loaded. On UNIX-like systems, default IO backend is ‘libevent’, which uses third-party library might be not available at the run-time. Read in lcb_cntl(3couchbase) man page in section LCB_CNTL_IOPS_DEFAULT_TYPES about how to determine effective IO plugin, when your code chose to use LCB_IO_OPS_DEFAULT during connection instantiation. The fallback mode doesn’t affect application which specify IO backend explicitly.
Issues : CCBC-246
Skip misconfigured nodes in the list. New lcb_cntl(3couchbase) added to control whether the library will skip nodes in initial node list, which listen on configuration port (8091 usually) but doesn’t meet required parameters (invalid authentication or missing bucket). By default report this issue and stop trying nodes from the list, like all previous release. Read more at man page lcb_cntl(3couchbase) in section LCB_CNTL_SKIP_CONFIGURATION_ERRORS_ON_CONNECT
Issues : CCBC-192
Distribute debug information with release binaries on Windows
Issues : CCBC-245
Fixes in 2.1.1
Do not use socket after failout. Fixes segmentation faults during rebalance.
Issues : CCBC-239
Use provided credentials for authenticating to the data nodes. With this fix, it is no longer possible to use Administrator credentials with a bucket. If your configuration does so, you must change the credentials you use before applying this update. No documentation guides use of Administrator credentials, so this change is not expected to affect few, if any deployments.
Do not disable config.h on UNIX-like platforms. It fixes build issue, when application is trying to include plugins from the tarball.
Issues : CCBC-248
New Features and Behavior Changes in 2.1.0
New backend select
. This backend is based on the select(2) system call and its
Windows version. It could be considered the most portable solution and is
available with the libcouchbase core.
API for durability operations. This new API is based on lcb_observe(3)
and
allows you to monitor keys more easily. See the man pages
lcb_durability_poll(3)
and lcb_set_durability_callback(3)
for more info.
Issues : CCBC-145
New backend libuv
. This backend previously was part of the
couchnode project and is now available
as a plugin. Because libuv doesn’t ship binary packages there is no binary
package libcouchbase2-libuv
. You can build plugin from the source
distribution, or through the libcouchbase-dev
or libcouchbase-devel
package
on UNIX like systems.
Issues : CCBC-236
New backend iocp
. This is a Windows specific backend, which uses “I/O
Completion Ports”. As a part of the change, a new version of plugin API was
introduced which is more optimized to this model of asynchronous IO.
New configuration interface lcb_cntl(3)
along with new tunable options of the
library and connection instances. In this release the following settings are
available. See the man page for more information and examples.:
LCB_CNTL_OP_TIMEOUT operation timeout (default 2.5 seconds)
LCB_CNTL_CONFIGURATION_TIMEOUT time to fetch cluster configuration. This is similar to a connection timeout (default 5 seconds)
LCB_CNTL_VIEW_TIMEOUT timeout for couchbase views (default 75 seconds)
LCB_CNTL_HTTP_TIMEOUT timeout for other HTTP operations like RESTful flush, bucket creating etc. (default 75 seconds)
LCB_CNTL_RBUFSIZE size of the internal read buffer (default 32768 bytes)
LCB_CNTL_WBUFSIZE size of the internal write buffer (default 32768 bytes)
LCB_CNTL_HANDLETYPE type of the lcb\_t
handler (readonly)
LCB_CNTL_VBCONFIG returns pointer to VBUCKET_CONFIG_HANDLE (readonly)
LCB_CNTL_IOPS get the implementation of IO (lcb_io_opt_t)
LCB_CNTL_VBMAP get vBucket ID for a given key
LCB_CNTL_MEMDNODE_INFO get memcached node info
LCB_CNTL_CONFIGNODE_INFO get config node info
LCB_CNTL_SYNCMODE control synchronous behavior (default LCB_ASYNCHRONOUS)
LCB_CNTL_IP6POLICY specify IPv4/IPv6 policy (default LCB_IPV6_DISABLED)
LCB_CNTL_CONFERRTHRESH control configuration error threshold (default 100)
LCB_CNTL_DURABILITY_TIMEOUT durability timeout (default 5 seconds)
LCB_CNTL_DURABILITY_INTERVAL durability polling interval (default 100 milliseconds)
LCB_CNTL_IOPS_DEFAULT_TYPES get the default IO types
LCB_CNTL_IOPS_DLOPEN_DEBUG control verbose printing of dynamic loading of IO plugins.
Fixes in 2.1.0
Fixed bug when REPLICA_SELECT
didn’t invoke callbacks for negative error codes
Issues : CCBC-228
Fixed bug when LCB_REPLICA_FIRST
fails if first try does not return key
Issues : CCBC-229
Known Issues in 2.1.0
From the release the 2.1.0 package libcouchbase2
will not install an IO
backend automatically. If you are upgrading, there are no changes because you
have already libcouchbase2-libev
or libcouchbase2-libevent
packages
installed. For new installations, a backend must be selected for the client
library to work correctly.
If for example you are using the PHP SDK, the old way, which works for pre-2.1.0 versions is:
# DEB-based systems
shell> sudo apt-get install libcouchbase2 libcouchbase-dev
# RPM-based systems
shell> sudo yum install libcouchbase2 libcouchbase-devel
But a more explicit way to do this, which works for all versions (including 2.1.0) is:
# DEB-based systems
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
# RPM-based systems
shell> sudo yum install libcouchbase2-libevent libcouchbase-devel
New Features and Behavior Changes in 2.0.7
Improve lcb\_get\_replica()
. Now it is possible to choose between three
strategies:
LCB_REPLICA_FIRST
: Previously accessible and now the default, the caller will
get a reply from the first replica to successfully reply within the timeout for
the operation or will receive an error.
LCB_REPLICA_ALL
: Ask all replicas to send documents/items back.
LCB_REPLICA_SELECT
: Select one replica by the index in the configuration
starting from zero. This approach can more quickly receive all possible replies
for a given topology, but it can also generate false negatives.
Note that applications should not assume the order of the replicas indicates more recent data is at a lower index number. It is up to the application to determine which version of a document/item it may wish to use in the case of retrieving data from a replica.
Issues : CCBC-183
New Features and Behavior Changes in 2.0.6
Added an example to properly use the bucket credentials for authentication instead of administrator credentials
Issues : CCBC-179
Add Host header in http request http://cbugg.hq.couchbase.com/bug/bug-555 points out that Host is a required field in HTTP 1.1
Issues : CCBC-201
Fixes in 2.0.6
Fix segfault when rebalancing. When a (!connected) server is reconnected, the tasks in its “pending” buffer will be moved into “output” buffer. If its connection is broken again immediately, relocate_packets() will go to wrong path.
Issues : CCBC-188
Don’t try to switch to backup nodes when timeout is reached
Issues : CCBC-202
Fix compile error with sun studio. "src/event.c", line 172: error: statement
not reached (E_STATEMENT_NOT_REACHED)
Don’t invoke HTTP callbacks after cancellation, because user code might assume a previously-freed resource is still valid
Check if SASL struct is valid before disposing
Issues : CCBC-188
example/yajl/couchview.c: pass cookie to the command Fixes coredump when executing./examples/yajl/couchview
New Features and Behavior Changes in 2.0.5
Fixes in 2.0.5
Try to search the –libdir for modules if dlopen fails to find the module in the default library path
New compat mode (experimental) for configuration caching. See man lcb_create_compat
Issues : CCBC-190
Fix reconnecting issues on windows (http://review.couchbase.org/25170 and http://review.couchbase.org/25155)
Fix build on FreeBSD (http://review.couchbase.org/25289)
Fixes in 2.0.4
Build error on solaris/sparc: -Werror=cast-align
Issues : CCBC-178
Fixed illegal memory access in win32 plugin
Issues : CCBC-147
Work properly on systems where EWOULDBLOCK != EAGAIN
Issues : CCBC-175
The library stops iterating backup nodes list if the next one isn’t accessible.
Issues : CCBC-182
The bootstrap URI is not parsed correctly
Issues : CCBC-185
Segmentation fault when the hostname resolved into several addresses and first of them reject couchbase connections.
Issues : CCBC-180
New Features and Behavior Changes in 2.0.3
Add a new library: libcouchbase_debug.so (see include/libcouchbase/debug.h) which is a new library that contains new debug functionality.
Added manual pages for the library.
Fixes in 2.0.3
Observe malfunctions in the case of multiple keys and server failure.
Issues : CCBC-155
Reset internal state on lcb_connect(). Allow caller to use lcb_connect() multiple times to implement reconnecting using the same lcb_t instance. Also it sets up the initial-connection timer for users who don’t use lcb_wait() and drive IO loop manually.
Issues : CCBC-153
Invalid read in libevent plugin, when the plugin compiled in 1.x mode
Issues : CCBC-171
Shrink internal lookup tables (and reduce the size of lcb_t)
Issues : CCBC-156
Fixes in 2.0.2
Document LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH. Enhance the lcb_strerror test to detect undocumented error codes.
Commands sent to multiple servers fail to detect the respose if mixed with other commands.
Issues : CCBC-150
Under high load the library could generate LCB_ETIMEDOUT errors without reason owing to internal limitations.
Issues : CCBC-153
Cancellation of the HTTP request might lead to memory leaks or to segfaults (2e3875c2).
Issues : CCBC-151
New Features and Behavior Changes in 2.0.1
Fixes in 2.0.1
Fix a memory leak on the use of http headers
Issues : CCBC-130
libev-plugin: delay all timers while the loop isn’t active. It will fix LCB_ETIMEOUT in the following scenario:
connect the instance
sleep for time greater than default timeout (e.g. 3 seconds)
schedule and execute a command (it will be timed out immediately)
Do not abort when purging SASL commands
Issues : CCBC-136
Fix possible SEGFAULT. Not-periodic timers are destroyed after calling user’s callback, after that library performed read from freed pointer.
Compensate for cluster nodes lacking couchApiBase
Issues : CCBC-131
Ensure HTTP works even when the network may be unreliable. This changeset encompasses several issues which had been found with HTTP requests during network errors and configuration changes. Specifically some duplicate code paths were removed, and the process for delivering an HTTP response back to the user is more streamlined.
libev-plugin: reset IO event on delete. We need to reset it, because it might be re-used later
Make library C89 friendly again
New Features and Behavior Changes in 2.0.0
Fixes in 2.0.0
Minor update of the packaging layout:
libcouchbase-all package comes without version
extract debug symbols from libcouchbase-{bin,core} to libcouchbase-dbg package
Install unlock callback in synchronous mode
New Features and Behavior Changes in 2.0.0beta3
Try all known plugins for LCB_IO_OPS_DEFAULT in run time
Allow to use ‘cbc-hash’ with files
Create man pages for cbc and cbcrc
Use dynamic versioning for plugins
Lookup the plugin symbol also in the current executable image.
Issues : CCBC-114
Allow the user to specify a different hash key. All of the data operations contains a hashkey and nhashkey field. This allows you to “group” items together in your cluster. A typical use case for this is if you’re storing lets say data for a single user in multiple objects. If you want to ensure that either all or none of the objects are available if a server goes down, it could be a good idea to locate them on the same server. Do bear in mind that if you do try to decide where objects is located, you may end up with an uneven distribution of the number of items on each node. This will again result in some nodes being more busy than others etc. This is why some clients doesn’t allow you to do this, so bear in mind that by doing so you might not be able to get your objects from other clients.
Issues : CCBC-119
Add documentation about the error codes
Issues : CCBC-87
Add lcb_verify_compiler_setup(). This function allows the “user” of the library to verify that the compiler use a compatible struct packing scheme.
Fixes in 2.0.0beta3
lcb_error_t member in the http callbacks shouldn’t reflect the HTTP response code. So the error code will be always LCB_SUCCESS if the library managed to receive the data successfully.
Issues : CCBC-118
Fix cbc-bucket-create. `sasl-password' is misspelled, and it fails to parse the command line option.
Remove libtool version from the plugins
Do not allow admin operations without authentication
check for ewouldblock/eintr on failed send
Purge stale OBSERVE packets
Issues : CCBC-120
Allow to use gethrtime() from C++
Remove unauthorized asserion (d344037). The lcb_server_send_packets() function later check if the server object connected and establish connection if not (with raising possible errors)
Issues : CCBC-113
Don’t use the time_t for win32. When compiling from php it turns out that it gets another size of the time_t type, causing the struct offsets to differ.
Reformat and refactor lcb_server_purge_implicit_responses:
move packet allocation out of GET handler
dropping NOOP command shouldn’t return error code
Issues : CCBC-120
Try to switch another server from backup list on timeout
Issues : CCBC-122
Timer in libev uses double for interval. Ref: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#code_ev_timer_code_relative_and_opti
Fix illegal memory access. Reconnect config listener if the config connection was gone without proper shutdown.
Issues : CCBC-104
Fix using freed memory (was introduced in 4397181)
Return zero from do_read_data() if operations_per_call reached. The `operations_per_call' limit was introduced to prevent from freezing event loop. But in the function variable rv could store two different results and in case of reaching this limit it is returning number of the processed records, which is wrong. The function should return either zero (success) or non-zero (failure).
Issues : CCBC-115
New Features and Behavior Changes in 2.0.0beta2
Implement a new libev plugin. It is compatible with both libev3 and libev4.
Allow libcouchbase to connect to an instance without specifying bucket. It is useful when the bucket not needed, e.g. when performing administration tasks.
Allow users to build the library without dependencies. For example, without plugins at all. This may be useful if the plugin is implemented by or built into the host application.
Allow users to use environment variables to pick the event plugin
Add a new interface version for creating IO objects via plugins
Allow to disable CXX targets
Allow users to install both libraries (2.x and 1.x) on the same system.
Cleanup HTTP callbacks. Use the same callbacks both for Management and View commands, and rename them to lcb_http_complete_callback and lcb_http_data_callback.
Add support for raw http requests. libcouchase already contains all the bits to execute a raw http request, except for the possibility to specify a host:port, username and password.
Make the content type optional for lcb_make_http_request()
Fixes in 2.0.0beta2
Fix invalid memory access in cbc tool. Affected command is cbc-bucket-create
Search ev.h also in ${includedir}/libev
Fix password memory leak in http.c (7e71493)
lcb_create: replace assert() with error code
Breakout event loop in default error_callback. This provides better default behavior for users who haven’t defined global error callback.
Issues : CCBC-105
Fix linked event/timer lists for win32
Issues : CCBC-103
Fix SEGFAULT if IO struct is allocated not by the lcb_create()
Fix memory leak after an unsuccessful connection
lcb_connect() should honor the syncmode setting. Automatically call lcb_wait() when in synchronous mode
New Features and Behavior Changes in 2.0.0beta
Bundle Windows packages as zip archives
Refactor the API. This is a full redesign of the current libcouchbase API that’ll allow us to extend parts of the API without breaking binary compatibility. Also it renames all functions to have lcb prefix instead of libcouchbase and LCB/LIBCOUCHBASE in macros.
Implement getter for number of nodes in the cluster: lcb_get_num_nodes()
Add lcb_get_server_list() to get current server list
Deliver HTTP headers via callbacks
Implement RESTful flush in the cbc toolset
Merge lcb_get_locked into lcb_get function
Bundle libvbucket
Fixes in 2.0.0beta
Fix a problem with allocating too few slots in the backup_nodes. Fixes illegal memory access.
Include sys/uio.h. Needed by OpenBSD
Added --enable-fat-binary
. Helps to solve issues when linking with fat
binaries on MacOS.
Differentiate between TMPFAILs. This allows a developer to know if the temporary condition where the request cannot be handled is due to a constraint on the client or the server.
Issues : CCBC-98
Correct buffer length for POST/PUT headers
Issues : CCBC-96
Fix switching to backup node in case of server outage
Issues : CCBC-91
Fix locking keys in multi-get mode
Release the memory allocated by the http parser
Issues : CCBC-89
Fix initialization of backup nodes array. The code switching nodes relies on NULL terminator rather than nbackup_nodes variable. Fixes illegal memory access.
Issues : CCBC-90
Default to IPv4 only
Issues : CCBC-80
Reset timer for commands with NOT_MY_VBUCKET response
Issues : CCBC-91
Sync memcached/protocol_binary.h. Pull extra protocol_binary_datatypes declarations.
Fix bug where HTTP method is not set
Don’t try to put the current node last in the backup list. This may cause “duplicates” in the list if the REST server returns another name for the server than you used. Ex: you specify “localhost” and the REST response contains 127.0.0.1
Release ringbuffer in lcb_purge_single_server
Issues : CCBC-92
Fixes in 1.1.0dp9
New Features and Behavior Changes in 1.1.0dp8
Allow the user to get the number of replicas using libcouchbase_get_num_replicas()
Separate HTTP callbacks for couch and management requests
Implement read replica
Issues : CCBC-82
Let users detect if the event loop running already using libcouchbase_is_waiting() function.
Add OBSERVE command
Issues : CCBC-15
Allow users to specify content type for HTTP request.
Allow a user to breakout from the event loop in callbacks using libcouchbase_breakout()
New cbc commands and options:
cbc-view (remove couchview example)
cbc-verbosity
cbc-admin
cbc-bucket-delete
cbc-bucket-create
Add -p and -r options to cbc-cp to control persistence (uses OBSERVE internally)
Allow the client to specify the verbosity level on the servers using lcb_set_verbosity() function.
Implement general purpose timers. It is possible for users to define their own timers using libcouchbase_timer_create() function. (See headers for more info). Implement multiple timers for windows
Issues : CCBC-85
Fixes in 1.1.0dp8
Claim that server has data in buffers if there are HTTP requests pending. Without this patch the event loop can be stopped prematurely.
Make libcouchbase_wait() re-entrable
Fix to handle the case when View base doesn’t have URI schema.
Bind timeouts to server sockets instead of commands. This means that from this point timeout interval will be started from the latest IO activity on the socket. This is a behavior change from the 1.0 series.
Use separate error code for ENOMEM on the client
Issues : CCBC-77
New Features and Behavior Changes in 1.1.0dp7
Implement function to execution management requests. Using libcouchbase_make_management_request() function you can configure the cluster, add/remove buckets, rebalance etc. It behaves like libcouchbase_make_couch_request() but works with another endpoint.
Add support for notification callbacks for configuration changes. Now it is possible to install a hook using function libcouchbase_set_configuration_callback(), and be notified about all configuration changes.
Fixes in 1.1.0dp7
New Features and Behavior Changes in 1.1.0dp6
Implement ‘help’ command for cbc tool
Issues : RCBC-71
Fixes in 1.1.0dp6
Undefine NDEBUG to avoid asserts to be optimized out
Fix compilation on macosx with gtest from homebrew
Issues : RCBC-72
Close dynamic libraries. Fixes small memory leak
Issues : RCBC-70
Include types definitions for POSIX systems. Fixes C++ builds on some systems.
Issues : RCBC-63
Fix win32 builds:
Add suffix to cbc command implementations;
Fix guards for socket errno macros;
Define size_t types to fix MSVC 9 build;
MSVC 9 isn’t C99, but has stddef.h, so just include it.
New Features and Behavior Changes in 1.1.0dp5
Fixes in 1.1.0dp5
Fixes in 1.1.0dp4
New Features and Behavior Changes in 1.1.0dp3
Implement GET_LOCKED (GETL) command
Issues : CCBC-68
Implement UNLOCK_KEY (UNL) command
Issues : CCBC-68
Fixes in 1.1.0dp3
Timeouts can occur during topology changes, rather than be correctly retried. Send the retry-packet to new server
Issues : CCBC-64
Fix ringbuffer_memcpy() (36afdb2). Fix ringbuffer_is_continous().
A hang could occur in libcouchbase_wait() after the timeout period. Check for breakout condition after purging servers
Issues : CCBC-62
Destroy view requests items when server get destroyed. Fixes memory leaks.
A fix for a buffer overflow with the supplied password as has been integrated. While it is a buffer overflow issue, this is not considered to be a possible security issue because the password to the bucket is not commonly supplied by an untrusted source
Issues : RCBC-33
A small memory leak can occur with frequent calls to libcouchbase_create() and libcouchbase_destroy()
Issues : CCBC-65
Use vbucket_found_incorrect_master() to get correct server index during relocating buffers. Pick up cookies from pending buffer unless node connected.
Do not call View callbacks for cancelled requests
hashset.c: iterate over whole set on rehashing. Fixes memory leaks related to hash collisions (905ef95)
Fixes in 1.1.0dp2
Don’t wait for empty buffers. If called with no operations queued, libcouchbase_wait() will block forever. This means that a single threaded application that calls libcouchbase_wait() at different times to make sure operations are sent to the server runs the risk of stalling indefinitely. This is a very likely scenario.
Issues : CCBC-59
Don’t define size_t and ssize_t for VS2008
Fix segfault while authorizing on protected buckets (211bb04)
New Features and Behavior Changes in 1.1.0dp
libcouchbase_make_couch_request()
function. See the associated
documentation and header files for more details.Fixes in 1.1.0dp
Request the tap bytes in a known byte order (adf2b30)
Issues : MB-4834
Check for newer libvbucket