A newer version of this software is available
You are viewing the documentation for an older version of this software. To find the documentation for the current version, visit the Couchbase documentation home page.
This guide provides information for developers who want to use the Couchbase C SDK to build applications that use Couchbase Server.
Now that you’ve installed Couchbase and have created a cluster of servers, you need a client library to read and write data from the cluster. The Couchbase C client library, also known as libcouchbase, can be used from your application to access Couchbase Server.
Here’s a quick outline of what you’ll learn in this chapter:
Get the client library.
Build the client library from source (optional).
Write a simple program to connecting to Couchbase and save some data.
Download the latest client library from the Couchbase download
site. It is available as a source
archive in both .zip
and .tar.gz
Packages are provided for RedHat and Ubuntu Linux systems by providing your package manager with the Couchbase repository information. The method for installation is dependent on your platform:
You must populate RPM with a new source, which is dependent on your RedHat version:
RHEL/CentOS 5.5
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos55-i386.repo
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos55-x86_64.repo
RHEL/CentOS 6.2
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos62-i686.repo
shell> sudo wget -O/etc/yum.repos.d/couchbase.repo \
http://packages.couchbase.com/rpm/couchbase-centos62-x86_64.repo
Then to install libcouchbase with libevent backend, run:
shell> sudo yum check-update
shell sudo yum install -y libcouchbase2-libevent libcouchbase-devel
You must update the apt-get
repository to install the client library:
Ubuntu 12.04 Precise Pangolin (Debian unstable)
Also compatible with recent versions, which have libevent2
.
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1204.list
Ubuntu 11.10 Oneiric Ocelot (Debian unstable)
Also compatible with recent versions, which have libevent2
.
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1110.list
Ubuntu 10.04 Lucid Lynx (Debian stable or testing)
shell> sudo wget -O/etc/apt/sources.list.d/couchbase.list \
http://packages.couchbase.com/ubuntu/couchbase-ubuntu1004.list
Also make sure you have the GPG key installed:
shell> wget -O- http://packages.couchbase.com/ubuntu/couchbase.key | sudo apt-key add -
Then to install libcouchbase with libevent backend, run:
shell> sudo apt-get update
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
This client library is available via a homebrew recipe. After you install homebrew, install libcouchbase:
shell> brew update
shell> brew install libcouchbase
Installation is like many common libraries, following the ./configure
, make
,
make install
conventions.
For libvbucket, extract the archive then cd
into the directory and run:
shell> ./configure
shell> make install
Standard configure options such as --prefix
can be passed to the configure
command. For additional information on configuration options, run./configure
with –help.
For libcouchbase, extract the archive, then cd into the directory and run:
shell> ./configure CPPFLAGS="-I/opt/couchbase/include" --disable-couchbasemock
shell> make install
The --disable-couchbasemock
simply disables some tests which are common during
the development of libcouchbase, but not required when installing a release.
Spin up your visual studio shell and run cmake from there. It is best practice that you make an out-of-tree build; thus like so:
Assuming Visual Studio 2010
C:\> git clone git://github.com/couchbase/libcouchbase.git
C:\> mkdir lcb-build
C:\> cd lcb-build
C:\> cmake -G "Visual Studio 10" ..\libcouchbase
C:\> msbuild /M libcouchbase.sln
This will generate and build a Visual Studio .sln
file.
Windows builds are known to work on Visual Studio versions 2008, 2010 and 2012.
The C client library, libcouchbase
, is a callback-oriented client which makes
it very easy to write high performance programs. There are a few ways you can
drive IO with the library. The simplest approach is to use the synchronous
interface over the asynch internals of the library. More advanced programs will
either call the libcouchbase_wait()
function after generating some operations
or drive the event loop themselves.
To connect, you first configure the connection options and then create an instance of the connection to the cluster:
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);
/*
* Initiate the connect sequence in libcouchbase
*/
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Run the event loop and wait until we've connected */
lcb_wait(instance);
Callbacks are used by the library and are simple functions which handle the result of operations. For example:
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Set up the handler to catch all errors! */
lcb_set_error_callback(instance, error_callback);
/*
* Initiate the connect sequence in libcouchbase
*/
if ((err = lcb_connect(instance)) != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* Run the event loop and wait until we've connected */
lcb_wait(instance);
Callbacks can be set up for all of your operations called in libcouchbase. In the API, you’ll note the use of a cookie. This is metadata from your application which is associated with the request. The libcouchbase library will not inspect any cookie or send the cookie to the server.
When you put the connect logic and the get callback together and plug them into a complete program with the include headers, you get:
#include <libcouchbase/couchbase.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
static void error_callback(lcb_t instance,
lcb_error_t err,
const char *errinfo)
{
fprintf(stderr, "Error %s: %s", lcb_strerror(instance, err),
errinfo ? errinfo : "");
exit(EXIT_FAILURE);
}
/* the callback invoked by the library when receiving a get response */
static void get_callback(lcb_t instance,
const void *cookie,
lcb_error_t error,
const lcb_get_resp_t *resp)
{
if (error != LCB_SUCCESS) {
fprintf(stderr, "Failed to retrieve \"");
fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
fprintf(stderr, "\": %s\n", lcb_strerror(instance, error));
} else {
fprintf(stderr, "Data for key: \"");
fwrite(resp->v.v0.key, 1, resp->v.v0.nkey, stderr);
fprintf(stderr, "\" is : ");
fwrite(resp->v.v0.bytes, 1, resp->v.v0.nbytes, stderr);
}
(void)cookie; /* ignore */
}
int main(void)
{
struct lcb_create_st create_options;
lcb_t instance;
lcb_error_t err;
memset(&create_options, 0, sizeof(create_options));
create_options.v.v0.host = "myserver:8091";
create_options.v.v0.user = "mybucket";
create_options.v.v0.passwd = "secret";
create_options.v.v0.bucket = "mybucket";
err = lcb_create(&instance, &create_options);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to create libcouchbase instance: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* set up the handler to catch all errors */
lcb_set_error_callback(instance, error_callback);
/* initiate the connect sequence in libcouchbase */
err = lcb_connect(instance);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to initiate connect: %s\n",
lcb_strerror(NULL, err));
return 1;
}
/* run the event loop and wait until we've connected */
lcb_wait(instance);
/* set up a callback for our get requests */
lcb_set_get_callback(instance, get_callback);
{
lcb_get_cmd_t cmd;
const lcb_get_cmd_t *commands[1];
commands[0] = &cmd;
memset(&cmd, 0, sizeof(cmd));
cmd.v.v0.key = "foo";
cmd.v.v0.nkey = 3;
err = lcb_get(instance, NULL, 1, commands);
if (err != LCB_SUCCESS) {
fprintf(stderr, "Failed to get: %s\n",
lcb_strerror(NULL, err));
return 1;
}
}
lcb_wait(instance);
lcb_destroy(instance);
exit(EXIT_SUCCESS);
}
To compile this sample program, you must link to libcouchbase
:
shell> gcc -o hellocb -lcouchbase hellocb.c
This tutorial assumes you have installed libcouchbase on your systems per the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.
If you need to set up a server node or data bucket, you can do so with Couchbase Administrative Console, Couchbase Command-Line Interface (CLI), or Couchbase REST API. For information and instructions, see:
Using the Couchbase Web Console, for information on using the Couchbase Administrative Console,
Couchbase CLI, for the command line interface,
Couchbase REST API, for creating and managing Couchbase resources.
After you’ve set up your Couchbase Server and installed the needed client libraries, you can compile and run the following basic program.
Before getting into a more complex example of the programming model to this library, we will walk through a straightforward example of a program which builds with libcouchbase, connects to a server, and sets and gets a value:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <libcouchbase/couchbase.h>
static void die(const char *msg, lcb_error_t err)
{
fprintf(stderr, "Got error %s. Code=0x%x, %s\n",
msg, err, lcb_strerror(NULL, err));
exit(EXIT_FAILURE);
}
static void get_callback(
lcb_t instance,
const void *cookie,
lcb_error_t err,
const lcb_get_resp_t *resp
);
static const char *key = "My Key";
static const char *value = "My Value";
int main(void)
{
lcb_t instance;
lcb_error_t err;
lcb_store_cmd_t s_cmd = { 0 }, *s_cmdlist = &s_cmd;
lcb_get_cmd_t g_cmd = { 0 }, *g_cmdlist = &g_cmd;
char scratch[4096];
err = lcb_create(&instance, NULL);
if (err != LCB_SUCCESS) {
die("Couldn't create instance", err);
}
lcb_set_get_callback(instance, get_callback);
err = lcb_connect(instance);
if (err != LCB_SUCCESS) {
die("Couldn't schedule connection", err);
}
lcb_wait(instance);
s_cmd.v.v0.key = key;
s_cmd.v.v0.nkey = strlen(key);
s_cmd.v.v0.bytes = value;
s_cmd.v.v0.nbytes = strlen(value);
s_cmd.v.v0.operation = LCB_SET;
err = lcb_store(instance, NULL, 1,
(const lcb_store_cmd_t * const *)&s_cmdlist);
if (err != LCB_SUCCESS) {
die("Couldn't schedule store operation!", err);
}
lcb_wait(instance);
g_cmd.v.v0.key = s_cmd.v.v0.key;
g_cmd.v.v0.nkey = s_cmd.v.v0.nkey;
err = lcb_get(instance, scratch, 1,
(const lcb_get_cmd_t * const *)&g_cmdlist);
if (err != LCB_SUCCESS) {
die("Couldn't schedule get operation!", err);
}
lcb_wait(instance);
/**
* Inside the get callback we've copied over the value data from the
* callback into the scratch buffer which we passed as a cookie
*/
{
char reversed[4096];
int ii;
int curlen = strlen(scratch);
curlen--;
for (ii = curlen; ii >= 0; ii--) {
reversed[curlen-ii] = scratch[ii];
}
s_cmd.v.v0.bytes = reversed;
s_cmd.v.v0.operation = LCB_APPEND;
/** Since it's reversed there's no need to re-set the size again */
err = lcb_store(instance, NULL, 1,
(const lcb_store_cmd_t * const *)&s_cmdlist);
if (err != LCB_SUCCESS) {
die("Couldn't schedule second APPEND", err);
}
lcb_wait(instance);
}
/** Now get the key back again */
err = lcb_get(instance, scratch, 1,
(const lcb_get_cmd_t * const *)&g_cmdlist);
if (err != LCB_SUCCESS) {
die("Could not schedule final get operation", err);
}
lcb_wait(instance);
printf("Buffer in server is now %s\n", scratch);
lcb_destroy(instance);
}
static void get_callback(
lcb_t instance,
const void *cookie,
lcb_error_t err,
const lcb_get_resp_t *resp)
{
char *target = (char *)cookie;
if (err != LCB_SUCCESS) {
die("Got error inside get callback", err);
}
memcpy(target, resp->v.v0.bytes, resp->v.v0.nbytes);
target[resp->v.v0.nbytes] = '\0';
}
In addition to the other sections of this manual, such as the Getting Started guide and the API reference, the libcouchbase package includes an examples directory and a tools directory. Each of these show simple Couchbase tools and an example libcouchbase programs.
This section contains information about handling errors and logging error messages.
Most operations return an lcb_error_t
status code. A successful operation is
defined by the return code of LCB_SUCCESS
. You can find a full list of error codes in the <libcouchbase/error.h>
header.
To handle the errors properly, you must understand what the errors mean and whether they indicate a data error, that the operation can be retried, or a fatal error.
Data errors are errors received from the cluster as a result of a
requested operation on a given item. For example, if an lcb_get
is performed on a key that does not exist, the callback will receive an
LCB_KEY_ENOENT
error. Other examples of conditions that can result
in data errors include:
Errors related to load or network issues might be received in exceptional conditions. Although you should not receive these types of errors in normal, well-provisioned deployments, your application must be prepared to handle them and take proper action.
The latter type of errors may be further divided into the following subgroups:
Transient. This error type indicates an environment and/or resource limitation
either on the server or on the network link between the client and
the server. Examples of transient errors include timeout errors or temporary
failures such as LCB_ETIMEDOUT
(took too long to get a reply)
and LCB_ETMPFAIL
(server was too busy). Transient errors are typically best handled on the application
side by backing off and retrying the operation, with the intent of reducing
stress on the exhausted resource. Some examples of transient error
causes:
Fatal. This error type indicates that the client has potentially entered
into an irrecoverable failed state, either because of invalid user
input (or client configuration), or because an administrator has
modified settings on the cluster (for example, a bucket has been
removed). Examples of fatal errors include
LCB_AUTH_ERROR
(authentication failed) and LCB_BUCKET_ENOENT
(bucket does not exist). Fatal errors typically require inspection of the
client configuration and a restart of the client application or
a reversal of the change performed at the cluster.
Some examples of fatal error causes:
The lcb_errflags_t
enumeration defines a set of flags that are associated
with each error code. These flags define the type of error. Some examples of error types:
LCB_ERRTYPE_INPUT
, which is set when a malformed parameter is passed to the libraryLCB_ERRTYPE_DATAOP
which is set when the server is unable to satisfy data constraints such as a missing key or a CAS mismatch.The LCB_EIF<TYPE>
methods, where <TYPE>
represents one of the errflags_t
flags, can be used to check whether an error is of a specific type.
The following example shows how to check the return codes:
static void get_callback(
lcb_t instance,
const void *cookie,
lcb_error_t err,
const lcb_get_resp_t *resp)
{
if (err == LCB_SUCCESS) {
printf("Successfuly retrieved key!\n");
} else if (LCB_EIFDATA(err)) {
switch (err) {
case LCB_KEY_ENOENT:
printf("Key not found!\n");
break;
default:
printf("Received other unhandled data error\n");
break;
}
} else if (LCB_EIFTMP(err)) {
printf("Transient error received. May retry\n");
}
}
Success and failure depend on the context. A successful return code for one of
the data operation APIs (for example, lcb_store
) does not mean the operation
itself succeeded and the key was successfully stored. Rather, it means the
key was successfully placed inside the library’s internal queue. The actual
error code is delivered within the response itself.
Errors might be received when scheduling operations inside the library. If a
scheduling API returns anything but LCB_SUCCESS
, then that implies the operation
itself failed as a whole and no callback will be delivered for that
operation. Conversely, if a scheduling API returns an LCB_SUCCESS
then the callback
will always be invoked.
The library might also mask errors during the scheduling phase and deliver them asynchronously to the user via a callback (for example, when implementation constraints do not easily allow the immediate returning of an error code).
You can use the library’s logging API to forward messages to your logging
framework of choice. Additionally, you can enable logging to the console’s
standard error by setting the LCB_LOGLEVEL
environment variable.
Setting the logging level via the environment variable allows applications linked against the library that do not offer native support for logging to still employ the use of the diagnostics provided by the library. You do something like this to display logging information:
$ LCB_LOGLEVEL=5 ./my_app
The value of the LCB_LOGLEVEL
environment variable is an integer from 1 to 5. The higher
the value, the more verbose the details. The value of 0 disables the
console logging.
To set up your own logger, you must define a logging callback to be invoked whenever the library emits a logging message
static void logger(
lcb_logprocs *procs,
unsigned int iid,
const char *module,
int severity,
const char *srcfile,
int srcline,
const char *fmt,
va_list ap)
{
char buf[4096];
vsprintf(buf, ap, buf);
dispatch_to_my_logging(buf);
}
static void apply_logging(lcb_t instance)
{
lcb_logprocs procs = { 0 };
procs.v.v0.callback = logger;
lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_LOGGER, &procs);
}
The lcb_logprocs
pointer must point to valid memory and must not be
freed by the user after passing it to the library until the instance associated
with it is destroyed.
The arguments to the logging function are:
procs
—The logging procedure structure passed to the lcb_cntl()
operationiid
—An integer used to identify the lcb_t
. This is useful if you have multiple
instances running in your applicationmodule
—A short string representing the subsystem that emitted the messageseverity
—An integer describing the severity of the message
(higher is more severe). Refer to the lcb_log_severity_t
enum
within <libcouchbase/types.h>
for a full listing.srcfile
and srcline
—File and line where this message was emittedfmt
—a format stringap
—arguments to the format stringAdditional diagnostic information is provided by the error callback. The error callback is a legacy interface and should generally not be used. The error callback, however, does allow programmatic capture of some errors—something that is not easy with the logging interface.
Specifically, the error callback receives error information when a bootstrap or configuration update has failed.
static void error_callback(lcb_t instance, lcb_error_t err, const char *msg)
{
/** ... */
}
lcb_set_error_callback(instance, error_callback);
libcouchbase version 2.2 and later supports the CRAM-MD5 authentication mechanism, which enables you to avoid passing the bucket password as plain text over the wire.
Along with this change, a new setting was introduced: LCB_CNTL_FORCE_SASL_MECH
, which forces a specific Simple Authentication and Security Layer (SASL) mechanism to use for authentication. This can allow a user to ensure a certain level of security and have the connection fail if the desired mechanism is not available.
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_FORCE_SASL_MECH, "CRAM-MD5");
libcouchbase 2.3 and later supports multiple methods of bootstrapping and configuration retrieval. Normally you should not change these settings because the library determines the best method available.
You can obtain the cluster configuration by using either the memcached protocol that uses the cluster configuration carrier protocol (CCCP) or the legacy mode that uses a connection to the REST API. CCCP is available only in cluster version 2.5 and later and libcouchbase version 2.3 and later.
The default behavior of the library is to first attempt bootstrap over CCCP and then fall back to HTTP if CCCP fails. CCCP bootstrap is preferable because it does not require a dedicated configuration socket, does not require the latency and overhead of the HTTP protocol, and is more efficient to the internals of the cluster.
To explicitly define the set of configuration modes to use, specify the modes inside the lcb_create_st
structure.
The following snippet disables HTTP bootstrapping:
struct lcb_create_st crparams = { 0 };
lcb_config_transport_t transports[] = {
LCB_CONFIG_TRANSPORT_CCCP,
LCB_CONFIG_TRANSPORT_LIST_END
};
/** Requires version 2 */
cparams.version = 2;
crparams.v.v2.transports = transports;
crparams.v.v2.host = "foo.com;bar.org;baz.net";
lcb_t instance;
lcb_create(&instance, &crparams);
The transports
field accepts an array of enabled transports followed by the
end-of-list element (LCB_CONFIG_TRANSPORT_LIST_END
). If this parameter is
specified, the library only uses the transports that are specified in the list.
The enabled transports remain valid for the duration of the instance. This means that it is used for the initial bootstrap and any subsequent configuration updates.
Memcached buckets do not support CCCP. You must ensure that HTTP is enabled for them (which is the default)
During the lifetime of a client, its cluster configuration might need to change or be retrieved. Reasons for this include:
These changes are collectively known as topology changes or configuration changes. As mentioned in the list, fetching a configuration might not always be the result of an actual topology change. For example, if a certain node becomes unreachable to the client, then the client will attempt to fetch a new configuration under the assumption that the unresponsive node has potentially been ejected from the cluster.
For proactive configuration fetches (such as in response to an error) there is effectively a responsiveness-for-efficiency trade off. If the client does not check for updated configurations often enough, it risks failing operations that could have succeeded if it knew about the updated topology. If the client fetches the configuration too quickly, it can cause a lot of traffic and increase load on the server (this especially holds true with HTTP). Because some server versions allocate quite a few resources for each HTTP connection, maintaining idle HTTP connections might be more expensive than re-establishing them on demand.
Rather than guess your hardware and infrastructure requirements, there are several settings in the library to help you tune it to your needs.
Each time a network-like error is encountered in the library (that is, a timeout or connection error), an internal error counter is incremented. When the counter reaches a specified value, the library fetches a new configuration and resets the counter to zero.
The intervals between successive increments are timed. If a call is made to increment the counter and the time between the last call exceeds a certain threshold, then the configuration is refetched again.
You can control both behaviors by using the lcb_cntl()
method. To modify the error
counter threshold, use the LCB_CNTL_CONFERRTHRESH
setting. To modify the
error delay threshold, use the LCB_CNTL_CONFDELAY_THRESH
setting.
The library allows you to cache the configuration settings in a file and then pass the file to create another instance. The new instance is configured without having to perform network I/O to retrieve the initial map.
The goal of this feature is to limit the number of initial connections for retrieving the cluster map and to reduce latency for performing simple operations. Such a usage is ideal for short lived applications that are spawned as simple scripts (for example, PHP scripts).
To use the configuration cache you must use a special form of initialization:
lcb_error_t err;
lcb_t instance;
struct lcb_create_st cropts;
cropts.v.v0.host = "foo.com";
err = lcb_create(&instance, &cropts);
if (err != LCB_SUCCESS) {
die("Couldn't create instance", err);
}
/* now set the cache */
const char *cachefile = "some_file";
err = lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_CONFIGCACHE, (void*)cachefile);
if (err != LCB_SUCCESS) {
die("Couldn't assign the configuration cache!\n");
}
If the cache file does not exist, the client bootstraps from the network
and writes the information to the file after it has received the map itself.
You can check to see if the client has loaded a configuration from the cache
(and does not need to fetch the configuration from the network) by using lcb_cntl()
int is_loaded = 0;
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
if (is_loaded) {
printf("Configuration cache file loaded OK\n");
} else {
printf("Configuration cache likely does not exist yet\n");
}
You need to be aware of the following limitations:
Memcached buckets are not supported with the configuration cache.
The file-based provider is recommended only for short-lived clients. It is intended to reduce network I/O and latency during the initial bootstrap process. It is not a recommended solution for reducing the total number of connections to the cluster.
Timeouts prevent a single operation from hanging or waiting indefinitely for an I/O operation to complete. Typically, timeouts are exact, but if they are not, they tend to fire early rather than late.
Internally, timeouts are implemented within an event loop (as opposed to a kernel-based signal timer). This means:
As an I/O library, libcouchbase has quite a few timeout settings. The main
timeout setting is the operation timeout, which indicates the amount of
time to wait from when the operation was scheduled. If a response is not
received within this time period, the library fails the operation and invokes
the callback, at which point the error will be set to LCB_ETIMEDOUT
. The
operation to control this via lcb_cntl
is LCB_CNTL_OP_TIMEOUT
.
The bootstrap timeout, which controls the amount of time the library waits for the initial bootstrap, is another important timeout. The bootstrap timeout comprises two timeout values, the absolute timeout and the per-node bootstrap timeout. The absolute timeout controls the overall amount of time the library should wait until a configuration has been received, while the per-node bootstrap timeout controls the amount of time per-node that the library will wait until the next node is retried.
You can access the absolute timeout via the
LCB_CNTL_CONFIGURATION_TIMEOUT
. This value places an
absolute limit on the amount of time the library waits until the
client has been bootstrapped before delivering an error. This timeout
is used only during the initial connection and has no effect thereafter.
You can access the bootstrap timeout via the LCB_CNTL_CONFIG_NODE_TIMEOUT
operation.
When fetching
the cluster configuration there are often multiple nodes that might function
as configuration sources. Configurations are retrieved once during
initialization, but might also be fetched later on, typically when an error condition
is detected by the library or when the cluster topology changes. In such
situations, the behavior is to start fetching the configuration from each
node in the list, in sequence. If the retrieval of the configuration from the first
node fails with a timeout (that is, the node is unresponsive) the library will
proceed to fetch from the next node and so on until all nodes are exhausted.
This tutorial assumes you have installed libcouchbase on your systems, following the installation instructions in the Getting Started section of this guide. Because the approach for building a program based on libcouchbase may vary between Linux/Mac OS and Windows, this tutorial will focus on the components of the program rather than how to build it.
The libcouchbase is written in C and can be integrated in various ways with your application. The simplest integration scenario is when your application is written in a scripting language (supported by a Couchbase client library) that already has a libcouchbase wrapper. Couchbase has built and maintains following libcouchbase-based libraries:
Users of these libraries automatically get an API, which looks natural for their platform, so that it doesn’t require knowledge of libcouchbase APIs. This tutorial is mostly about two other use cases:
Add libcouchbase into an existing application to implement persistence layer. This section will contain two parts: first we will create a simple echo server written using libev library. Second, we will show how to persist each message going through the server to Couchbase.
Get complete code here: https://github.com/couchbaselabs/libev-couchbase-example
Build applications around libcouchbase. Here we’ll show how to build a proxy server for regular memcached clients. Building this proxy server is just like building a moxi server, but the proxy server will be more limited more limited in abilities.
Get complete code here: https://github.com/couchbaselabs/libcouchbase-proxy-sample
Let’s start with a specific existing application for this example. Our application is the asynchronous single-threaded echo server, which leverages libev to solve C10K problem and efficiently serves a huge number of concurrent connections. It should listen to the given port (in this sample 4567) and send back everything the client submitted to it.
The libev library implements reactor
pattern which uses the term
event loop
to represent an entity which encapsulates the register of handlers
for various IO events and the timers. To start, we have the application code
from the main function:
int main()
{
struct ev_loop *loop = ev_default_loop(0);
int sd;
struct sockaddr_in addr;
struct ev_io server;
if ((sd = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
fail("socket error");
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(PORT_NO);
addr.sin_addr.s_addr = INADDR_ANY;
if (bind(sd, (struct sockaddr *) &addr, sizeof(addr)) != 0) {
fail("bind error");
}
if (listen(sd, 2) < 0) {
fail("listen error");
}
ev_io_init(&server, accept_cb, sd, EV_READ);
ev_io_start(loop, &server);
printf("Listen on %d\n", PORT_NO);
ev_run(loop, 0);
return 0;
}
Here we are create an event loop instance, and create and bind a new socket to
local port PORT_NO (4567)
. After that we can mark this socket as listening and
ask libev to call the function accept_cb
when the socket become ready to read,
that is when the new client will come to us. The final step is to run ev_run()
which does all the server interaction.
The accept_cb
is in charge of allocating client structures 60 the buffer for
messages. The sample is using ring buffers from the libcouchbase code for easier
managing memory for IO buffers.
void accept_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
struct client_s *client;
int flags;
if (EV_ERROR & revents) {
fail("got invalid event");
}
client = malloc(sizeof(struct client_s));
if (!client) {
fail("client watcher alloc");
}
if (!ringbuffer_initialize(&client->buf, BUFFER_SIZE)) {
fail("client buffer alloc");
}
client->naddr = sizeof(client->addr);
client->fd = accept(watcher->fd, (struct sockaddr *)&client->addr,
&client->naddr);
if (client->fd < 0) {
fail("accept error");
}
if ((flags = fcntl(client->fd, F_GETFL, NULL)) == -1) {
fail("fcntl(F_GETFL)");
}
if (fcntl(client->fd, F_SETFL, flags | O_NONBLOCK) == -1) {
fail("fcntl(F_SETFL, O_NONBLOCK)");
}
printf("Successfully connected with client\n");
ev_io_init((ev_io *)client, client_cb, client->fd, EV_READ);
ev_io_start(loop, (ev_io *)client);
}
The most interesting part of the function above is in the last two lines. It
registers the new handler, client_cb
, upon reading an event for the client
socket. This handler will be called each time when the kernel receives new data
from the socket, so that it could be read with a non-blocking call.
The last snippet of our application depicts the client_cb
function, which
handles all the application logic. It reads everything from the socket if it is
ready, registers itself for write events, and then sends all the contents out to
the network. It is also the handling end of a stream situation, which means that
the client closed the connection with the Couchbase server.
void client_cb(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
struct lcb_iovec_st iov[2];
ssize_t nbytes;
struct client_s *client = (struct client_s *)watcher;
if (EV_READ & revents) {
ringbuffer_ensure_capacity(&client->buf, BUFFER_SIZE);
ringbuffer_get_iov(&client->buf, RINGBUFFER_WRITE, iov);
nbytes = io_recvv(watcher->fd, iov);
if (nbytes < 0) {
fail("read error");
} else if (nbytes == 0) {
ev_io_stop(loop, watcher);
ringbuffer_destruct(&client->buf);
free(client);
printf("Peer disconnected\n");
return;
} else {
ringbuffer_produced(&client->buf, nbytes);
printf("Received %zu bytes\n", nbytes);
ev_io_stop(loop, watcher);
ev_io_set(watcher, watcher->fd, EV_WRITE);
ev_io_start(loop, watcher);
}
} else if (EV_WRITE & revents) {
ringbuffer_get_iov(&client->buf, RINGBUFFER_READ, iov);
nbytes = io_sendv(watcher->fd, iov);
if (nbytes < 0) {
fail("write error");
} else if (nbytes == 0) {
ev_io_stop(loop, watcher);
ringbuffer_destruct(&client->buf);
free(client);
printf("Peer disconnected\n");
return;
} else {
ringbuffer_consumed(&client->buf, nbytes);
printf("Sent %zu bytes\n", nbytes);
ev_io_stop(loop, watcher);
ev_io_set(watcher, watcher->fd, EV_READ);
ev_io_start(loop, watcher);
}
} else {
fail("got invalid event");
}
}
Again, the whole application is stored on github at https://github.com/couchbaselabs/libev-couchbase-example. Let’s see it in action. To clone and build the applications, use the following commands:
~ $ git clone git://github.com/couchbaselabs/libev-couchbase-example.git
~ $ cd libev-couchbase-example/step1
step1 $ ./autogen.sh && ./configure && make
As a client for an echo application, we can build any telnet-like application. The following listing simulates two terminals by splitting them with a vertical line:
step 1 $ ./server | step1 $ telnet localhost 4567
Listen on 4567 | Trying 127.0.0.1...
Successfully connected with client | Connected to localhost.
Received 14 bytes | Escape character is '^]'.
Sent 14 bytes | Hello world!
Received 9 bytes | Hello world!
Sent 9 bytes | foo bar
Peer disconnected | foo bar
| ^]
| telnet> Connection closed.
Now that we described our application, let’s integrate Couchbase as the database
using libcouchbase. The server.c
file will have some small additions. In the
main
function, before initializing server socket, we will call storage_init
and assign the connection information to the server structure variable.
server.handle = storage_init(loop, "localhost:8091", "default", null);
Also in the client_cb
handler, each time libcouchbase receives data from the
client, it will send the data to Couchbase server with storage_put
where
the_key
is a string "example"
.
storage_put(client, the_key, val, nbytes);
Here, the initial application from step one is already using its own custom event loop to handle other IO in the application. The application could be more complex, but libcouchbase has a separate interface to inject your IO implementation into the library. In the simplest case, for example if your application is using one of the IO libraries supported by libcouchbase, you can just pass your event loop instance into the initializer, and libcouchbase won’t create new one. Instead it will register all its events on this external loop. In case your application is using its own implementation of a reactor pattern, such as nginx, or you use linux epoll API, you can easily write your own IO plugin and pass it to the connection initializer.
The echo server is using libev library, and libcouchbase provides a plugin to
libev out-of-the-box. In this case we can imagine that it doesn’t already have
it. In this case, we can copy the plugin from libcouchbase distribution into the
directory step2/lcb-plugin/
and update build files appropriately.
Now we inspect the storage.c
, where our storage_init()
and storage_put()
defined.
lcb_t storage_init(struct ev_loop *loop, const char *host, const char *bucket, const char *password)
{
struct lcb_create_st opts;
struct lcb_create_io_ops_st io_opts;
lcb_t handle;
lcb_error_t err;
io_opts.version = 1;
io_opts.v.v1.sofile = NULL;
io_opts.v.v1.symbol = "lcb_create_libev_io_opts";
io_opts.v.v1.cookie = loop;
opts.version = 0;
opts.v.v0.host = host;
opts.v.v0.bucket = bucket;
opts.v.v0.user = bucket;
opts.v.v0.passwd = password;
err = lcb_create_io_ops(&opts.v.v0.io, &io_opts);
if (err != LCB_SUCCESS) {
error_callback(NULL, err, "failed to create IO object");
return NULL;
}
err = lcb_create(&handle, &opts);
if (err != LCB_SUCCESS) {
error_callback(NULL, err, "failed to create connection object");
return NULL;
}
(void)lcb_set_error_callback(handle, error_callback);
(void)lcb_set_store_callback(handle, storage_callback);
err = lcb_connect(handle);
if (err != LCB_SUCCESS) {
error_callback(handle, err, "failed to connect to the server");
return NULL;
}
return handle;
}
The end of the function may look familiar, but the io_opts
structure is more
interesting. It defines how to look up our custom IO plugin. The
io_opts.v.v1.sofile
is set to NULL
to specify that our plugin compiled into
the current executable image. In this case, symbol
is the string name of the
function with the following signature:
lcb_error_t lcb_create_libev_io_opts(int version, lcb_io_opt_t *io, void *loop);
This initializes the IO plugin and then the cookie
will be passed to this
function as the loop
argument. You can refer to the lcb_create_io_ops(3)
manpage for other ways to initialize IO subsystem.
The libcouchbase client is purely asynchronous; therefore, each data operation isplits into two parts: a function-scheduler, which validates and copies all arguments to internal buffers for further handing, and a function-callback, which will be called with the results of the operation. It is possible to implement a wrapper which will provide a more synchronous API, but it is difficult to write a library API that is both generic and efficient. Since all communication is done by using function arguments, it is easier to maintain backward compatible APIs by versioning both incoming structures and results. Let’s see how we did it in our sample:
void storage_put(struct client_s *client, const char *key,
const void *val, size_t nval)
{
lcb_error_t err;
lcb_store_cmd_t cmd;
const lcb_store_cmd_t *cmds[] = { &cmd };
memset(&cmd, 0, sizeof(cmd));
cmd.version = 0;
cmd.v.v0.key = key;
cmd.v.v0.nkey = strlen(key);
cmd.v.v0.bytes = val;
cmd.v.v0.nbytes = nval;
cmd.v.v0.operation = LCB_SET;
err = lcb_store(client->handle, client, 1, cmds);
if (err != LCB_SUCCESS) {
error_callback(client->handle, err, "failed to schedule store operation");
}
}
void storage_callback(lcb_t instance, const void *cookie,
lcb_storage_t operation, lcb_error_t error,
const lcb_store_resp_t *resp)
{
struct client_s *client = (struct client_s *)cookie;
struct ev_io* watcher = (struct ev_io*)client;
char cas_str[128];
ssize_t nb;
nb = snprintf(cas_str, 128, "%"PRIu64"\n", resp->v.v0.cas);
if (nb < 0) {
fail("output CAS value");
}
ringbuffer_ensure_capacity(&client->out, nb);
if (ringbuffer_write(&client->out, cas_str, nb) != nb) {
fail("write CAS into the buffer");
}
ev_io_stop(client->loop, watcher);
ev_io_set(watcher, watcher->fd, EV_WRITE);
ev_io_start(client->loop, watcher);
(void)operation;
(void)resp;
}
Here, the storage_put()
function is a wrapper over lcb_store(3)
. The
storage_put()
function translates arguments to the versioned structure and
runs lcb_store
to pass it to the library. The data won’t be sent immediately,
but when the library connects to the data socket, it will be ready to accept
data. After the server processes the request, the application will be notified
asynchronously via storage_callback
. This callback will copy CAS value to the
output buffer to be sent to the client.
Time to demonstrate an application. This application is built the same as
before, except you need to navigate to step2/
directory. As previously noted,
the script will be split to demonstrate both server and the client:
step2 $ ./server | step2 $ telnet localhost 4567
Listen on 4567 | Trying 127.0.0.1...
Successfully connected with client | Connected to localhost.
Received 14 bytes | Escape character is '^]'.
Sent 20 bytes | Hello world!
Peer disconnected | 2916447493390860288
| ^]
| telnet> Connection closed.
Now you can install cbc
tools from package libcouchbase2-bin
. If you
installed libcouchbase from source, you most likely have it already. Then you
can check that it created a key example and that the CAS values are matching:
step2 $ printf "%x\n" 2916447493390860288
28794d8b11a40000
step2 $ cbc cat example
"example" Size:14 Flags:0 CAS:28794d8b11a40000
Hello world!
You can use libcouchbase when you are building Couchbase-enabled application and want to bootstrap quickly. To do so you can build your application using the IO abstraction provided with libcouchbase.
In this example, we create a thin proxy application which will expose the memcached-compatible API for the system. Similar approaches exist for moxi server. Also this application will use less hard-coded logic and more look like real-life service. Full source code can be found at https://github.com/couchbaselabs/libcouchbase-proxy-sample.
For clarity, we skip parsing and initialization of the Couchbase connection from
the main
function. The proxy code begins in run_proxy
, which is similar to
the main function in the previous section. The important difference here is that
we are using the IO abstraction from libcouchbase, to register events, and drive
an event loop.
void run_proxy(lcb_t conn)
{
lcb_io_opt_t io;
struct sockaddr_in addr;
int sock, rv;
server_t server;
io = opts.v.v0.io;
info("starting proxy on port %d", port);
sock = socket(PF_INET, SOCK_STREAM, 0);
if (sock == -1) {
fail("socket()", strerror(errno));
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
addr.sin_addr.s_addr = INADDR_ANY;
rv = bind(sock, (struct sockaddr *)&addr, sizeof(addr));
if (rv == -1) {
fail("bind()");
}
rv = listen(sock, 10);
if (rv == -1) {
fail("listen()");
}
server.conn = conn;
server.io = io;
server.event = io->v.v0.create_event(io);
if (server.event == NULL) {
fail("failed to create event for proxy");
}
io->v.v0.update_event(io, sock, server.event, LCB_READ_EVENT,
&server, proxy_accept_callback);
lcb_set_error_callback(conn, error_callback);
lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);
info("use ctrl-c to stop");
io->v.v0.run_event_loop(io);
}
As in the previous example, we create a listening socket, and then we bind a
handler to process read events. The handler is proxy_accept_callback
, and it
will be triggered for each new client. In turn it will create a new client
structure, defined as follows:
typedef struct client_st client_t;
struct client_st {
int id;
server_t *server;
int sock;
ringbuffer_t in;
ringbuffer_t out;
void *event;
};
This initializes input and output buffers,and makes the socket descriptors
non-blocking. At the end, the code will register the handler function
proxy_client_callback
for read events, and then wait for incoming data.
The proxy implements a very limited number of protocol commands: GET, SET, and VERSION. But other commands can be easily added. Here’s an example:
void proxy_client_callback(lcb_socket_t sock, short which, void *data)
{
struct lcb_iovec_st iov[2];
ssize_t rv;
lcb_io_opt_t io;
client_t *cl = data;
io = cl->server->io;
if (which & LCB_READ_EVENT) {
for (;;) {
/* read in chunks of BUFFER_SIZE */
ringbuffer_ensure_capacity(&cl->in, BUFFER_SIZE);
ringbuffer_get_iov(&cl->in, RINGBUFFER_WRITE, iov);
rv = io->v.v0.recvv(io, cl->sock, iov, 2);
if (rv == -1) {
if (io->v.v0.error == EINTR) {
/* interrupted by signal */
continue;
} else if (io->v.v0.error == EWOULDBLOCK) {
/* nothing to read right now */
io->v.v0.update_event(io, cl->sock, cl->event,
LCB_WRITE_EVENT, cl,
proxy_client_callback);
break;
} else {
fail("read error");
}
} else if (rv == 0) {
/* end of stream */
io->v.v0.destroy_event(io, cl->event);
ringbuffer_destruct(&cl->in);
ringbuffer_destruct(&cl->out);
close(cl->sock);
info("[%d] disconnected", cl->id);
free(cl);
return;
} else {
ringbuffer_produced(&cl->in, rv);
for (;;) {
protocol_binary_request_header req;
lcb_size_t nr, sz;
char *buf;
/* make sure the buffer is aligned */
if (ringbuffer_ensure_alignment(&cl->in) != 0) {
fail("cannot align the buffer");
}
/* take the packet header from the buffer */
nr = ringbuffer_peek(&cl->in, req.bytes, sizeof(req));
if (nr < sizeof(req)) {
break;
}
/* make sure the buffer has whole the body */
sz = ntohl(req.request.bodylen) + sizeof(req);
if (cl->in.nbytes < sz) {
break;
}
/* copy packet into intermediate buffer */
buf = malloc(sizeof(char) * sz);
if (buf == NULL) {
fail("cannot allocate buffer for packet");
}
nr = ringbuffer_read(&cl->in, buf, sz);
if (nr < sizeof(req)) {
fail("input buffer doesn't contain enough data");
}
/* handle packet and deallocate the intermediate
* buffer */
handle_packet(cl, buf);
free(buf);
}
}
}
}
if (which & LCB_WRITE_EVENT) {
/* check if we have something to send */
ringbuffer_get_iov(&cl->out, RINGBUFFER_READ, iov);
if (iov[0].iov_len + iov[1].iov_len == 0) {
io->v.v0.delete_event(io, cl->sock, cl->event);
return;
}
rv = io->v.v0.sendv(io, cl->sock, iov, 2);
if (rv < 0) {
fail("write error");
} else if (rv == 0) {
io->v.v0.destroy_event(io, cl->event);
ringbuffer_destruct(&cl->in);
ringbuffer_destruct(&cl->out);
close(cl->sock);
info("[%d] disconnected", cl->id);
free(cl);
return;
} else {
ringbuffer_consumed(&cl->out, rv);
io->v.v0.update_event(io, cl->sock, cl->event,
LCB_READ_EVENT, cl,
proxy_client_callback);
}
}
(void)sock;
}
The function proxy_client_callback
logically divided into two parts.
The first part occurs when the event loop notifies us that the socket is ready
for non-blocking reading. At this point, the function tries to read all the data
available on the socket. It finds the packets and passes aligned byte arrays to
handle_packet
. Once the code gets a special code from the IO subsystem get
from the IO subsystem special code, that it cannot read anything without
blocking, it stops processing and registers itself for write events, because in
nearest future, responses will appear in the output buffer.
The second part of the function reacts to write events and is much simpler. It just checks to see if there is data in the output buffer. If there is data in the output buffer, it will try to send that data to the client. If not, the functionwill unregister itself from write events to save CPU cycles.
The function handle_packet
is a good place to start playing with this example
if you’d like to add new features there. It decodes protocol packet and
translates it into libcouchbase calls.
void handle_packet(client_t *cl, char *buf)
{
protocol_binary_request_header *req = (void *)buf;
protocol_binary_response_header res;
union {
lcb_get_cmd_t get;
lcb_store_cmd_t set;
} cmd;
union {
const lcb_get_cmd_t *get[1];
const lcb_store_cmd_t *set[1];
} cmds;
cookie_t *cookie;
cookie = malloc(sizeof(cookie_t));
if (cookie == NULL) {
fail("cannot allocate buffer for command cookie");
}
cookie->client = cl;
cookie->opaque = req->request.opaque;
cookie->opcode = req->request.opcode;
memset(&cmd, 0, sizeof(cmd));
switch (req->request.opcode) {
case PROTOCOL_BINARY_CMD_GET:
cmds.get[0] = &cmd.get;
cmd.get.v.v0.nkey = ntohs(req->request.keylen);
cmd.get.v.v0.key = buf + sizeof(*req);
info("[%d] get \"%.*s\"", cl->id,
(int)cmd.get.v.v0.nkey, (char *)cmd.get.v.v0.key);
lcb_get(cl->server->conn, (const void*)cookie, 1, cmds.get);
break;
case PROTOCOL_BINARY_CMD_SET:
cmds.set[0] = &cmd.set;
cmd.set.v.v0.operation = LCB_SET;
cmd.set.v.v0.nkey = ntohs(req->request.keylen);
cmd.set.v.v0.key = buf + sizeof(*req) + 8;
cmd.set.v.v0.nbytes = ntohl(req->request.bodylen) - 8 - ntohs(req->request.keylen);
cmd.set.v.v0.bytes = buf + sizeof(*req) + 8 + ntohs(req->request.keylen);
cmd.set.v.v0.cas = req->request.cas;
cmd.set.v.v0.datatype = req->request.datatype;
memcpy(&cmd.set.v.v0.flags, buf + sizeof(*req), 4);
cmd.set.v.v0.flags = ntohl(cmd.set.v.v0.flags);
memcpy(&cmd.set.v.v0.exptime, buf + sizeof(*req) + 4, 4);
cmd.set.v.v0.exptime = ntohl(cmd.set.v.v0.exptime);
info("[%d] set \"%.*s\"", cl->id,
(int)cmd.set.v.v0.nkey, (char *)cmd.set.v.v0.key);
lcb_store(cl->server->conn, (const void*)cookie, 1, cmds.set);
break;
case PROTOCOL_BINARY_CMD_VERSION:
free(cookie);
info("[%d] version", cl->id);
res.response.magic = PROTOCOL_BINARY_RES;
res.response.opcode = req->request.opcode;
res.response.keylen = 0;
res.response.extlen = 0;
res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
res.response.status = PROTOCOL_BINARY_RESPONSE_SUCCESS;
res.response.bodylen = htonl(sizeof(version_msg));
res.response.opaque = req->request.opaque;
res.response.cas = 0;
ringbuffer_ensure_capacity(&cl->out, sizeof(res));
ringbuffer_write(&cl->out, res.bytes, sizeof(res));
ringbuffer_write(&cl->out, version_msg, sizeof(version_msg));
break;
default:
free(cookie);
info("[%d] unsupported command: 0x%02x", cl->id, req->request.opcode);
res.response.magic = PROTOCOL_BINARY_RES;
res.response.opcode = req->request.opcode;
res.response.keylen = 0;
res.response.extlen = 0;
res.response.datatype = PROTOCOL_BINARY_RAW_BYTES;
res.response.status = PROTOCOL_BINARY_RESPONSE_NOT_SUPPORTED;
res.response.bodylen = htonl(sizeof(notsup_msg));
res.response.opaque = req->request.opaque;
res.response.cas = 0;
ringbuffer_ensure_capacity(&cl->out, sizeof(notsup_msg) + sizeof(res));
ringbuffer_write(&cl->out, res.bytes, sizeof(res));
ringbuffer_write(&cl->out, notsup_msg, sizeof(notsup_msg));
}
}
To maintain a logical link between requests and responses for the clients, the
protocol defines a special field opaque
, which can be a special tag or
sequence number. To preserve this field in the response, we will put it into a
cookie_t
structure. During the proxy initialization, we defined two
callback-functions for libcouchbase:
lcb_set_get_callback(conn, get_callback);
lcb_set_store_callback(conn, store_callback);
In this kind of application, these functions just build a protocol response from
libcouchbase return value. They also register the proxy_client_callback
on
write events to send out data recently written into the buffer.
void get_callback(lcb_t conn, const void *cookie, lcb_error_t err,
const lcb_get_resp_t *item)
{
cookie_t *c = (cookie_t *)cookie;
client_t *cl = c->client;
lcb_io_opt_t io = cl->server->io;
protocol_binary_response_get res;
res.message.header.response.magic = PROTOCOL_BINARY_RES;
res.message.header.response.opcode = PROTOCOL_BINARY_CMD_GET;
res.message.header.response.keylen = 0;
res.message.header.response.extlen = 4;
res.message.header.response.datatype = item->v.v0.datatype;
res.message.header.response.status = htons(map_status(err));
res.message.header.response.bodylen = htonl(4 + item->v.v0.nbytes);
res.message.header.response.opaque = c->opaque;
res.message.header.response.cas = item->v.v0.cas;
res.message.body.flags = htonl(item->v.v0.flags);
ringbuffer_ensure_capacity(&cl->out, sizeof(res.bytes) + item->v.v0.nbytes);
ringbuffer_write(&cl->out, res.bytes, sizeof(res.bytes));
ringbuffer_write(&cl->out, item->v.v0.bytes, item->v.v0.nbytes);
io->v.v0.update_event(io, cl->sock, cl->event, LCB_WRITE_EVENT,
cl, proxy_client_callback);
free(c);
(void)conn;
}
These are viritually all the integration points with libcouchbase. As we demonstrated, libcouchbase more than a protocol parser; it also an abstract and portable IO framework, allowing you to build asynchronous applications around Couchbase Server.
To conclude, let’s run your favourite memcached client and verify our proxy. First you need to clone and build it:
$ git clone git://github.com/couchbaselabs/libcouchbase-proxy-sample.git
$ cd libcouchbase-proxy-sample
libcouchbase-proxy-sample $ make
Run the server and send couple of commands to it. We will use the dalli ruby gem here:
libcouchbase-proxy-sample $ ./proxy | $ irb
connecting to bucket "default" at "localhost:8091" | irb:001:0> require 'dalli'
starting proxy on port 1987 | true
use ctrl-c to stop | irb:002:0> m = Dalli::Client.new "localhost:1987"
[1] connected | #<Dalli::Client:0x007f716321ce20 @servers=["localhost:1987"], @options={}, @ring=nil>
[1] version | irb:003:0> m.set("foo", "bar")
[1] set "foo" | true
[1] get "foo" | irb:004:0> m.get("foo")
[1] disconnected | "bar"
The following sections provide release notes for individual release versions of the C Couchbase Client Library. To browse or submit new issues, see Couchbase Client Library C Issues Tracker.
Fixes in 2.3.1
Fixed a bug where randomizing host lists during the initial connect would skip the first node in the list.
Issues: CCBC-433
Assign the cccp_cookie
to the provider. This fixes
an issue where a configuration request over CCCP to an already-connected
node would not be received, potentially causing delays in the client
receiving the newest cluster topology.
Issues: CCBC-414
Fixed a bug where stale commands (and their cookies) would be confused with newer responses. This issue caused segmentation faults when requesting a new configuration on a connected node via CCCP because the CCCP cookie format was being overlaid on a different area of memory. This bug also manifested itself during multi-get responses where some items were not found.
Issues: CCBC-435
Don’t attempt to remap keys to nodes without any vBuckets.
When a NOT_MY_VBUCKET
error is received, the behavior of the library is to attempt
to map the key to another node in the cluster. It determines the node by the
forward map, and if that doesn’t exist it selects a random node in the
cluster. Sometimes a node will exist within the cluster map only because it
is in an ‘eject wait’ state where the node has already been ejected from the
cluster and is just returning NOT_MY_VBUCKET
for all new requests. The
issue arises when the library attempts to connect to this node. In this
case, the node may either not respond to the new connection or it may respond
with an authentication failure (because the node is really no longer part of
the cluster). This new version fixes this issue by only remapping items to
nodes that actually contain at least one vBucket.
Issues: CCBC-420
New Features and Behavior Changes in 2.3.1
Add lcb_cntl()
interface for configuration cache
Configuration cache options can be set after instantiation using lcb_cntl()
with the new LCB_CNTL_CONFIGCACHE
operation. The old-style lcb_create_compat
creation path is deprecated.
Issues: CCBC-395
Compare configuration revision information
for memcached cluster bootstrap. Previously we would refresh the
configuration upon receipt
of any new configuration update from memcached. This is fixed in 2.3.1 where
the configuration is applied only if it is deemed to be newer than the
current configuration. With memcached bootstrap this is true only if the
configuration’s rev
field is higher than the current one.
Fixes in 2.3.1
Segfault in connmgr_invoke_request
Occasionally a segmentation fault would happen when a connection was being
released as a result of a connection failure. This was because of invalid
list tracking.
Issues: CCBC-404
Get-with-replica occasionally crashes on Windows and UV during topology changes. This was due to not allocating a buffer if one did not exist.
Issues CCBC-394
Application binary interface (ABI) compatibility broken between 2.x and 2.3 for
lcb_create_compat
. This has been fixed by symbol aliasing between versions.
Use the lcb_cntl()
API to set the
configuration cache, as specified in CCBC-395.
Issues: CCBC-392
Failed assertion on get-with-replica when connection fails.
If a connection fails with a CMD_GET_REPLICA
command still in the queue an
assertion failure will crash the library. This has been fixed by handling the
opcode in the failout_single_request
function.
Issues: CCBC-385
Unknown Winsock error codes crash application. This was fixed by providing proper handlers for Winsock codes that were not explicitly converted into their POSIX equivalents.
Issues: CCBC-384
Fix memory leak in configuration parsing. A leak was introduced in version 2.3.0 by not freeing the JSON pool structure. This has been fixed in 2.3.1
Issues: CCBC-376
lcb_get_host
and lcb_get_port
might return host-port
combinations from different servers. If multiple servers are listening on
different ports this might result in yielding an invalid endpoint by combining
the output from those two functions. This has been fixed in 2.3.1 by returning
the host and port from the first host in lieu of a currently-connected REST
endpoint.
Issues: CCBC-370
Initial bootstrapping failure may mask LCB_BUCKET_ENOENT
calls with LCB_ETIMEDOUT
. This has been fixed by not retrying configuration
retrieval if an explicit HTTP 404 code is received. Note that when using
bootstrap over memcached, a missing bucket might still be manifest as
LCB_AUTH_ERROR
.
Issues: CCBC-368
Ensure lcb_get_host
does not return NULL
when the
associated lcb_t
is of LCB_TYPE_CLUSTER
. This would cause crashes in some
applications that relied on this function to not return NULL
.
Issues: CCBC-367
Fixed spurious timeouts being delivered in asynchronous
use cases.
In applications that do not use lcb_wait()
the library will potentially
time out commands internally triggering an erroneous configuration refresh.
While this issue would not end up failing operations it will cause unnecessary
network traffic for retrieving configurations. Applications using lcb_wait()
are not affected because that function resets the timeout handler.
Issues: CCBC-389
New Features and Behavior Changes in 2.3.0
Cluster Configuration Carrier Publication) (CCCP)
CCCP is the new and more efficient way to bootstrap from a cluster using the native memcached protocol and is quicker than the previous HTTP bootstrap mechanism, dramatically improving startup times and reducing load on the server. This feature is available in Couchbase Server version 2.5 and later. The existing HTTP configuration is still supported and is employed as a fallback in the event that CCCP is not supported.
In conjunction with this, a new struct version has been added to the lcb_create_st
parameters structure for use with lcb_create
. The new struct allows you to get more control over how the client is initialized. The following example configures a client to always use the CCCP protocol and never attempt to fall back on HTTP:
lcb_t instance;
struct lcb_create_st options;
lcb_config_transport_t enabled_transports[] = {
LCB_CONFIG_TRANSPORT_CCCP,
LCB_CONFIG_TRANSPORT_LIST_END
};
memset(&options, 0, sizeof(options));
options.version = 2;
options.v.v2.mchosts = "example.com:11210";
options.v.v2.transports = enabled_transports;
lcb_error_t rc = lcb_create(&instance, &options);
if (rc != LCB_SUCCESS) {
fprintf(stderr, "Failed to create instance: %s\n", lcb_strerror(instance, rc));
}
The CCCP implementation required a significant rewrite in how sockets were created and re-used. Particularly, a connection pooling feature was implemented.
The cbc
command now has an additional -C
option that accepts the preferred configuration mechanism to use.
Issues: CCBC-234
Logging hooks
This improvements adds various levels of diagnostic logging with the library
itself. It may be utilized via the environment (by setting the LCB_LOGLEVEL
environment variable to a positive integer – the higher the number the more
verbose the logging).
Integrators may also use the logging API specified in <libcouchbase/types.h>
to proxy the library’s logging messages into your own application.
Current events logged include connection initialization, destruction, connection pool management, configuration changes, and timeouts.
By default the library is silent.
Issues: CCBC-305
Per-node bootstrap/config timeouts
This change provides fine-grained control over how long to wait per-node to receive updated configuration information. This setting helps adjust the initial and subsequent bootstrap processes to help ensure each node gets a slice of time.
Issues: CCBC-316
Master-only observe option
This adds a new
struct version to the lcb_observe_cmd_t
which allows you to select only the
master node. You can use this to efficiently check whether the key exists (without
retrieving it). It also allows you to get the item’s CAS without fetching it.
Issues: CCBC-152
Fixes in 2.3.0
Handle spurious EWOULDBLOCK on UV/Win32 This issue caused odd errors on Windows when large amounts of data would be received on the socket.
Issues: CCBC-297
Correctly parse ""
, "0"
and "1"
for environment
variables. In previous versions having the entry set to an empty string
or 0
would still be treated by the library as a true value for various
environment variables. This has been fixed so that clear “False” values
such as the empty string or 0 are treated as such.
Issues: CCBC-340
Fix spurious callback deliveries on failed multi-packet commands. Multi-packet commands will no longer deliver spurious callbacks on failure. Previously these commands would be relocated to the same server during a configuration change, resulting in multiple callbacks for the same command. In this case the client would think all the commands had been completed, and when the next response arrived it would incorrectly map it to a different request.
Fix assertion upon receiving a slow HTTP streaming update which does not
receive a new topology. In such cases a user would receive an error like
this: src/bconf_io.c:175: instance_timeout_handler: Assertion instance->confstatus != LCB_CONFSTATE_CONFIGURED' failed.
.
Issues: CCBC-320
Known Issues in 2.3.0
SASL password stored in plaintext with configuration cache. This should be stripped from the file before it is written to disk
Issues: CCBC-347
Memcached buckets may receive operation failures during rebalance. The library should attempt to rehash the key and forward the item to the appropriate server
Issues: CCBC-331
New rev
field in configuration not used. Ideally the library should use this
to compare the validity of two configurations.
Issues: CCBC-332
Users of the lcb_compat_st
structure will need to recompile their application
to work with the new library. This is because the embedded lcb_create_st
structure has change. This affects any application using the configuration
cache as it is created using said structure.
Issues: CCBC-392
New Features and Behavior Changes in 2.2.0
Handle 302 redirects in HTTP (views and administrative requests). By
default the library follows up to three redirects. After the
limit is reached, the request is terminated with code
LCB_TOO_MANY_REDIRECTS
. The limit is configurable through
LCB_CNTL_MAX_REDIRECTS
. If it is set to -1, it disables the redirect
limit. The following example shows how to set the limit:
int new_value = 5;
lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_MAX_REDIRECTS, &new_value);
Issues: CCBC-169
Replace isasl with cbsasl. cbsasl implements both PLAIN and CRAM-MD5 authentication mechanisms.
LCB_CNTL_MEMDNODE_INFO
command updated to include effective
SASL mechanism:
cb_cntl_server_t node;
node.version = 1;
node.v.v1.index = 0; /* first node */
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_MEMDNODE_INFO, &node);
if (node.v.v1.sasl_mech) {
printf("authenticated via SASL '%s'\n",
node.v.v1.sasl_mech);
}
You can force a specific authentication mechanism for
the connection handle by using the LCB_CNTL_FORCE_SASL_MECH
command:
lcb_cntl(instance, LCB_CNTL_SET, LCB_CNTL_FORCE_SASL_MECH, "PLAIN");
Issues: CCBC-243
Stricter, more inspectable behavior for the configuration cache. This provides a
test and an additional lcb_cntl
operation to check the status of
the configuration cache. Also, it switches off the configuration cache with
memcached buckets.
int is_loaded;
lcb_cntl(instance, LCB_CNTL_GET, LCB_CNTL_CONFIG_CACHE_LOADED, &is_loaded);
if (is_loaded) {
printf("Configuration cache saved us a trip to the config server\n");
} else {
printf("We had to contact the configuration server for some reason\n");
}
Fixes in 2.2.0
libuv plugin: use the same CRT for free and malloc.
Issues : CCBC-286
Fail NOT_MY_VBUCKET
responses on time-out.
Issues : CCBC-288
Do a full purge when negotiation times out. In this case we must purge the server from all commands and not simply pop individual items.
Issues : CCBC-275
Reset the server’s buffers upon reconnection. This fixes a crash
experienced when requesting a new read with the previous buffer
still intact. This was exposed by calling lcb_failout_server
on a
time-out error while maintaining the same server struct.
Issues : CCBC-275
Make server buffers reentrant-safe. When purging implicit commands, we invoke callbacks that might in turn cause other LCB entry points to be invoked, which can shift the contents or positions of the ring buffers that we’re reading from.
Issues : CCBC-282
Use common config retry mechanism for bad configuration cache. This uses the
same error handling mechanism as when a bad configuration has been
received from the network. New LCB_CONFIG_CACHE_INVALID
error code
to notify the user of such a situation
Issues : CCBC-278
Handle getl and unl when purging the server (thanks Robert Groenenberg).
Issues : CCBC-274
Don’t fail out all commands on a time-out. Only fail those commands that are old enough to have timed out already.
Don’t record and use TTP/TTR from observe. Just poll at a fixed interval, because the responses from the server side can be unreliable.
Issues : CCBC-269
Allow hooks for mapping server codes to errors. This also helps handle sane behavior if a new error code is introduced, or allow user-defined logging when a specific error code is received.
lcb_errmap_callback default_callback;
lcb_error_t user_map_error(lcb_t instance, lcb_uint16_t in)
{
if (in == PROTOCOL_BINARY_RESPONSE_ETMPFAIL) {
fprintf(stderr, "temporary failure on server\n");
}
return default_callback(instance, in);
}
...
default_callback = lcb_set_errmap_callback(conn, user_map_error);
Add an example of a connection pool. See example/instancepool
directory
Force lcb_wait
return result of wait operation instead of
lcb_get_last_error
. It returns last_error
if and only if the
handle is not yet configured
Issues : CCBC-279
cbc-pillowfight
: compute item size correctly during set If
minSize
and maxSize
are set to the same value it can sometimes
crash since it may try to read out of memory bounds from the
allocated data buffer.
Issues : CCBC-284
Apply key prefix CLI option in cbc-pillowfight
Issues : CCBC-283
Add --enable-maintainer-mode
. Maintainer mode enables
--enable-werror --enable-warnings --enable-debug
, forces all
plugins to be installed and forces all tests, tools, and examples to
be built
Expose LCB_MAX_ERROR
to allow user-defined codes
Issues : CCBC-255
New Features and Behavior Changes in 2.1.3
Use cluster type connection for cbc-bucket-flush. Although flush command is accessible for bucket type connections, cbc-bucket-flush doesn’t use provided bucket name to connect to, therefore it will fail if the bucket name isn’t “default”.
Allow to make connect order deterministic. It allows the user to toggle between deterministic and random connect order for the supplied nodes list. By default it will randomize the list.
Fixes in 2.1.3
Updated gtest to version 1.7.0. Fixes issue with building test suite with new XCode 5.0 version being released later this month.
Do not try to parse config for LCB_TYPE_CLUSTER
handles. It fixes timouts for management operations (like ‘cbc
bucket-create’, ‘cbc bucket-flush’, ‘cbc bucket-delete’ and ‘cbc
admin’)
Issues : CCBC-265
Skip unfinished SASL commands on rebalance. During rebalance, it is possible that the newly added server doesn’t have chance to finish SASL auth before the cluster will push config update, in this case packet relocator messing cookies. Also the patch makes sure that SASL command/cookie isn’t mixing with other commands
Issues : CCBC-263
Do not allow to use Administrator account for
LCB_TYPE_BUCKET
Fig segmentation faults during tests load of
node.js. Sets inside_handler
on socket_connected
. Previously we
were always using SASL auth, and as such, we wouldn’t flush packets
from the cmd_log
using server_send_packets
(which calls
apply_want
). apply_want
shouldn’t be called more than once per
event loop entry – so this sets and unsets the inside_handler
flag.
Issues : CCBC-258
Added support of libuv 0.8
Close config connection before trying next node. It will fix
asserts in case of the config node becomes unresponsive, and the
threshold controlled by LCB_CNTL_CONFERRTHRESH
and lcb_cntl(3)
Fixes in 2.1.2
Use bucket name in SASL if username omitted. Without this fix, you can may encounter a segmentation faults for buckets, which are not protected by a password.
Preserve IO cookie in options_from_info
when using v0 plugins
with user-provided IO loop instance. This issue was introduced in
2.1.0.
Display the effective IO backend in cbc-version
. This is helpful
to quickly detect what is the effective IO plugin on a given
system.
New Features and Behavior Changes in 2.1.1
Fallback to ‘select’ IO plugin if default plugin cannot be loaded. On UNIX-like systems, default IO backend is ‘libevent’, which uses third-party library might be not available at the run-time. Read in lcb_cntl(3couchbase) man page in section LCB_CNTL_IOPS_DEFAULT_TYPES about how to determine effective IO plugin, when your code chose to use LCB_IO_OPS_DEFAULT during connection instantiation. The fallback mode doesn’t affect application which specify IO backend explicitly.
Issues : CCBC-246
Skip misconfigured nodes in the list. New lcb_cntl(3couchbase) added to control whether the library will skip nodes in initial node list, which listen on configuration port (8091 usually) but doesn’t meet required parameters (invalid authentication or missing bucket). By default report this issue and stop trying nodes from the list, like all previous release. Read more at man page lcb_cntl(3couchbase) in section LCB_CNTL_SKIP_CONFIGURATION_ERRORS_ON_CONNECT
Issues : CCBC-192
Distribute debug information with release binaries on Windows
Issues : CCBC-245
Fixes in 2.1.1
Do not use socket after failout. Fixes segmentation faults during rebalance.
Issues : CCBC-239
Use provided credentials for authenticating to the data nodes. With this fix, it is no longer possible to use Administrator credentials with a bucket. If your configuration does so, you must change the credentials you use before applying this update. No documentation guides use of Administrator credentials, so this change is not expected to affect few, if any deployments.
Do not disable config.h on UNIX-like platforms. It fixes build issue, when application is trying to include plugins from the tarball.
Issues : CCBC-248
New Features and Behavior Changes in 2.1.0
New backend select
. This backend is based on the select(2) system call and its
Windows version. It could be considered the most portable solution and is
available with the libcouchbase core.
API for durability operations. This new API is based on lcb_observe(3)
and
allows you to monitor keys more easily. See the man pages
lcb_durability_poll(3)
and lcb_set_durability_callback(3)
for more info.
Issues : CCBC-145
New backend libuv
. This backend previously was part of the
couchnode project and is now available
as a plugin. Because libuv doesn’t ship binary packages there is no binary
package libcouchbase2-libuv
. You can build plugin from the source
distribution, or through the libcouchbase-dev
or libcouchbase-devel
package
on UNIX like systems.
Issues : CCBC-236
New backend iocp
. This is a Windows specific backend, which uses “I/O
Completion Ports”. As a part of the change, a new version of plugin API was
introduced which is more optimized to this model of asynchronous IO.
New configuration interface lcb_cntl(3)
along with new tunable options of the
library and connection instances. In this release the following settings are
available. See the man page for more information and examples.:
LCB_CNTL_OP_TIMEOUT operation timeout (default 2.5 seconds)
LCB_CNTL_CONFIGURATION_TIMEOUT time to fetch cluster configuration. This is similar to a connection timeout (default 5 seconds)
LCB_CNTL_VIEW_TIMEOUT timeout for couchbase views (default 75 seconds)
LCB_CNTL_HTTP_TIMEOUT timeout for other HTTP operations like RESTful flush, bucket creating etc. (default 75 seconds)
LCB_CNTL_RBUFSIZE size of the internal read buffer (default 32768 bytes)
LCB_CNTL_WBUFSIZE size of the internal write buffer (default 32768 bytes)
LCB_CNTL_HANDLETYPE type of the lcb\_t
handler (readonly)
LCB_CNTL_VBCONFIG returns pointer to VBUCKET_CONFIG_HANDLE (readonly)
LCB_CNTL_IOPS get the implementation of IO (lcb_io_opt_t)
LCB_CNTL_VBMAP get vBucket ID for a given key
LCB_CNTL_MEMDNODE_INFO get memcached node info
LCB_CNTL_CONFIGNODE_INFO get config node info
LCB_CNTL_SYNCMODE control synchronous behavior (default LCB_ASYNCHRONOUS)
LCB_CNTL_IP6POLICY specify IPv4/IPv6 policy (default LCB_IPV6_DISABLED)
LCB_CNTL_CONFERRTHRESH control configuration error threshold (default 100)
LCB_CNTL_DURABILITY_TIMEOUT durability timeout (default 5 seconds)
LCB_CNTL_DURABILITY_INTERVAL durability polling interval (default 100 milliseconds)
LCB_CNTL_IOPS_DEFAULT_TYPES get the default IO types
LCB_CNTL_IOPS_DLOPEN_DEBUG control verbose printing of dynamic loading of IO plugins.
Fixes in 2.1.0
Fixed bug when REPLICA_SELECT
didn’t invoke callbacks for negative error codes
Issues : CCBC-228
Fixed bug when LCB_REPLICA_FIRST
fails if first try does not return key
Issues : CCBC-229
Known Issues in 2.1.0
From the release the 2.1.0 package libcouchbase2
will not install an IO
backend automatically. If you are upgrading, there are no changes because you
have already libcouchbase2-libev
or libcouchbase2-libevent
packages
installed. For new installations, a backend must be selected for the client
library to work correctly.
If for example you are using the PHP SDK, the old way, which works for pre-2.1.0 versions is:
# DEB-based systems
shell> sudo apt-get install libcouchbase2 libcouchbase-dev
# RPM-based systems
shell> sudo yum install libcouchbase2 libcouchbase-devel
But a more explicit way to do this, which works for all versions (including 2.1.0) is:
# DEB-based systems
shell> sudo apt-get install libcouchbase2-libevent libcouchbase-dev
# RPM-based systems
shell> sudo yum install libcouchbase2-libevent libcouchbase-devel
New Features and Behavior Changes in 2.0.7
Improve lcb\_get\_replica()
. Now it is possible to choose between three
strategies:
LCB_REPLICA_FIRST
: Previously accessible and now the default, the caller will
get a reply from the first replica to successfully reply within the timeout for
the operation or will receive an error.
LCB_REPLICA_ALL
: Ask all replicas to send documents/items back.
LCB_REPLICA_SELECT
: Select one replica by the index in the configuration
starting from zero. This approach can more quickly receive all possible replies
for a given topology, but it can also generate false negatives.
Note that applications should not assume the order of the replicas indicates more recent data is at a lower index number. It is up to the application to determine which version of a document/item it may wish to use in the case of retrieving data from a replica.
Issues : CCBC-183
New Features and Behavior Changes in 2.0.6
Added an example to properly use the bucket credentials for authentication instead of administrator credentials
Issues : CCBC-179
Add Host header in http request http://cbugg.hq.couchbase.com/bug/bug-555 points out that Host is a required field in HTTP 1.1
Issues : CCBC-201
Fixes in 2.0.6
Fix segfault when rebalancing. When a (!connected) server is reconnected, the tasks in its “pending” buffer will be moved into “output” buffer. If its connection is broken again immediately, relocate_packets() will go to wrong path.
Issues : CCBC-188
Don’t try to switch to backup nodes when timeout is reached
Issues : CCBC-202
Fix compile error with sun studio. "src/event.c", line 172: error: statement
not reached (E_STATEMENT_NOT_REACHED)
Don’t invoke HTTP callbacks after cancellation, because user code might assume a previously-freed resource is still valid
Check if SASL struct is valid before disposing
Issues : CCBC-188
example/yajl/couchview.c: pass cookie to the command Fixes coredump when executing./examples/yajl/couchview
New Features and Behavior Changes in 2.0.5
Fixes in 2.0.5
Try to search the –libdir for modules if dlopen fails to find the module in the default library path
New compat mode (experimental) for configuration caching. See man lcb_create_compat
Issues : CCBC-190
Fix reconnecting issues on windows (http://review.couchbase.org/25170 and http://review.couchbase.org/25155)
Fix build on FreeBSD (http://review.couchbase.org/25289)
Fixes in 2.0.4
Build error on solaris/sparc: -Werror=cast-align
Issues : CCBC-178
Fixed illegal memory access in win32 plugin
Issues : CCBC-147
Work properly on systems where EWOULDBLOCK != EAGAIN
Issues : CCBC-175
The library stops iterating backup nodes list if the next one isn’t accessible.
Issues : CCBC-182
The bootstrap URI is not parsed correctly
Issues : CCBC-185
Segmentation fault when the hostname resolved into several addresses and first of them reject couchbase connections.
Issues : CCBC-180
New Features and Behavior Changes in 2.0.3
Add a new library: libcouchbase_debug.so (see include/libcouchbase/debug.h) which is a new library that contains new debug functionality.
Added manual pages for the library.
Fixes in 2.0.3
Observe malfunctions in the case of multiple keys and server failure.
Issues : CCBC-155
Reset internal state on lcb_connect(). Allow caller to use lcb_connect() multiple times to implement reconnecting using the same lcb_t instance. Also it sets up the initial-connection timer for users who don’t use lcb_wait() and drive IO loop manually.
Issues : CCBC-153
Invalid read in libevent plugin, when the plugin compiled in 1.x mode
Issues : CCBC-171
Shrink internal lookup tables (and reduce the size of lcb_t)
Issues : CCBC-156
Fixes in 2.0.2
Document LCB_SERVER_BUG and LCB_PLUGIN_VERSION_MISMATCH. Enhance the lcb_strerror test to detect undocumented error codes.
Commands sent to multiple servers fail to detect the respose if mixed with other commands.
Issues : CCBC-150
Under high load the library could generate LCB_ETIMEDOUT errors without reason owing to internal limitations.
Issues : CCBC-153
Cancellation of the HTTP request might lead to memory leaks or to segfaults (2e3875c2).
Issues : CCBC-151
New Features and Behavior Changes in 2.0.1
Fixes in 2.0.1
Fix a memory leak on the use of http headers
Issues : CCBC-130
libev-plugin: delay all timers while the loop isn’t active. It will fix LCB_ETIMEOUT in the following scenario:
connect the instance
sleep for time greater than default timeout (e.g. 3 seconds)
schedule and execute a command (it will be timed out immediately)
Do not abort when purging SASL commands
Issues : CCBC-136
Fix possible SEGFAULT. Not-periodic timers are destroyed after calling user’s callback, after that library performed read from freed pointer.
Compensate for cluster nodes lacking couchApiBase
Issues : CCBC-131
Ensure HTTP works even when the network may be unreliable. This changeset encompasses several issues which had been found with HTTP requests during network errors and configuration changes. Specifically some duplicate code paths were removed, and the process for delivering an HTTP response back to the user is more streamlined.
libev-plugin: reset IO event on delete. We need to reset it, because it might be re-used later
Make library C89 friendly again
New Features and Behavior Changes in 2.0.0
Fixes in 2.0.0
Minor update of the packaging layout:
libcouchbase-all package comes without version
extract debug symbols from libcouchbase-{bin,core} to libcouchbase-dbg package
Install unlock callback in synchronous mode
New Features and Behavior Changes in 2.0.0beta3
Try all known plugins for LCB_IO_OPS_DEFAULT in run time
Allow to use ‘cbc-hash’ with files
Create man pages for cbc and cbcrc
Use dynamic versioning for plugins
Lookup the plugin symbol also in the current executable image.
Issues : CCBC-114
Allow the user to specify a different hash key. All of the data operations contains a hashkey and nhashkey field. This allows you to “group” items together in your cluster. A typical use case for this is if you’re storing lets say data for a single user in multiple objects. If you want to ensure that either all or none of the objects are available if a server goes down, it could be a good idea to locate them on the same server. Do bear in mind that if you do try to decide where objects is located, you may end up with an uneven distribution of the number of items on each node. This will again result in some nodes being more busy than others etc. This is why some clients doesn’t allow you to do this, so bear in mind that by doing so you might not be able to get your objects from other clients.
Issues : CCBC-119
Add documentation about the error codes
Issues : CCBC-87
Add lcb_verify_compiler_setup(). This function allows the “user” of the library to verify that the compiler use a compatible struct packing scheme.
Fixes in 2.0.0beta3
lcb_error_t member in the http callbacks shouldn’t reflect the HTTP response code. So the error code will be always LCB_SUCCESS if the library managed to receive the data successfully.
Issues : CCBC-118
Fix cbc-bucket-create. `sasl-password' is misspelled, and it fails to parse the command line option.
Remove libtool version from the plugins
Do not allow admin operations without authentication
check for ewouldblock/eintr on failed send
Purge stale OBSERVE packets
Issues : CCBC-120
Allow to use gethrtime() from C++
Remove unauthorized asserion (d344037). The lcb_server_send_packets() function later check if the server object connected and establish connection if not (with raising possible errors)
Issues : CCBC-113
Don’t use the time_t for win32. When compiling from php it turns out that it gets another size of the time_t type, causing the struct offsets to differ.
Reformat and refactor lcb_server_purge_implicit_responses:
move packet allocation out of GET handler
dropping NOOP command shouldn’t return error code
Issues : CCBC-120
Try to switch another server from backup list on timeout
Issues : CCBC-122
Timer in libev uses double for interval. Ref: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#code_ev_timer_code_relative_and_opti
Fix illegal memory access. Reconnect config listener if the config connection was gone without proper shutdown.
Issues : CCBC-104
Fix using freed memory (was introduced in 4397181)
Return zero from do_read_data() if operations_per_call reached. The `operations_per_call' limit was introduced to prevent from freezing event loop. But in the function variable rv could store two different results and in case of reaching this limit it is returning number of the processed records, which is wrong. The function should return either zero (success) or non-zero (failure).
Issues : CCBC-115
New Features and Behavior Changes in 2.0.0beta2
Implement a new libev plugin. It is compatible with both libev3 and libev4.
Allow libcouchbase to connect to an instance without specifying bucket. It is useful when the bucket not needed, e.g. when performing administration tasks.
Allow users to build the library without dependencies. For example, without plugins at all. This may be useful if the plugin is implemented by or built into the host application.
Allow users to use environment variables to pick the event plugin
Add a new interface version for creating IO objects via plugins
Allow to disable CXX targets
Allow users to install both libraries (2.x and 1.x) on the same system.
Cleanup HTTP callbacks. Use the same callbacks both for Management and View commands, and rename them to lcb_http_complete_callback and lcb_http_data_callback.
Add support for raw http requests. libcouchase already contains all the bits to execute a raw http request, except for the possibility to specify a host:port, username and password.
Make the content type optional for lcb_make_http_request()
Fixes in 2.0.0beta2
Fix invalid memory access in cbc tool. Affected command is cbc-bucket-create
Search ev.h also in ${includedir}/libev
Fix password memory leak in http.c (7e71493)
lcb_create: replace assert() with error code
Breakout event loop in default error_callback. This provides better default behavior for users who haven’t defined global error callback.
Issues : CCBC-105
Fix linked event/timer lists for win32
Issues : CCBC-103
Fix SEGFAULT if IO struct is allocated not by the lcb_create()
Fix memory leak after an unsuccessful connection
lcb_connect() should honor the syncmode setting. Automatically call lcb_wait() when in synchronous mode
New Features and Behavior Changes in 2.0.0beta
Bundle Windows packages as zip archives
Refactor the API. This is a full redesign of the current libcouchbase API that’ll allow us to extend parts of the API without breaking binary compatibility. Also it renames all functions to have lcb prefix instead of libcouchbase and LCB/LIBCOUCHBASE in macros.
Implement getter for number of nodes in the cluster: lcb_get_num_nodes()
Add lcb_get_server_list() to get current server list
Deliver HTTP headers via callbacks
Implement RESTful flush in the cbc toolset
Merge lcb_get_locked into lcb_get function
Bundle libvbucket
Fixes in 2.0.0beta
Fix a problem with allocating too few slots in the backup_nodes. Fixes illegal memory access.
Include sys/uio.h. Needed by OpenBSD
Added --enable-fat-binary
. Helps to solve issues when linking with fat
binaries on MacOS.
Differentiate between TMPFAILs. This allows a developer to know if the temporary condition where the request cannot be handled is due to a constraint on the client or the server.
Issues : CCBC-98
Correct buffer length for POST/PUT headers
Issues : CCBC-96
Fix switching to backup node in case of server outage
Issues : CCBC-91
Fix locking keys in multi-get mode
Release the memory allocated by the http parser
Issues : CCBC-89
Fix initialization of backup nodes array. The code switching nodes relies on NULL terminator rather than nbackup_nodes variable. Fixes illegal memory access.
Issues : CCBC-90
Default to IPv4 only
Issues : CCBC-80
Reset timer for commands with NOT_MY_VBUCKET response
Issues : CCBC-91
Sync memcached/protocol_binary.h. Pull extra protocol_binary_datatypes declarations.
Fix bug where HTTP method is not set
Don’t try to put the current node last in the backup list. This may cause “duplicates” in the list if the REST server returns another name for the server than you used. Ex: you specify “localhost” and the REST response contains 127.0.0.1
Release ringbuffer in lcb_purge_single_server
Issues : CCBC-92
Fixes in 1.1.0dp9
New Features and Behavior Changes in 1.1.0dp8
Allow the user to get the number of replicas using libcouchbase_get_num_replicas()
Separate HTTP callbacks for couch and management requests
Implement read replica
Issues : CCBC-82
Let users detect if the event loop running already using libcouchbase_is_waiting() function.
Add OBSERVE command
Issues : CCBC-15
Allow users to specify content type for HTTP request.
Allow a user to breakout from the event loop in callbacks using libcouchbase_breakout()
New cbc commands and options:
cbc-view (remove couchview example)
cbc-verbosity
cbc-admin
cbc-bucket-delete
cbc-bucket-create
Add -p and -r options to cbc-cp to control persistence (uses OBSERVE internally)
Allow the client to specify the verbosity level on the servers using lcb_set_verbosity() function.
Implement general purpose timers. It is possible for users to define their own timers using libcouchbase_timer_create() function. (See headers for more info). Implement multiple timers for windows
Issues : CCBC-85
Fixes in 1.1.0dp8
Claim that server has data in buffers if there are HTTP requests pending. Without this patch the event loop can be stopped prematurely.
Make libcouchbase_wait() re-entrable
Fix to handle the case when View base doesn’t have URI schema.
Bind timeouts to server sockets instead of commands. This means that from this point timeout interval will be started from the latest IO activity on the socket. This is a behavior change from the 1.0 series.
Use separate error code for ENOMEM on the client
Issues : CCBC-77
New Features and Behavior Changes in 1.1.0dp7
Implement function to execution management requests. Using libcouchbase_make_management_request() function you can configure the cluster, add/remove buckets, rebalance etc. It behaves like libcouchbase_make_couch_request() but works with another endpoint.
Add support for notification callbacks for configuration changes. Now it is possible to install a hook using function libcouchbase_set_configuration_callback(), and be notified about all configuration changes.
Fixes in 1.1.0dp7
New Features and Behavior Changes in 1.1.0dp6
Implement ‘help’ command for cbc tool
Issues : RCBC-71
Fixes in 1.1.0dp6
Undefine NDEBUG to avoid asserts to be optimized out
Fix compilation on macosx with gtest from homebrew
Issues : RCBC-72
Close dynamic libraries. Fixes small memory leak
Issues : RCBC-70
Include types definitions for POSIX systems. Fixes C++ builds on some systems.
Issues : RCBC-63
Fix win32 builds:
Add suffix to cbc command implementations;
Fix guards for socket errno macros;
Define size_t types to fix MSVC 9 build;
MSVC 9 isn’t C99, but has stddef.h, so just include it.
New Features and Behavior Changes in 1.1.0dp5
Fixes in 1.1.0dp5
Fixes in 1.1.0dp4
New Features and Behavior Changes in 1.1.0dp3
Implement GET_LOCKED (GETL) command
Issues : CCBC-68
Implement UNLOCK_KEY (UNL) command
Issues : CCBC-68
Fixes in 1.1.0dp3
Timeouts can occur during topology changes, rather than be correctly retried. Send the retry-packet to new server
Issues : CCBC-64
Fix ringbuffer_memcpy() (36afdb2). Fix ringbuffer_is_continous().
A hang could occur in libcouchbase_wait() after the timeout period. Check for breakout condition after purging servers
Issues : CCBC-62
Destroy view requests items when server get destroyed. Fixes memory leaks.
A fix for a buffer overflow with the supplied password as has been integrated. While it is a buffer overflow issue, this is not considered to be a possible security issue because the password to the bucket is not commonly supplied by an untrusted source
Issues : RCBC-33
A small memory leak can occur with frequent calls to libcouchbase_create() and libcouchbase_destroy()
Issues : CCBC-65
Use vbucket_found_incorrect_master() to get correct server index during relocating buffers. Pick up cookies from pending buffer unless node connected.
Do not call View callbacks for cancelled requests
hashset.c: iterate over whole set on rehashing. Fixes memory leaks related to hash collisions (905ef95)
Fixes in 1.1.0dp2
Don’t wait for empty buffers. If called with no operations queued, libcouchbase_wait() will block forever. This means that a single threaded application that calls libcouchbase_wait() at different times to make sure operations are sent to the server runs the risk of stalling indefinitely. This is a very likely scenario.
Issues : CCBC-59
Don’t define size_t and ssize_t for VS2008
Fix segfault while authorizing on protected buckets (211bb04)
New Features and Behavior Changes in 1.1.0dp
libcouchbase_make_couch_request()
function. See the associated
documentation and header files for more details.Fixes in 1.1.0dp
Request the tap bytes in a known byte order (adf2b30)
Issues : MB-4834
Check for newer libvbucket