Couchbase C Client  2.5.6
Durability

Detailed Description

Ensure persistence and mutation of documents.

Wait for a mutation to be persisted/replicated

enum  lcb_DURMODE
 Type of durability polling to use. More...
 
lcb_MULTICMD_CTXlcb_endure3_ctxnew (lcb_t instance, const lcb_durability_opts_t *options, lcb_error_t *err)
 Return a new command context for scheduling endure operations. More...
 
lcb_error_t lcb_storedur3 (lcb_t instance, const void *cookie, const lcb_CMDSTOREDUR *cmd)
 
lcb_error_t lcb_durability_validate (lcb_t instance, lcb_U16 *persist_to, lcb_U16 *replicate_to, int options)
 
#define LCB_CMDENDURE_F_MUTATION_TOKEN
 Must specify this flag if using the 'mutation_token' field, as it was added in a later version.
 
#define LCB_DURABILITY_VALIDATE_CAPMAX
 

Retrieve current persistence/replication status

enum  lcb_observe_t
 Possible statuses for keys in OBSERVE response. More...
 
lcb_MULTICMD_CTXlcb_observe3_ctxnew (lcb_t instance)
 Create a new multi context for an observe operation. More...
 
lcb_error_t lcb_observe_seqno3 (lcb_t instance, const void *cookie, const lcb_CMDOBSEQNO *cmd)
 Get the persistence/replication status for a given mutation token. More...
 
#define LCB_CMDOBSERVE_F_MASTER_ONLY
 Set this bit in the cmdflags field to indicate that only the master node should be contacted.
 

Function Documentation

lcb_MULTICMD_CTX* lcb_endure3_ctxnew ( lcb_t  instance,
const lcb_durability_opts_t options,
lcb_error_t err 
)

Return a new command context for scheduling endure operations.

Stability
Committed:

Ensure a key is replicated to a set of nodes

The lcb_endure3_ctxnew() API is used to wait asynchronously until the item have been persisted and/or replicated to at least the number of nodes specified in the durability options.

The command is implemented by sending a series of OBSERVE broadcasts (see lcb_observe3_ctxnew()) to all the nodes in the cluster which are either master or replica for a specific key. It polls repeatedly (see lcb_DURABILITYOPTSv0::interval) until all the items have been persisted and/or replicated to the number of nodes specified in the criteria, or the timeout (aee lcb_DURABILITYOPTsv0::timeout) has been reached.

The lcb_DURABILITYOPTSv0::persist_to and lcb_DURABILITYOPTS::replicate_to control the number of nodes the item must be persisted/replicated to in order for the durability polling to succeed.

Parameters
instancethe instance
optionsa structure containing the various criteria needed for durability requirements
[out]errError code if a new context could not be created
Returns
the new context, or NULL on error. Note that in addition to memory allocation failure, this function might also return NULL because the options structure contained bad values. Always check the err result.
Scheduling Errors
The following errors may be encountered when scheduling:
Callback Errors
The following errors may be returned in the callback:
Creating request context
memset(&dopts, 0, sizeof dopts);
dopts.v.v0.persist_to = -1;
dopts.v.v0.replicate_to = -1;
dopts.v.v0.cap_max = 1;
mctx = lcb_endure3_ctxnew(instance, &dopts, &rc);
// Check mctx != NULL and rc == LCB_SUCCESS
Adding keys - CAS
This can be used to add keys using CAS-based durability. This shows an example within a store callback.
lcb_RESPSTORE *resp = get_store_response();
lcb_CMDENDURE cmd = { 0 };
LCB_CMD_SET_KEY(&cmd, resp->key, resp->nkey);
cmd.cas = resp->cas;
rc = mctx->addcmd(mctx, (const lcb_CMDBASE*)&cmd);
rc = mctx->done(mctx, cookie);
Adding keys - explicit sequence number
Shows how to use an explicit sequence number as a basis for polling
// during instance creation:
lcb_cntl_string(instance, "fetch_mutation_tokens", "true");
lcb_connect(instance);
// ...
lcb_RESPSTORE *resp = get_store_response();
lcb_CMDENDURE cmd = { 0 };
LCB_CMD_SET_KEY(&cmd, resp->key, resp->nkey);
cmd.mutation_token = &resp->mutation_token;
rc = mctx->addcmd(mctx, (const lcb_CMDBASE*)&cmd);
rc = mctx->done(mctx, cookie);
Adding keys - implicit sequence number
Shows how to use an implicit mutation token (internally tracked by the library) for durability:
// during instance creation
lcb_cntl_string(instance, "fetch_mutation_tokens", "true");
lcb_cntl_string(instance, "dur_mutation_tokens", "true");
lcb_connect(instance);
// ...
lcb_CMDENDURE cmd = { 0 };
LCB_CMD_SET_KEY(&cmd, "key", strlen("key"));
mctx->addcmd(mctx, (const lcb_CMDBASE*)&cmd);
mctx->done(mctx, cookie);
Response
lcb_install_callback3(instance, LCB_CALLBACK_ENDURE, dur_callback);
void dur_callback(lcb_t instance, int cbtype, const lcb_RESPBASE *rb)
{
const lcb_RESPENDURE *resp = (const lcb_RESPENDURE*)rb;
printf("Durability polling result for %.*s.. ", (int)resp->nkey, resp->key);
if (resp->rc != LCB_SUCCESS) {
printf("Failed: %s\n", lcb_strerror(NULL, resp->rc);
return;
}
}
lcb_error_t lcb_storedur3 ( lcb_t  instance,
const void *  cookie,
const lcb_CMDSTOREDUR cmd 
)
Stability
Volatile:

Schedule a storage operation with subsequent durability checking.

This is a compound of a logical lcb_store3() followed by an lcb_endure3_ctxnew() upon success.

lcb_error_t lcb_durability_validate ( lcb_t  instance,
lcb_U16 *  persist_to,
lcb_U16 *  replicate_to,
int  options 
)
Stability
Committed:

Validate durability settings.

This function will validate (and optionally modify) the settings. This is helpful to ensure the durability options are valid before issuing a command

Parameters
instancethe instance
[in,out]persist_toThe desired number of servers to persist to. If the CAPMAX flag is set, on output this will contain the effective number of servers the item can be persisted to
[in,out]replicate_toThe desired number of servers to replicate to. If the CAPMAX flag is set, on output this will contain the effective number of servers the item can be replicated to
optionsOptions to use for validating. The only recognized option is currently LCB_DURABILITY_VALIDATE_CAPMAX which has the same semantics as lcb_DURABILITYOPTSv0::cap_max.
Returns
LCB_SUCCESS on success
LCB_DURABILITY_ETOOMANY if the requirements exceed the number of servers currently available, and CAPMAX was not specifie
LCB_EINVAL if both persist_to and replicate_to are 0.
lcb_MULTICMD_CTX* lcb_observe3_ctxnew ( lcb_t  instance)

Create a new multi context for an observe operation.

Stability
Committed:
Parameters
instancethe instance
Returns
a new multi command context, or NULL on allocation failure.

Note that the callback for this command will be invoked multiple times, one for each node. To determine when no more callbacks will be invoked, check for the LCB_RESP_F_FINAL flag inside the lcb_RESPOBSERVE::rflags field.

void callback(lcb_t, lcb_CALLBACKTYPE, const lcb_RESPOBSERVE *resp)
{
if (resp->rflags & LCB_RESP_F_FINAL) {
return;
}
printf("Got status for key %.*s\n", (int)resp->nkey, resp->key);
printf(" Node Type: %s\n", resp->ismaster ? "MASTER" : "REPLICA");
printf(" Status: 0x%x\n", resp->status);
printf(" Current CAS: 0x%"PRIx64"\n", resp->cas);
}
lcb_CMDOBSERVE cmd = { 0 };
LCB_CMD_SET_KEY(&cmd, "key", 3);
mctx->addcmd(mctx, (lcb_CMDBASE *)&cmd);
mctx->done(mctx, cookie);
lcb_install_callback3(instance, LCB_CALLBACK_OBSERVE, (lcb_RESP_cb)callback);
Warning
Operations created by observe cannot be undone using lcb_sched_fail().
lcb_error_t lcb_observe_seqno3 ( lcb_t  instance,
const void *  cookie,
const lcb_CMDOBSEQNO cmd 
)

Get the persistence/replication status for a given mutation token.

Stability
Volatile:
Parameters
instancethe handle
cookiecallback cookie
cmdthe command

Data Structure Documentation

struct lcb_DURABILITYOPTSv0
Data Fields
lcb_U32 timeout Upper limit in microseconds from the scheduling of the command.

When this timeout occurs, all remaining non-verified keys will have their callbacks invoked with LCB_ETIMEDOUT.

If this field is not set, the value of LCB_CNTL_DURABILITY_TIMEOUT will be used.

lcb_U32 interval The durability check may involve more than a single call to observe - or more than a single packet sent to a server to check the key status.

This value determines the time to wait (in microseconds) between multiple probes for the same server. If not set, the LCB_CNTL_DURABILITY_INTERVAL will be used instead.

lcb_U16 persist_to how many nodes the key should be persisted to (including master).

If set to 0 then persistence will not be checked. If set to a large number (i.e. UINT16_MAX) and cap_max is also set, will be set to the maximum number of nodes to which persistence is possible (which will always contain at least the master node).

The maximum valid value for this field is 1 + the total number of configured replicas for the bucket which are part of the cluster. If this number is higher then it will either be automatically capped to the maximum available if (cap_max is set) or will result in an LCB_DURABILITY_ETOOMANY error.

lcb_U16 replicate_to how many nodes the key should be persisted to (excluding master).

If set to 0 then replication will not be checked. If set to a large number (i.e. UINT16_MAX) and cap_max is also set, will be set to the maximum number of nodes to which replication is possible (which may be 0 if the bucket is not configured for replicas).

The maximum valid value for this field is the total number of configured replicas which are part of the cluster. If this number is higher then it will either be automatically capped to the maximum available if (cap_max is set) or will result in an LCB_DURABILITY_ETOOMANY error.

lcb_U8 check_delete this flag inverts the sense of the durability check and ensures that the key does not exist.

This should be used if checking durability after an lcb_remove3() operation.

lcb_U8 cap_max If replication/persistence requirements are excessive, cap to the maximum available.
lcb_U8 pollopts Set the polling method to use.

The value for this field should be one of the lcb_DURMODE constants.

struct lcb_durability_opts_t

Options for lcb_endure3_ctxnew() (wrapper)

See also
lcb_DURABILITYOPTSv0
Data Fields
int version
union lcb_durability_opts_t v
struct lcb_CMDENDURE

Command structure for endure.

If the lcb_CMDENDURE::cas field is specified, the operation will check and verify that the CAS found on each of the nodes matches the CAS specified and only then consider the item to be replicated and/or persisted to the nodes. If the item exists on the master's cache with a different CAS then the operation will fail

Data Fields
lcb_U32 cmdflags Common flags for the command.

These modify the command itself. Currently the lower 16 bits of this field are reserved, and the higher 16 bits are used for individual commands.

lcb_U32 exptime Specify the expiration time.

This is either an absolute Unix time stamp or a relative offset from now, in seconds. If the value of this number is greater than the value of thirty days in seconds, then it is a Unix timestamp. This field is used in mutation operations (lcb_store3()) to indicate the lifetime of the item. It is used in lcb_get3() with the lcb_CMDGET::lock option to indicate the lock expiration itself.

lcb_U64 cas The known CAS of the item.

This is passed to mutation to commands to ensure the item is only changed if the server-side CAS value matches the one specified here. For other operations (such as lcb_CMDENDURE) this is used to ensure that the item has been persisted/replicated to a number of servers with the value specified here.

lcb_KEYBUF key The key for the document itself.

This should be set via LCB_CMD_SET_KEY()

lcb_KEYBUF _hashkey
Stability
Volatile:
const lcb_MUTATION_TOKEN * mutation_token
struct lcb_RESPENDURE

Response structure for endure.

Data Fields
void * cookie Application-defined pointer passed as the cookie parameter when scheduling the command.
const void * key Key for request.
lcb_SIZE nkey Size of key.
lcb_CAS cas CAS for response (if applicable)
lcb_error_t rc Status code.
lcb_U16 version ABI version for response.
lcb_U16 rflags Response specific flags.

see lcb_RESPFLAGS

lcb_U16 nresponses Total number of polls (i.e.

how many packets per server) did this operation require

lcb_U8 exists_master Whether this item exists in the master in its current form.

This can be true even if rc is not successful

lcb_U8 persisted_master True if item was persisted on the master node.

This may be true even if rc is not successful.

lcb_U8 npersisted Total number of nodes (including master) on which this mutation has been persisted.

Valid even if rc is not successful.

lcb_U8 nreplicated Total number of replica nodes to which this mutation has been replicated.

Valid even if rc is not successful.

struct lcb_CMDSTOREDUR

Command structure for lcb_storedur3() This is much like lcb_CMDSTORE, but also includes durability options.

Data Fields

lcb_U32 cmdflags
 Common flags for the command. More...
 
lcb_U32 exptime
 Specify the expiration time. More...
 
lcb_U64 cas
 The known CAS of the item. More...
 
lcb_KEYBUF key
 The key for the document itself. More...
 
lcb_KEYBUF _hashkey
 
lcb_VALBUF value
 
lcb_U32 flags
 
lcb_storage_t operation
 
char persist_to
 Number of nodes to persist to. More...
 
char replicate_to
 Number of nodes to replicate to. More...
 

Field Documentation

lcb_U32 cmdflags

Common flags for the command.

These modify the command itself. Currently the lower 16 bits of this field are reserved, and the higher 16 bits are used for individual commands.

lcb_U32 exptime

Specify the expiration time.

This is either an absolute Unix time stamp or a relative offset from now, in seconds. If the value of this number is greater than the value of thirty days in seconds, then it is a Unix timestamp. This field is used in mutation operations (lcb_store3()) to indicate the lifetime of the item. It is used in lcb_get3() with the lcb_CMDGET::lock option to indicate the lock expiration itself.

lcb_U64 cas

The known CAS of the item.

This is passed to mutation to commands to ensure the item is only changed if the server-side CAS value matches the one specified here. For other operations (such as lcb_CMDENDURE) this is used to ensure that the item has been persisted/replicated to a number of servers with the value specified here.

The key for the document itself.

This should be set via LCB_CMD_SET_KEY()

lcb_KEYBUF _hashkey
Stability
Volatile:
lcb_U32 flags
char persist_to

Number of nodes to persist to.

If negative, will be capped at the maximum allowable for the current cluster.

See also
lcb_DURABILITYOPTSv0::persist_to
char replicate_to

Number of nodes to replicate to.

If negative, will be capped at the maximum allowable for the current cluster.

See also
lcb_DURABILITYOPTSv0::replicate_to
struct lcb_RESPSTOREDUR

Response structure for `LCB_CALLBACK_STOREDUR.

Data Fields
void * cookie Application-defined pointer passed as the cookie parameter when scheduling the command.
const void * key Key for request.
lcb_SIZE nkey Size of key.
lcb_CAS cas CAS for response (if applicable)
lcb_error_t rc Status code.
lcb_U16 version ABI version for response.
lcb_U16 rflags Response specific flags.

see lcb_RESPFLAGS

const lcb_RESPENDURE * dur_resp Internal durability response structure.

This should never be NULL

int store_ok If the rc field is not LCB_SUCCESS, this field indicates what failed.

If this field is nonzero, then the store operation failed, but the durability checking failed. If this field is zero then the actual storage operation failed.

struct lcb_CMDOBSERVE

Structure for an observe request.

To request the status from only the master node of the key, set the LCB_CMDOBSERVE_F_MASTERONLY bit inside the lcb_CMDOBSERVE::cmdflags field

Data Fields
lcb_U32 cmdflags Common flags for the command.

These modify the command itself. Currently the lower 16 bits of this field are reserved, and the higher 16 bits are used for individual commands.

lcb_U32 exptime Specify the expiration time.

This is either an absolute Unix time stamp or a relative offset from now, in seconds. If the value of this number is greater than the value of thirty days in seconds, then it is a Unix timestamp. This field is used in mutation operations (lcb_store3()) to indicate the lifetime of the item. It is used in lcb_get3() with the lcb_CMDGET::lock option to indicate the lock expiration itself.

lcb_U64 cas The known CAS of the item.

This is passed to mutation to commands to ensure the item is only changed if the server-side CAS value matches the one specified here. For other operations (such as lcb_CMDENDURE) this is used to ensure that the item has been persisted/replicated to a number of servers with the value specified here.

lcb_KEYBUF key The key for the document itself.

This should be set via LCB_CMD_SET_KEY()

lcb_KEYBUF _hashkey
Stability
Volatile:
const lcb_U16 * servers_ For internal use: This determines the servers the command should be routed to.

Each entry is an index within the server.

size_t nservers_
struct lcb_RESPOBSERVE

Response structure for an observe command.

Note that the lcb_RESPOBSERVE::cas contains the CAS of the item as it is stored within that specific server. The CAS may be incorrect or stale unless lcb_RESPOBSERVE::ismaster is true.

Data Fields
void * cookie Application-defined pointer passed as the cookie parameter when scheduling the command.
const void * key Key for request.
lcb_SIZE nkey Size of key.
lcb_CAS cas CAS for response (if applicable)
lcb_error_t rc Status code.
lcb_U16 version ABI version for response.
lcb_U16 rflags Response specific flags.

see lcb_RESPFLAGS

lcb_U8 status Bit set of flags.
lcb_U8 ismaster Set to true if this response came from the master node.
lcb_U32 ttp Unused.

For internal requests, contains the server index

lcb_U32 ttr Unused.
struct lcb_CMDOBSEQNO

Command structure for lcb_observe_seqno3().

Note key, #nkey, and cas are not used in this command.

Data Fields
lcb_U32 cmdflags Common flags for the command.

These modify the command itself. Currently the lower 16 bits of this field are reserved, and the higher 16 bits are used for individual commands.

lcb_U32 exptime Specify the expiration time.

This is either an absolute Unix time stamp or a relative offset from now, in seconds. If the value of this number is greater than the value of thirty days in seconds, then it is a Unix timestamp. This field is used in mutation operations (lcb_store3()) to indicate the lifetime of the item. It is used in lcb_get3() with the lcb_CMDGET::lock option to indicate the lock expiration itself.

lcb_U64 cas The known CAS of the item.

This is passed to mutation to commands to ensure the item is only changed if the server-side CAS value matches the one specified here. For other operations (such as lcb_CMDENDURE) this is used to ensure that the item has been persisted/replicated to a number of servers with the value specified here.

lcb_KEYBUF key The key for the document itself.

This should be set via LCB_CMD_SET_KEY()

lcb_KEYBUF _hashkey
Stability
Volatile:
lcb_U16 server_index Server index to target.

The server index must be valid and must also be either a master or a replica for the vBucket indicated in vbid

lcb_U16 vbid vBucket ID to query
lcb_U64 uuid UUID known to client which should be queried.
struct lcb_RESPOBSEQNO

Response structure for lcb_observe_seqno3()

Note that key, nkey and cas are empty because the operand is the relevant mutation token fields in lcb_CMDOBSEQNO

Data Fields
void * cookie Application-defined pointer passed as the cookie parameter when scheduling the command.
const void * key Key for request.
lcb_SIZE nkey Size of key.
lcb_CAS cas CAS for response (if applicable)
lcb_error_t rc Status code.
lcb_U16 version ABI version for response.
lcb_U16 rflags Response specific flags.

see lcb_RESPFLAGS

lcb_U16 vbid vBucket ID (for potential mapping)
lcb_U16 server_index Input server index.
lcb_U64 cur_uuid UUID for this vBucket as known to the server.
lcb_U64 persisted_seqno Highest persisted sequence.
lcb_U64 mem_seqno Highest known sequence.
lcb_U64 old_uuid In the case where the command's uuid is not the most current, this contains the last known UUID.
lcb_U64 old_seqno If old_uuid is nonzero, contains the highest sequence number persisted in the old_uuid snapshot.

Enumeration Type Documentation

Type of durability polling to use.

Enumerator
LCB_DURABILITY_MODE_DEFAULT 

Use the preferred durability.

If LCB_CNTL_FETCH_MUTATION_TOKENS is enabled and the server version is 4.0 or greater then LCB_DURABILITY_MODE_SEQNO is used. Otherwise LCB_DURABILITY_MODE_CAS is used.

LCB_DURABILITY_MODE_CAS 

Explicitly request CAS-based durability.

This is done by checking the CAS of the item on each server with the item specified in the input. The durability operation is considered complete when all items' CAS values match. If the CAS value on the master node changes then the durability operation will fail with LCB_KEY_EEXISTS.

Note
CAS may change either because of a failover or because of another subsequent mutation. These scenarios are possible (though unlikely). The LCB_DURABILITY_MODE_SEQNO mode is not subject to these constraints.
LCB_DURABILITY_MODE_SEQNO 

Use sequence-number based polling.

This is done by checking the current "mutation sequence number" for the given mutation. To use this mode either an explicit lcb_MUTATION_TOKEN needs to be passed, or the LCB_CNTL_DURABILITY_MUTATION_TOKENS should be set, allowing the caching of the latest mutation token for each vBucket.

Possible statuses for keys in OBSERVE response.

Enumerator
LCB_OBSERVE_FOUND 

The item found in the memory, but not yet on the disk.

LCB_OBSERVE_PERSISTED 

The item hit the disk.

LCB_OBSERVE_NOT_FOUND 

The item missing on the disk and the memory.

LCB_OBSERVE_LOGICALLY_DELETED 

No knowledge of the key :)