You are viewing the documentation for a prerelease version.

View Latest

Frequently Asked Questions

    This section provides answers to commonly asked questions pertaining to the Eventing Service and Functions.


    • Is Eventing an MDS-enabled service?

      Yes. MDS enables workload isolation in a Couchbase cluster. Eventing nodes can be added as more Functions are added, or if the mutation rates are high.

    • What kind of nodes do I run my Eventing Service on?

      Eventing leverages the latest trends in multi-core CPUs; therefore nodes that you select for the Eventing Service should optimally have a higher number of cores than those for indexing.

    • Can the Eventing Service be co-located with other services?

      Yes, the Eventing Service node can be co-located with other MDS-enabled services.

    • Is it supported for both text and binary formats of the documents?

      No, the value or the document must always be text. Eventing currently doesn’t process binary documents (binary documents will not generate mutations). Additionally, an Eventing handler can’t currently read a binary value from a bucket alias. The documents are based on a key-value format where the value can be arbitrary text. However, in practice the value is typically a well formatted JSON text payload. There is one exception, Eventing handlers via the curl() function can handle binary data to and from an external REST endpoint.

    Change Capture

    • Will all the updates to a document be seen in DCP?

      When a document is updated multiple times in a small time-window, not all updates are seen in DCP. This means that the event handlers see only those deduplicated events that are published in DCP.

    • Can a Function listen to all changes across Couchbase Server?

      A defined Function listens only to changes that are exposed using the DCP for the buckets (Couchbase and Ephemeral) in the data-nodes. Memcached buckets are not supported. The Function cannot listen to changes happening in other Couchbase components, such as FTS, Views, or GSI; nor can it listen to system events.

    • Can we determine what has changed on the document?

      No. But you can implement versioning as part of the application logic. If versioning is important, include it as part of the data architecture.

    • Can I get old and new values of the document inside the Function?

      No. We do not support versioning of documents; therefore, this feature is not available out of the box. However, you can have a parent bucket and its replica version. The replica bucket would retain the version of documents before the current set of changes.

    • Is the ordering of document mutations enforced?

      All changes from a document are always processed in order.


    • Are Functions similar to a Database Trigger?

      In a rough sense, Functions are similar to the Post-Triggers of the database world. But with Functions, the action is already completed at the data-layer, and the event handler gives an interface by which developers can key in the logic of what needs to happen ‘after’ the action is done. What a Function sees is the actual event of the change, and hence it does not directly correlate with Database Triggers.

    • Are Functions similar to a Rules Engine?

      Not exactly. A Rules Engine enforces ordering and other semantics that is not possible out of the box with Functions. The Function in its purest form offers a rule to implemented closer to the data, but cannot trigger another Function directly.

    • Are Functions similar to a Stored Procedure?

      Stored Procedures enforce a request-response model. A stored procedure will not be invoked automatically; it has to be invoked by a calling method. Functions are based on the idea of events. Changes to the data or mutations are the events that trigger Functions, and hence this is not a request-response model in its purest sense.

    • Do I need a separate middleware to consume the Functions? How do I consume changes in my middleware/application?

      No. Functions do not require any additional infrastructure mutations are consumed and acted upon by the defined Function. There is no other programmatic way of accessing the changes processed by a Function (such as by using an SDK, or some other form of middleware). REST endpoints are exposed, to perform administrative operations.

    • Can I import my own JS libraries or Node Modules?

      No. We do not allow import of node modules or external JS libraries or Node modules. Though you can define other JavaScript functions inside a Function (outside the scope of OnUpdate and OnDelete) that can be invoked any number of times from either of the event handlers.

    • Are Functions supported for both Data Node and Document storage?

      The Eventing Service listens to changes that appear in the Database Change Protocol (DCP) stream. Since DCP is valid for the Data Service, and Eventing Functions only operate on documents that are text based. The documents are based on a key-value format where the value can be arbitrary text. However in practice the value is typically a well formatted JSON text payload. Note, the key is passed as and the value is passed as doc to the OnUpdate handler and only the key as is passed to the OnDelete handler.

    • Can Functions interact with external processes?

      Yes. Functions support a cURL API as a way of interacting with external entities or REST endpoints using HTTP. This functionality allows propagation of data changes to other systems, notifying the application about interesting events, enriching documents with data from external resources, and so on. The native cURL that lets users propagate events of interest to other APIs when mutation rates are low.

    • Can a Function be debugged and what happens when a Function is debugged?

      Yes. Functions offer multiple diagnosability solutions (debugger, logs, and statistics), all designed to have minimal impact on overall performance and scalability. When debugging a function a single mutation is blocked and handed off to the debugger session, while the rest of the mutations continue to be serviced by the event handler, refer to the Debugging and Diagnosability section.

    • Can the logic in a Function in production be altered while it is running?

      Yes. Functions can be "paused, edited and resumed" without losing a single mutation; as such continuity is maintained. Note this sequence is similar to the operations of "undeploy, edited, and deployed with a feed boundary ‘From now’", however in this later case you can lose mutations.

    • How do I perform a Function’s lifecycle operations from CI/CD?

      To perform Functions lifecycle operations from CI/CD, refer to the CLI Eventing section.

    • How to invoke a REST endpoint from inside the Function?

      To invoke a REST Endpoint from inside the Function, refer to the Functions REST API section.

    • How does the Functions offering compare with the Couchbase’s Kafka Connector?

      The Functions offering is about server-side processing or compute of business logic; it does not require any additional infrastructure layer or middleware to be deployed or managed. Couchbase’s Kafka connector is an SDK component that needs an application container or middleware to run.

    Function Handler Code

    • What languages are supported?

      Only JavaScript (ECMAScript 6) is supported. However, to support the ability to shard and scale Function execution automatically, some capabilities have been removed (Global state, Asynchrony, etc.), refer to the Language Constructs: Removed Language Features section.

    • Why can’t I create global variables?

      Functions do not allow global variables, this restriction is mandatory for the Function logic to shard and scale and remain agnostic during rebalance. All state must be saved and retrieved from persistence providers, therefore KV bucket(s) made available to the Function through bindings can be used to store any required global state.

    • What is in the "meta" Function parameter (OnUpdate, OnDelete)? Is this the metadata we currently write in order to figure out what has changed in the document?

      No, the meta parameter does not include information on what fields changed or mutated in the document. This parameter is composed of the meta fields associated with the document. For more information, refer to the metadata section.

      It should be noted, “document metadata” is different from the “metadata bucket”, described in the next section, used by the Eventing Service to maintain state and checkpoints.

    • What is the metadata bucket? Do I need to create a separate bucket?

      To provide better resiliency and restartability semantics across Eventing nodes, some of the metadata that is used by the Eventing service needs to be stored in a standard Couchbase bucket (hereafter referred to as the 'metadata bucket').

      After provisioning the Eventing nodes in your cluster, you’ll need to create the metadata bucket before you can start using the Eventing service. All Eventing functions within a cluster can share the same metadata bucket (this is a best practice but not a requirement), regardless of the number of functions, or their source and destination buckets. (Setting up a metadata bucket is a one-time activity for the cluster should you choose to follow this best practice.)

      Some additional requirements of the metadata bucket are as follows:

      • You should enable vBucket replicas on the metadata bucket to allow for failure recovery.

      • You should reserve the metadata bucket solely for Eventing housekeeping. It shouldn’t be used for any other data storage.

      • Each Eventing function always requires a fixed amount of space of about 2MB (1024 docs * 1884 bytes).

      • If an Eventing function uses timers, then an additional fixed amount of space of about 0.2MB (1024 * docs of 196 bytes) is needed.

      • If an Eventing function uses timers, then for each active timer, an additional amount of space between 832 and 1856 bytes (832 bytes + sizeof(context)) is needed. Where by default the context cannot be larger than 1024 bytes and the maximum number of active timers is based on both the business logic and the mutation rate. Note, the "timer_context_size" can be overridden on a per function bases via the Eventing: REST API. It is best to keep the size of the context small by using a reference rather than passing and storing a massive document in the timer.

    • Can there be more than one Function listening to changes to a bucket?

      Yes. More than one Function can be defined for the same source bucket. This lets you process the change according to the business logic that you enforce. But there is no enforced ordering; for example, if bucket 'wine' has three different Functions, which are FunctionA, FunctionB, and FunctionC, you cannot enforce the order in which these Functions are executed.

      However, for each Function you start a set of DCP streams so for a busy system you will get better performance by coalescing multiple Eventing Functions that have the same source bucket into a single Function. This merging is easy to do with a JavaScript switch statement or a simple if-then-else block.

    • Is it possible to get additional state during a Function execution?

      Yes. For example, you can fetch related data from another document (using a document id) from any other bucket that is exposed to the Function via a binding. It is also possible to utilize the cURL API to read additional state from an external REST endpoint.

    • Is it possible to update state (or change a document) during a Function execution?

      Yes. For example, you can your enrich or update a document with data from another document (using a document id) from any other bucket that is exposed to the Function via a binding with access level of "Read Write" inclusive of the source bucket.

    Timer Behavior

    • Timer Delays: When I schedule a timer to fire at an exact time, I see some delay. Why?

      The timer implementation is designed to handle large numbers of distributed timers (i.e., millions of timers) and the only promise is to run timers as soon as possible, e.g. no timers lost.

      In a steady state you may see a 3-4 second delay from the scheduled time, however if scheduling timers close to the system wall-clock this delay may increase to about 14 seconds. For more details on Timer scheduling refer to Timers: Wall-clock Accuracy section.

    • Can I cancel a Timer?

      Yes. As of the release 6.6.0 Eventing Timers can be cancelled using cancelTimer() function, or by creating a new Timer with the same reference as an existing Timer refer to Timers: Limitations.

    • Can I create a Recurring timer?

      Yes. As of the release 6.6.0 Recurring Timers are fully supported, i.e. a function that is invoked by a Timer callback can reliably create fresh Timers refer to Timers: Limitations.

    • Can I schedule a Timer far into the future?

      Yes. As of the release 6.6.0, recurring Timers can be created days, weeks, or years in the future with no adverse performance impact on an otherwise idle Eventing system. Refer to Timers: Limitations.

    • Why do I see a burst of activity in metadata bucket OPs after a timer is paused for an extended period of time?

      Resuming a timer handler after a prolonged period of time where the Function was in the paused state (like days) will cause a period of high metadata bucket OPs upon resuming the Function.  In addition mutation processing is blocked until the timer scan is completed which can take some time (this delay is proportional to the duration of pause).

    • Why do I see unexpected documents in the metadata bucket when I cancel or overwrite an Eventing Timer?

      When overwriting or canceling a Timer only one of possible three documents, i.e. the "context", is immediately cleared from the metadata bucket. The extra documents, an "alarm" document associated with each Timer and a "root" document (1 per vBucket for the specific time) are left in the metadata bucket. These items are cleaned up at the time that the original execution time the the Timer was scheduled to fire.

    Cluster Behavior

    • What happens to the Eventing Service during a failover condition?

      When the Data service experiences a failover condition, mutations may be lost and these lost mutations are not processed by the Eventing service. When the Eventing node experiences a failover condition, few mutations may be processed more than once.

    • Does a rebalance have any effect on the firing of events?

      No. Functions do not lose any mutations during a rebalance operation.

    • I have Functions deployed on my cluster, when can I perform an Eventing rebalance operation?

      The Function lifecycle operations (deploying, undeploying, pausing, resuming, and deleting) and the Eventing rebalance operation are mutually exclusive. The Eventing rebalance operation fails when a Function lifecycle operation is currently in progress. Likewise, when the Eventing rebalance operation is in progress, you cannot perform a Function lifecycle operation.

    • How do I increase performance of an Eventing Function?

      You can scale up vertically by adding additional workers (in the handler’s settings) to increase performance for a specific Function. You can also scale out horizontally via Couchbase’s elastic scaling option by adding another node and rebalancing. In this case each eventing node is assigned a subset of vBuckets. Note this sharding increases overall performance for all Functions.

      However keep in mind that sometimes the Function is limited by the overall performance of the Data Service. In this case it is appropriate to scale the Data service.