Frequently Asked Questions
This section provides answers to commonly asked questions pertaining to the Eventing Service and Functions.
Is Eventing an MDS-enabled service?
Yes. MDS enables workload isolation in a Couchbase cluster. Eventing nodes can be added as more Functions are added, or if the mutation rates are high.
What kind of nodes do I run my Eventing Service on?
Eventing leverages the latest trends in multi-core CPUs; therefore nodes that you select for the Eventing Service should optimally have a higher number of cores than those for indexing.
Can the Eventing Service be co-located with other services?
Yes, the Eventing Service node can be co-located with other MDS-enabled services.
Is it supported for both text and binary formats of the documents?
Yes only for versions 6.6.2+
For versions 6.6.2 and later, the value or the document can be either text or binary. Starting with version 6.6.2, Eventing processes binary documents. Eventing Functions can read a binary value from a bucket alias or via an Advanced Accessor. The documents are based on a key-value format where the value can be arbitrary text or binary data. However, in practice the value is typically a well formatted JSON text payload. Note, Eventing Functions via the curl() function can also handle binary data to and from an external REST endpoint. Furthermore, every mutation OnUpdate(doc,meta) will now have a property "datatype" set to either "binary" or "json", however this information is not available via OnDelete(meta,options) or Basic Bucket Ops.
No for versions 6.6.1 and earlier.
For versions 6.6.1 and earlier, the value or the document must always be text. Eventing currently doesn’t process binary documents (binary documents will not generate mutations). Additionally, an Eventing Function can’t currently read a binary value from a bucket alias. The documents are based on a key-value format where the value can be arbitrary text. However, in practice the value is typically a well formatted JSON text payload. There is one exception, Eventing Functions via the curl() function can handle binary data to and from an external REST endpoint.
Will all the updates to a document be seen in DCP?
When a document is updated multiple times in a small time-window, not all updates are seen in DCP. This means that the event entry points of OnUpdate and OnDelete to the Eventing Function will see only those deduplicated events that are published in DCP.
Can a Function listen to all changes across Couchbase Server?
A defined Function listens only to changes that are exposed using the DCP for the source collection (Couchbase and Ephemeral) in the data-nodes. Memcached buckets (and thus underlying collections) are not supported. The Function cannot listen to changes happening in other Couchbase components, such as FTS, Views, or GSI; nor can it listen to system events.
Can we determine what has changed on the document?
No. But you can implement versioning as part of the application logic. If versioning is important, include it as part of the data architecture.
Can I get old and new values of the document inside the Function?
No. We do not support versioning of documents; therefore, this feature is not available out of the box. However, you can have a parent bucket and its replica version. The replica bucket would retain the version of documents before the current set of changes. Look at the Eventing examples and scriptlets for various work arounds.
Is the ordering of document mutations enforced?
All changes from a document are always processed in order.
Are Functions similar to a Database Trigger?
In a rough sense, Eventing Functions are similar to the Post-Triggers of the database world. But with Functions, the action is already completed at the data-layer, and the entry points of OnUpdate and OnDelete give an interface by which developers can key in the logic of what needs to happen ‘after’ the action is done. What an Eventing Function sees is the actual event of the change, and hence it does not directly correlate with Database Triggers.
Are Functions similar to a Rules Engine?
Not exactly. A Rules Engine enforces ordering and other semantics that is not possible out of the box with Functions. The Function in its purest form offers a business rule to implemented closer to the data, but cannot trigger another Function directly.
Are Functions similar to a Stored Procedure?
Stored Procedures enforce a request-response model. A stored procedure will not be invoked automatically; it has to be invoked by a calling method. Functions are based on the idea of events. Changes to the data or mutations are the events that trigger Functions, and hence this is not a request-response model in its purest sense.
Do I need a separate middleware to consume the Functions? How do I consume changes in my middleware/application?
No. Functions do not require any additional infrastructure mutations are consumed and acted upon by the defined Function. There is no other programmatic way of accessing the changes processed by a Function (such as by using an SDK, or some other form of middleware). REST endpoints are exposed, to perform administrative operations.
Can I import my own JS libraries or Node Modules?
Are Functions supported for both Data Node and Document storage?
The Eventing Service listens to changes that appear in the Database Change Protocol (DCP) stream. Since DCP is valid for the Data Service, and Eventing Functions only operate on documents that are text based. The documents are based on a key-value format where the value can be arbitrary text. However in practice the value is typically a well formatted JSON text payload. Note, the key is passed as meta.id and the value is passed as doc to the OnUpdate handler and only the key as meta.id is passed to the OnDelete handler.
Can Functions interact with external processes?
Yes. Functions support a cURL API as a way of interacting with external entities or REST endpoints using HTTP. This functionality allows propagation of data changes to other systems, notifying the application about interesting events, enriching documents with data from external resources, and so on. The native cURL that lets users propagate events of interest to other APIs when mutation rates are low.
Can a Function be debugged and what happens when a Function is debugged?
Yes. Functions offer multiple diagnosability solutions (debugger, logs, and statistics), all designed to have minimal impact on overall performance and scalability. When debugging a function a single mutation is blocked and handed off to the debugger session, while the rest of the mutations continue to be serviced by the Eventing Function, refer to the Debugging and Diagnosability section.
Can the logic in a Function in production be altered while it is running?
Yes. Functions can be "paused, edited and resumed" without losing a single mutation; as such continuity is maintained. Note this sequence is similar to the operations of "undeploy, edited, and deployed with a feed boundary ‘From now’", however in this later case you can lose mutations.
How do I perform a Function’s lifecycle operations from CI/CD?
To perform Functions lifecycle operations from CI/CD, refer to the CLI Eventing section.
How to invoke a REST endpoint from inside the Function?
To invoke a REST Endpoint from inside the Function, refer to the Functions REST API section.
How does the Functions offering compare with the Couchbase’s Kafka Connector?
The Functions offering is about server-side processing or compute of business logic; it does not require any additional infrastructure layer or middleware to be deployed or managed. Couchbase’s Kafka connector is an SDK component that needs an application container or middleware to run.
Do I have to update and/or deploy my Functions on each Eventing node?
No. The Eventing service will properly distribute the Function code or lifecycle requests across all Eventing nodes. It is a best practice to only have one (1) Admin UI to a single node in your cluster when developing or modifying your Eventing Functions. Note that if you edit Eventing Functions (code or settings) in two browser windows or tabs (to same node or different nodes), you might inadvertently deploy a slightly older or “stale” definition if you switch back and forth between different UI sessions.
Eventing Function Code
What languages are supported?
Why can’t I create global variables?
Functions do not allow global variables, this restriction is mandatory for the Function logic to shard and scale and remain agnostic during rebalance. All state must be saved and retrieved from persistence providers, therefore KV bucket(s) made available to the Function through bindings can be used to store any required global state. Eventing Functions (as of 7.0.0) do however support global constants via the "Constant alias" binding.
What is in the "meta" Function parameter (OnUpdate, OnDelete)? Is this the metadata we currently write in order to figure out what has changed in the document?
No, the meta parameter does not include information on what fields changed or mutated in the document. This parameter is composed of the meta fields associated with the document. For more information, refer to the metadata section.
It should be noted, “document metadata” is different from the "Eventing Storage" keyspace (metadata collection), described in the next section, used by the Eventing Service to maintain state and checkpoints.
Can there be more than one Function listening to changes to a collection?
Yes. More than one Function can be defined for the same source collection. This lets you process the change according to the business logic that you enforce. But there is no enforced ordering; for example, if collection 'wine' has three different Functions, which are FunctionA, FunctionB, and FunctionC, you cannot enforce the order in which these Functions are executed.
Is it possible to get additional state during a Function execution?
Yes. For example, you can fetch related data from another document (using a document id) from any other collection that is exposed to the Function via a "bucket binding". It is also possible to utilize the cURL API to read additional state from an external REST endpoint.
Is it possible to update state (or change a document) during a Function execution?
Yes. For example, you can your enrich or update a document with data from another document (using a document id) from any other collection that is exposed to the Function via a binding with access level of "Read Write" inclusive of the source collection.
The Eventing Storage keyspace (or metadata collection)
What is the Eventing Storage keyspace? Do I need to create a separate collection?
To provide better resiliency and restartability semantics across Eventing nodes, some of the metadata that is used by the Eventing service needs a collection to be stored in a standard Couchbase bucket (hereafter referred to as the 'metadata collection').
After provisioning the Eventing nodes in your cluster, you’ll need to create the metadata collection before you can start using the Eventing service. All Eventing functions within a cluster can share the same metadata collection (this is a best practice but not a requirement), regardless of the number of functions, or their source and destination collection. (Setting up a metadata collection is a one-time activity for the cluster should you choose to follow this best practice.)
Some additional requirements of the metadata collection are as follows:
You should enable vBucket replicas on the metadata collection to allow for failure recovery.
You should reserve the metadata collection solely for Eventing housekeeping. It shouldn’t be used for any other data storage.
Each Eventing function always requires a fixed amount of space of about 2MB (1024 docs * 1884 bytes).
If an Eventing function uses timers, then an additional fixed amount of space of about 0.2MB (1024 * docs of 196 bytes) is needed. From version 6.6.1 on only 0.04MB (256 docs * 196 bytes) is needed if the function uses timers.
If an Eventing function uses timers, then for each active timer, an additional amount of space between 832 and 1856 bytes (832 bytes + sizeof(context)) is needed. Where by default the context cannot be larger than 1024 bytes and the maximum number of active timers is based on both the business logic and the mutation rate. Note, the "timer_context_size" can be overridden on a per function bases via the Eventing: REST API. It is best to keep the size of the context small by using a reference rather than passing and storing a massive document in the timer.
Every timer requires up to three documents (root, alarm, and context) which are stored in the Eventing Storage (or metadata collection). Note sometimes only two (2) additional documents are needed if the timer shares the same scan interval or root document with a previous timer.
Why is the metadata collection not getting cleared when I cancel a timer or a set of timers.
When a timer is canceled the context document is removed immediately, however the root and alarm documents are removed in a lazy fashion when the canceled timer was originally scheduled to fire. Thus is 100K timers are scheduled to fire one (1) year in the future and canceled up to 2 additional documents will persist for one (1) year. Note the cancelTimer() function was introduced in version 6.6.0.
Timer Delays: When I schedule a timer to fire at an exact time, I see some delay. Why?
The timer implementation is designed to handle large numbers of distributed timers (i.e., millions of timers) and the only promise is to run timers as soon as possible, e.g. no timers lost.
In a steady state you may see a 3-4 second delay from the scheduled time, however if scheduling timers close to the system wall-clock this delay may increase to about 14 seconds. For more details on Timer scheduling refer to Timers: Wall-clock Accuracy section.
Can I cancel a Timer?
Yes. As of the release 6.6.0 Eventing Timers can be cancelled using cancelTimer() function, or by creating a new Timer with the same reference as an existing Timer refer to Timers: Limitations.
Can I create a Recurring timer?
Yes. As of the release 6.6.0 Recurring Timers are fully supported, i.e. a function that is invoked by a Timer callback can reliably create fresh Timers refer to Timers: Limitations.
Can I schedule a Timer far into the future?
Yes. As of the release 6.6.0, recurring Timers can be created days, weeks, or years in the future with no adverse performance impact on an otherwise idle Eventing system. Refer to Timers: Limitations.
Why do I see a burst of activity in bucket OPs (in the bucket that holds the metadata collection) after a timer is paused for an extended period of time?
Resuming an Eventing Function with a timer callback or handler after a prolonged period of time where the Function was in the paused state (like days) will cause a period of high bucket OPs upon resuming the Function. In addition mutation processing is blocked until the timer scan is completed which can take some time (this delay is proportional to the duration of pause).
Why do I see unexpected documents in the metadata collection when I cancel or overwrite an Eventing Timer?
When overwriting or canceling a Timer only one of possible three documents, i.e. the "context", is immediately cleared from the metadata bucket. The extra documents, an "alarm" document associated with each Timer and a "root" document (1 per vBucket for the specific time) are left in the metadata bucket. These items are cleaned up at the original execution time that the Timer was scheduled to fire.
Can I pass a binding (Bucket or URL alias) in a Timer’s context?
No. Bindings, whether a Bucket alias or an URL alias, are not serializable objects and only exist in the scope of the executing V8 worker. When a Timer fires it can execute on a different thread. Of course the Timer’s callback can reference the binding directly. However you can pass a "Constant alias" to a timer callback.
What happens to the Eventing Service during a failover condition?
When the Data service experiences a failover condition, mutations may be lost and these lost mutations are not processed by the Eventing service. When the Eventing node experiences a failover condition, few mutations may be processed more than once.
Does a rebalance have any effect on the firing of events?
No. Functions do not lose any mutations during a rebalance operation.
I have Functions deployed on my cluster, when can I perform an Eventing rebalance operation?
The Function lifecycle operations (deploying, undeploying, pausing, resuming, and deleting) and the Eventing rebalance operation are mutually exclusive. The Eventing rebalance operation fails when a Function lifecycle operation is currently in progress. Likewise, when the Eventing rebalance operation is in progress, you cannot perform a Function lifecycle operation.
Due to a regression, MB-43343, impacting only 6.6.1 during a rebalance in of an Eventing node a race can occur resulting in Eventing functions becoming hung in deploying state. Users can run into this issue when they have multiple functions deployed against the same source collection and they try to rebalance-in an eventing node. The workaround is to ensure that you pause all Eventing Functions before any rebalance.
How do I increase performance of an Eventing Function?
You can scale up vertically by adding additional workers (in the Eventing Function’s settings) to increase performance for a specific Function. You can also scale out horizontally via Couchbase’s elastic scaling option by adding another node and rebalancing. In this case each eventing node is assigned a subset of vBuckets. Note this sharding increases overall performance for all Functions.
In 7.0.0 the default number of workers for new Eventing Functions is now one (1), previously it was one (3). Thus pay attention to your performance and adjust the number of works to fit your application. Note, all upgrades of existing Functions will keep the number of workers they were created with.
However keep in mind that sometimes the Function is limited by the overall performance of the Data Service. In this case it is appropriate to scale the Data service.
When I maximize the workers Eventing Function I sometimes see a stall in processing?
When scaling up vertically by adding additional workers (in the Eventing Function’s settings) typically above 48 workers (the issues is workload dependent and occurs typically on source collection updates) you may see a stall in Eventing Function’s progress. This is typically related to resources given to the Eventing service and can be solved by adding additional Memory Quota to Eventing in the Cluster Settings. By default Eventing allocates 256 MB, raising this value to 512 MB will typically solve this resource issue (this is one of the rare instances that you may need to raise the Memory Quota for Eventing).
However keep in mind that sometimes the Function is limited by both the number of cores in the Eventing instance the overall performance of the Data Service. In these cases it is appropriate to either scale compute power of the Eventing node, scale the Eventing service, or scale the Data service.
Does Eventing support node-to-node encryption ?
Yes, node-to-node encryption is available in the Couchbase Server for the Eventing Service in the 7.X train starting with version 7.0.2 and the 6.X train starting with version 6.6.5. Therefore, on earlier versions (in either train), when all is specified, the data passed to and from all other Couchbase Services is passed in encrypted form, whereas the data passed to and from the Eventing Service continues to be passed in unencrypted form.
When node-to-node encryption is enabled or disabled all deployed Eventing Functions need to be temporarily paused and can be resumed after the necessary changes have been made (to prevents the potential loss of mutations during the encryption change.)