Eventing Service: Fundamentals

Enterprise Edition

The Couchbase Eventing Service is a framework to operate on changes to data in real time. Events are changes to data in the Couchbase cluster.

The term 'mutation' generally refers to document changes happening in the Couchbase cluster. The Couchbase cluster considers create, update, expiry, or delete operations as a data mutation.

Natively integrated with the Couchbase Data Platform, the Eventing Service does not require any third party solutions to manage and respond to data mutations. In an event-based architecture, all data changes are reactive and in real-time.

For example, consider the number of mutations in the data cluster of an e-Commerce application when a customer places a new order. You can observe that: a new transaction record gets created, a new purchase-transaction gets appended to the User’s account, seller’s stock-inventory gets updated with a new purchase order. Each of these data mutations qualifies as an event in the cluster.

A few sample events are listed below:

Capturing the sensor output from a patient.
Placing an order in an e-Commerce application.
A customer’s order causing a drop in the inventory.
Change in the status of a project.
Removing a user from a transaction scoring application.
News article expires after a certain period of time.
Notify a customer that their order has shipped.

Using the Eventing Service, you can:

Set alerts in a document when a preconfigured threshold is breached
Monitor specific parameters in a document
Propagate changes to other systems
Enrich a document in real time
Cascade deletes to avoid orphaned documents

Why use the Eventing Service?

The Eventing Service helps you to streamline your business workflows. Using the Couchbase Eventing Service you can:

Manage a data-driven business logic across business-critical applications in a timely manner, thereby increasing your customer engagement.
Handle inconsistencies in business logic while working with multiple client applications across domains.
Design new data products by leveraging lower technical barriers in the Eventing Service.
Develop a reliable infrastructure that can execute your business logic in a rapidly changing data platform.
Scale your throughput without making changes to your data configuration and infrastructure.
Create specialized point tools to clean, enhance, or transform your data.
Maximize your return on investment by minimizing your TCO.

Eventing Service versus Message Queue and Polling Implementations

The following table compares the implementation of the Eventing Service with the message queue method that is used to solve the tracking of data mutations.

Message Queue or Bus Implementation	Eventing Service Implementation
Needs an additional layer to propagate data changes.	Inherent to Couchbase Server and does not need a new layer to propagate data changes.
Encounters dual-write problems. Every write operation gets pushed twice; once to the message queue and the second time to the cluster.	Eliminates the dual-write problem. Multiple application servers can perform simultaneous write operations.
At any point, a write-failure condition can happen.	Eliminates the write-failure condition; no transaction semantics.
No easy debug option during troubleshooting.	Integrated with native debugger support.
Leads to inefficient data governance and data leakages.	Provides a centralized control for aspects such as data auditing and data governance, thereby reduces data leakages.
With the added license, infrastructure, and deployment expenses, the Total Cost of Ownership (TCO) increases many fold.	No additional expenses, the TCO is reduced.

Message Queue or Bus Implementation

Eventing Service Implementation

Needs an additional layer to propagate data changes.

Inherent to Couchbase Server and does not need a new layer to propagate data changes.

Encounters dual-write problems. Every write operation gets pushed twice; once to the message queue and the second time to the cluster.

Eliminates the dual-write problem. Multiple application servers can perform simultaneous write operations.

At any point, a write-failure condition can happen.

Eliminates the write-failure condition; no transaction semantics.

No easy debug option during troubleshooting.

Integrated with native debugger support.

Leads to inefficient data governance and data leakages.

Provides a centralized control for aspects such as data auditing and data governance, thereby reduces data leakages.

With the added license, infrastructure, and deployment expenses, the Total Cost of Ownership (TCO) increases many fold.

No additional expenses, the TCO is reduced.

The following table compares the implementation of the Eventing Service with the polling method that is used to solve the tracking of data mutations.

Polling Implementation	Eventing Service Implementation
To record and propagate data changes, multiple applications are needed.	Can record and propagate data changes to a database, message queue, an end-point, or to another bucket inside the Couchbase cluster.
Batch systems are highly inefficient and are not reactive.	Handles data mutations in real-time.
Consumes a lot of CPU resources.	Implements as a state-less compute operation and utilizes latest trends in compute (multi-core CPU).
Leads to code duplications across multiple infrastructure applications.	No code duplication.
Difficult to scale. You need to scale for applications and also scale for transport layer requirements.	Built in support to provide easy horizontal or vertical scaling options.

Polling Implementation

Eventing Service Implementation

To record and propagate data changes, multiple applications are needed.

Can record and propagate data changes to a database, message queue, an end-point, or to another bucket inside the Couchbase cluster.

Batch systems are highly inefficient and are not reactive.

Handles data mutations in real-time.

Consumes a lot of CPU resources.

Implements as a state-less compute operation and utilizes latest trends in compute (multi-core CPU).

Leads to code duplications across multiple infrastructure applications.

No code duplication.

Difficult to scale. You need to scale for applications and also scale for transport layer requirements.

Built in support to provide easy horizontal or vertical scaling options.

Couchbase Eventing Functions

Couchbase Eventing Functions offer a computing paradigm that developers can use to handle data changes. In the Couchbase cluster, you can use the Functions to process and respond to data-changes according to an Event-Condition-Action model.

For application developers, Functions offer a platform in which you can focus on building business logic, rather than configuration or infrastructure. Functions handle and process events that are generated when documents are created, updated, expired, or deleted. The serverless computing infrastructure gets closer to the datastore as the Couchbase Functions integrates with the Couchbase Data Platform.

Features of Functions

Since Couchbase is a NoSQL document-oriented data platform that utilizes the JSON data model, thus it is only natural that the Eventing Service exposes the ability to write JavaScript code to analyze and manipulate your JSON documents on any type mutations.

You can configure Functions to generate threshold-based alerts. If your business logic requires you to monitor specific parameters inside a document, then you can use Functions to trigger an alert upon logic such as a threshold breach.

Additionally as part of Functions you can

Data Service integration to Read, Write, and Delete documents from within a Function.
Data Service integration to work with Atomic Counters, CAS and TTLs from within a Function.
Query Service integration to utilize inline SQL++ (or N1QL) queries or statements from within a Function.
Enable a Timer from within a Function, essentially a callback in the future, for the Function to perform more work.
Integrate with external REST endpoints via cURL functionality from within a Function.

Apart from notifications and alerts, Functions provide an option to propagate data changes via mutations such as data enrichment. You can also propagate data changes by altering other documents or moving documents between buckets inside your Couchbase cluster. For example, Functions can easily be used to perform cascading deletes.

Benefits of Functions

Improves customer experience and engagement
- Data enrichment: Before the introduction of the Eventing Service, data enrichment was accomplished through batch jobs. These batch jobs were not in real-time and often resulted in increasing the cost of infrastructure and management. Using the Eventing Service, the data enrichment capability was achievable in real-time. Functions involve moderate coding effort, time to market and restart capabilities can be achieved easily.
- Simple to use: Since Functions are developed within the Eventing Service framework, tracking data changes in your cluster is manageable.
Faster innovation
- With a focus on business logic, development cycles are reduced. The Eventing Service platform offers a developer-friendly environment, which in turn aids the faster creation of Minimum-Viable-Products (MVPs).
- Using Functions, Application Developers can rapidly remodel their business workflows and thereby stay in-sync with any business change conditions.
- Functions offer a lower barrier to technology-adoption by emphasizing on business operations.
Reducing infrastructure and operations-cost
- Since the implementation of the Eventing Service is intrinsic to the Couchbase cluster, it offers a simple to deploy working model.
- The Eventing Service provides optimum utilization of resources and controls essential aspects such as data auditing, data governance, and node scaling.

Use Cases

As an Organization, you can use the Eventing Service in a wide variety of use cases. Be it in domains such as retail, healthcare, telecommunications, finance, marketing, media, or travel; you can leverage the Eventing Service to track data mutations.

For an easy understanding, consider a sample use case in the banking and financial domain. Let us say the user performs a credit card transaction. Using the Event-Condition-Action model, you can design a custom workflow based on factors such as user’s credit limit, usage currency, and risk propositions.

As another sample use case, consider an organization operating in the Supply Chain Management domain. As a developer, using the Function’s Event-Condition-Action model, you can design a custom workflow in your inventory for stock replenishment. Functions help you to construct a business workflow that automatically triggers new stock replacements and maintains a set stock threshold.

The rows in the below table list some popular scenarios where the Eventing Service across domains can be used.

Table 1. Eventing Service - Use Cases
Domain	Eventing Trigger	Condition Check	Sample Workflow
Banking & Financial Services	Card transaction	Transaction threshold	Generate risk alerts and quarantine user upon threshold breach.
Inventory/ Warehousing	New sales voucher	Stock availability	Generate invoice for stock replenishment.
Inventory/ Warehousing	New purchase order	Saved wishlist/cart	Notify price alerts for wishlist items.
Airline	New booking	Booking history	Enroll for frequent flyer program and notify special promotions.
Airline	Enquiry	User profile	Notify price drop alerts.
Healthcare	New report	Check for vitals	Schedule an appointment.
Sports/ Gaming	New user creation	User profile	Generate notification about leaderboard and other statistics.
Media/ Entertainment	Breaking news	Query archives	Enrich existing news with archival information.

Eventing Service - Onboarding Information

In your organization, if you are using the Couchbase data platform, then the Eventing Service is a good fit for managing data mutations. The use of MDS services in Couchbase enables workload isolation and independent scalability of any service in a Couchbase cluster. Like Data, Query, GSI, FTS, and Analytics, the Eventing Service supports Multi-Dimensional Scaling (MDS). Adding a new Eventing node is a simple process allowing your business logic to scale in addition the Eventing Service integrates seamlessly with other Couchbase services such as the Data, Query, GSI, FTS, and Analytics nodes.

Figure 1. Eventing Service Onboarding Information

Following are a few aspects of the onboarding process:

The Eventing Service is intrinsic to Couchbase Server; unlike Polling and Message Queue based external systems, it eliminates the need for an additional layer without involving multiple applications for tracking data mutations. All data mutations are handled in real-time, and the Eventing Service offers a centralized control for data governance.
When you transition to leverage the Eventing Service, application developers can use Couchbase Functions to manage business workflow changes swiftly. Application developers can program, test, debug and troubleshoot on a single Eventing Service platform, instead of managing multiple applications across different network layers.
After onboarding, you can manage and optimize the system throughput efficiently. If your data resides in the Couchbase cluster, based on aspects such as data workload, data mutation rate, and Function execution latency, you can either scale up vertically by adding additional workers or scale out horizontally via Couchbase’s elastic scaling option by adding another node.
The Eventing Service provides an export and import option for code portability. Using this option, you can reuse the Eventing Function code to validate the execution logic in different environments with workload variations.
The Eventing Service is highly performant during the recursive restartability operations. You can undeploy a Function, pause for few cycles and then start the Eventing Function code. Deploying a Function after a time-lapse ensures that the Function execution is tested for restartability.
Eventing is compute oriented and leverages the latest trends in multi-core CPUs; therefore nodes selected for Eventing should optimally have a higher number of cores than nodes used for indexing.