Webhooks

    +

    Introducing Sync Gateway events and event handling with Webhooks
    You can configure webhooks to detect document_changed events and post the changed documents to specified URLs.

    Related integration topics: Changes Feed | External Stores | Prometheus Integration

    Caveats

    Webhooks post your application’s data, which might include user data, to URLs. Consider the security implications.

    Introduction

    Sync Gateway provides the ability to extend the replication process and build responsive services that react to changes in documents, adding value to the end to end process.

    For example, by initiating the sending of notifications, or specialist audit processes, when certain document changes are detected.

    This can be done using either the changes feed or document_changed events — see Table 1 for a comparison of these scenarios.

    Sync Gateway’s webhook event handlers perform both document filtering, and HTTP POST operations, asynchronously.

    In addition to providing the opportunity to integrate with external systems, this minimizes:

    • The performance impact on Sync Gateway’s regular processing

    • The amount of Sync Gateway node CPU resources consumed by slow response times from the HTTP POST operations.

    Behavior

    Webhooks work on the push-cycle of a replication.

    If a webhook event handler is defined:

    • Sync Gateway adds a document_changed event to the event queue [1] whenever it updates a document in a Couchbase Server bucket. These changes [2] can arise from Sync Gateway’s Public REST API and-or Couchbase Lite push replications.

    • Whenever the event queue contains a data_changed event, Sync Gateway spawns a process to:

      • FILTER — decide which changed documents to post.
        The event process executes the webhook’s event handler (the filter) on the associated document. The filter determines which documents need to be POSTed.

        If there is no filter all document changes are passed to POST.

      • POST — send selected changed documents to the URL endpoint.
        HTTP/HTTPS is used to POST the document changes selected by the filter to the defined url.

    Table 1. Changes Feed vs Webhooks
    Scenario Changes feed (pull) Webhooks (push)

    Sequence/Ordered

    Yes

    No

    User Access Control

    Fine Grain

    Limited

    Scalable

    Yes

    No

    Data Stream replay on Failure

    Yes

    No

    Definition

    Define Webhooks in Sync Gateway’s Configuration Properties using the database level this_db.event_handlers properties.

    You can define multiple webhook event handlers. For example, you could define webhooks with different filtering criteria, which post changed documents to different URLs — see: Example 1.

    Each event handler definition comprises the following properties:

    Document Change Properties

    A Filter

    The filter is a JavaScript function used to determine which documents to post. It accepts the document body as input and returns a boolean value.

    • If the filter function returns true, then Sync Gateway posts the document.

    • If the filter function returns false, then Sync Gateway does not post the document.

    • If no filter function is defined, then Sync Gateway posts all changed documents.

    Filtering only determines which documents to post. It does not extract specific content from documents and post only that.

    An event handler type

    Property name: this_db.event_handlers.document_changed.handler

    Sets the event handler’s type; currently, this must be webhook.

    A timeout value

    Sets the time (in seconds) to wait for a response to the POST operation. It ensures that slow-running POST operations don’t cause the webhook event queue to back up. When the timeout limit is reached, Sync Gateway stops listening for a response and discards the operation.

    You should not need to adjust the default setting to tune performance.

    URL

    Sets the address to which documents are posted.

    Example 1. Sample Webhook Definitions
    • Simple Webhook

    • Multiple Webhooks

    In this simple example of a webhook event handler we define a single instance with no filter. It simply listens for the document_changed event and immediately sends the changed document to the URL http://someurl.com.

    "event_handlers": {
        "document_changed": [
            {
                "handler": "webhook",
                "url": "http://someurl.com"
            }
        ]
    }

    In this example we define two webhook event handlers, both of which use filters to decide how to process the changed document.

    The filter function in the first handler recognizes documents with doc.type equal to A and posts the documents to the URL http://someurl.com/type_A.

    The filter function in the second handler recognizes documents with doc.type equal to B and posts the documents to the URL http://someurl.com/type_B.

    "event_handlers": {
          "document_changed": [
            {"handler": "webhook",
             "url": "http://someurl.com/type_A",
             "filter": `function(doc) {
                  if (doc.type == "A") {
                    return true;
                  }
                  return false;
                }`
             },
            {"handler": "webhook",
             "url": "http://someurl.com/type_B",
             "filter": `function(doc) {
                  if (doc.type == "B") {
                    return true;
                  }
                  return false;
                }`
            }
         ]
      }

    Event Processing Properties

    Limited Concurrent Processes

    Sets the maximum number of events that can be processed concurrently. The default value should work well in the majority of cases. You should not need to adjust it to tune performance. However, if you wish to ensure that most webhook posts are sent, you can set it to sufficiently high value.

    Limited Full-Queue Wait Time

    Sets the maximum time (milliseconds) that event processing will wait for a free process, if an event is received whilst the event queue is full. You should not need to adjust it to tune performance.

    To avoid blocking standard Sync Gateway processing, set a zero value. Any events arriving whilst the queue is full are then immediately discarded — see also Logging.

    Logging

    Sync Gateway creates a log whenever an event is discarded, and not added to the event queue.

    You can configure the console logging of events using the configuration file and-or the ADMIN Rest API — see Use the Logging API. The log_key you need to include is Event; or Events+ for more verbose output.


    1. An asynchronous event-processing queue
    2. Creations, updates, and deletions