Webhooks and Changes Feed

This guide describes two approaches for integrating Sync Gateway with other servers. These approaches can be used to build services that react to changes in documents. Examples of use cases include:

  • Sending notifications when specific documents change, for example, by email or SMS

  • Customized auditing or logging

The integration approaches are:

Changes Feed

The changes feed returns a sorted list of changes made to documents in the database.

Webhooks

Sync Gateway can detect document updates and post the updated documents to one or more external URLs.

Here’s a table that compares each API in different scenarios:

Scenario Changes feed (pull) Webhooks (push)

Sequence/Ordered

Yes

No

User Access Control

Fine Grain

Limited

Scalable

Yes

No

Data Stream replay on Failure

Yes

No

Changes Feed

This article describes how to use the changes feed API to integrate Sync Gateway with other backend processes. For instance if you have a channel called "needs-email" you could have a bot that sends an email and then saves the document back with a flag to keep it out of the "needs-email" channel.

The changes feed API is a REST API endpoint (/{db}/_changes) that returns a sorted list of changes made to documents in the database. It permits applications to implement business logic that reacts to changes in documents. There are several methods of connecting to the changes feed (also know as the feed type). The first 3 methods (polling, longpoll and continuous) are based on the CouchDB API. The last method (websocket) is specific to Sync Gateway.

polling (default)

Returns the list of changes immediately. A new request must be sent to get the next set of changes.

longpolling

In addition to regular polling, if the request is sent with a special last_seq parameter, it will stay open until a new change occurs and is posted.

continuous

The continuous changes API allows you to receive change notifications as they come, in a single HTTP connection. You make a request to the continuous changes API and both you and Sync Gateway will hold the connection open “forever.”

WebSockets

The WebSocket mode is conceptually the same as continuous mode but it should avoid issues with proxy servers and gateways that cause continuous mode to fail in many real-world mobile use cases.

WebSockets

The primary problem with the continuous mode is buggy HTTP chunked-mode body parsing that buffers up the entire response before sending any of it on; since the continuous feed response never ends, nothing gets through to the client. This can often be a problem with proxy servers but can be avoided by using the WebSocket method.

The client requests WebSockets by setting the _changes URL’s feed query parameter to websocket, and opening a WebSocket connection to that URL:

GET /db/_changes?feed=websocket HTTP/1.1
Connection: Upgrade
Upgrade: websocket
...

Specifying Options

After the connection opens, the client MUST send a single textual message to the server, specifying the feed options. This message is identical to the body of a regular HTTP POST to _changes, i.e. it’s a JSON object whose keys are the parameters (for example, {"since": 112233, "include_docs": true}). Depending on which client you use, make sure that options are sent as binary.

Messages

Once the server receives the options, it will begin to send text-format messages. The messages are JSON; each contains one or more change notifications (in the same format as the regular feed) wrapped in an array:

[ {"seq":1022,"id":"beer_Indiana_Amber","changes":[{"rev":"1-e8f6b2e1f220fa4c8a64d65e68469842"}]},
  {"seq":1023,"id":"beer_Iowa_Pale_Ale","changes":[{"rev":"1-db962c6d93c3f1720cc7d3b6e50ac9df"}]}
]

(The current server implementation sends at most one notification per message, but this could change. Clients should accept any number.)

An empty array is a special case: it denotes that at this point the feed has finished sending the backlog of existing revisions, and will now wait until new revisions are created. It thus indicates that the client has "caught up" with the current state of the database.

The websocket mode behaves like the continuous mode: after the backlog of notifications (if any) is sent, the connection remains open and new notifications are sent as they occur.

Compressed Feed

For efficiency, the feed can be sent in compressed form; this greatly reduces the bandwidth and is highly recommended.

To signal that it accepts a compressed feed, the client adds "accept_encoding":"gzip" to the feed options in the initial message it sends.

Compressed messages are sent from the server as binary. This is of course necessary as they contain gzip data, and it also lets the client distinguish them from uncompressed messages. (The server will only ever send one kind.)

The compressed messages sent from the server constitute a single stream of gzip-compressed data. They cannot be decompressed individually! Instead, the client should open a gzip decompression session when the feed opens, and write each binary message to it as input as it arrives. The output from the decompressor consists of a sequence of JSON arrays, each of which has the same interpretation as a text message (above).

Webhooks

Since Sync Gateway 1.1, you can configure webhooks to detect document changes and to post changed documents to URLs that you specify. In more detail, the steps for a single webhook event handler are:

  1. Raise and listen for events: Document changes (creations, updates, and deletions) that are made through Sync Gateway’s Public REST API, including document changes that result from Couchbase Lite push replications, raise events that webhook event handlers listen for.

  2. Filter: You can define a filter function to examine the contents of the changed documents, and to decide which ones to post.

  3. Post: Sync Gateway uses asynchronous HTTP or HTTPS POSTs to post the changed documents identified by the filter function to the specified URL. Without a filter function, Sync Gateway posts all changed documents.

You can define multiple webhook event handlers. For example, you could define webhooks with different filtering criteria and that post changed documents to different URLs.

Webhooks post your application’s data, which might include user data, to URLs. Consider the security implications.

When events are raised

Sync Gateway raises a document_changed event every time it writes a document to a Couchbase Server bucket, such as during a Couchbase Lite push replication session.

You can configure event handlers for webhooks with the event_handlers property in the database configuration section of the JSON configuration file.

Examples

Following is a simple example of a webhook event handler. In this case, a single instance of a webhook event handler is defined for the event document_changed. Every time a document changes, the document is sent to the URL http://someurl.com.

"event_handlers": {
    "document_changed": [
        {
            "handler": "webhook",
            "url": "http://someurl.com"
        }
    ]
}

Following is an example that defines two webhook event handlers. The filter function in the first handler recognizes documents with doc.type equal to A and posts the documents to the URL http://someurl.com/type_A. The filter function in the second handler recognizes documents with doc.type equal to B and posts the documents to the URL http://someurl.com/type_B.

"event_handlers": {
      "document_changed": [
        {"handler": "webhook",
         "url": "http://someurl.com/type_A",
         "filter": `function(doc) {
              if (doc.type == "A") {
                return true;
              }
              return false;
            }`
         },
        {"handler": "webhook",
         "url": "http://someurl.com/type_B",
         "filter": `function(doc) {
              if (doc.type == "B") {
                return true;
              }
              return false;
            }`
        }
     ]
  }