Create a Recurring Timer with a REST API

Capella Operational

Create a recurring Timer that fetches documents from an external REST endpoint until you manually cancel it.

This page walks you through creating an Eventing Function that contains an OnUpdate handler with a cURL GET request in a Timer callback function.

The OnUpdate JavaScript handler listens to mutations or data changes within a specified source collection. When you create or modify data in the source collection, the Eventing Function executes its JavaScript code and fetches exchange rates.

The Timer callback function creates the initial Timer 30 seconds into the future. It then creates the next Timer 2-N at the start of the following day.

The Timer callback function relies on a control document which, if mutated, controls whether a recurring Timer is created or cancelled.

This page contains the following:

An example where a control document is created of mutated, which creates a Timer. This Timer initially fires 30 seconds into the future. It fetches a document from an external REST endpoint and writes the document to the source bucket.

The original control document does not change.

The Timer is rearmed at the start of the following day, and continues to fetch daily exchange rates until you cancel it.
An example where the control document is mutated, which cancels any existing Timer with a reference that matches the control document’s meta.id. This has no effect if the Timer created has already fired.

Prerequisites

Before trying out the examples on this page, you must first test your API connection and create buckets, scopes, and collections.

Test API Connection

The examples on this page rely on a public REST API that you can access through a cURL GET operation. You can test your connection to make sure the REST endpoint is live.

To test your connection, enter the following command in your terminal:

curl -q -X GET 'https://api.frankfurter.app/latest'

If the URL is offline, you can start your own Python server. To do so, enter the following in your terminal:

#!/usr/bin/env python3

from http.server import HTTPServer, BaseHTTPRequestHandler
from http import HTTPStatus
from datetime import datetime
import json
import time

class _RequestHandler(BaseHTTPRequestHandler):
    # Borrowing from https://gist.github.com/nitaku/10d0662536f37a087e1b
    def _set_headers(self):
        self.send_response(HTTPStatus.OK.value)
        self.send_header('Content-type', 'application/json')
        self.send_header('Access-Control-Allow-Origin', '*')
        self.end_headers()

    def do_GET(self):
        message = '{"amount":1.0,"base":"EUR","date":"' + \
            datetime.today().strftime('%Y-%m-%d') + \
            '","rates":{"AUD":1.5907,"BGN":1.9558,"USD":1.1802}}'
        self._set_headers()
        self.wfile.write(json.dumps(message).encode('utf-8'))

def run_server():
    server_address = ('', 8001)
    httpd = HTTPServer(server_address, _RequestHandler)
    print('serving at %s:%d' % server_address)
    httpd.serve_forever()

if __name__ == '__main__':
    run_server()

A successful JSON payload should look similar to the following:

{
  "base": "EUR",
  "rates": {
    "GBP": 0.90265,
    "HKD": 9.1871,
    "IDR": 17247.57,
    "ILS": 4.0397,
    "DKK": 7.4508,
    "INR": 88.709,
    "CHF": 1.077,
    "MXN": 26.7125,
    "CZK": 26.097,
    "SGD": 1.6228,
    "THB": 36.759,
    "HRK": 7.4683,
    "MYR": 4.9698,
    "NOK": 10.6585,
    "CNY": 8.2277,
    "BGN": 1.9558,
    "PHP": 58.12,
    "SEK": 10.2865,
    "PLN": 4.3935,
    "ZAR": 20.4221,
    "CAD": 1.5703,
    "ISK": 160.2,
    "BRL": 6.2311,
    "RON": 4.8345,
    "NZD": 1.7809,
    "TRY": 8.3311,
    "JPY": 125.37,
    "RUB": 86.3692,
    "KRW": 1405.74,
    "USD": 1.1854,
    "HUF": 344.5,
    "AUD": 1.6415
  },
  "date": "2020-08-05"
}

If you’re running your own Python server, the returning JSON data is static but contains a current date.

curl -q -X GET 'https://localhost:8001/api/latest'

{
    "amount": 1.0,
    "base": "EUR",
    "date": "2021-07-18",
    "rates": {
        "AUD": 1.5907,
        "BGN": 1.9558,
        "USD": 1.1802
    }
}

You must use a real IP address if you’re not on the Couchbase Server.

Create Keyspaces

Create the following keyspaces:

Create two buckets called bulk and rr100 with a minimum size of 100MB.
Inside the bulk bucket, create one keyspace called bulk.data.source.
Inside the rr100 bucket, create one keyspace called rr100.eventing.metadata.

For more information about creating buckets, scopes, and collections, see Manage Buckets.

Do not add, modify, or delete documents in the Eventing storage keyspace rr100.eventing.metadata while your Eventing Functions are in a deployed state.

Setup

Before following the examples on this page, you must set up a control document and an Eventing Function.

Create the Control Document

To create the control document:

Go to Data Tools Documents.
Select the keyspace bulk.data.source in the Get documents from list.
Click Create Document.
In the Document ID field, enter recurring_timer::1.

Replace the JSON text with the following:

{
  "type": "recurring_timer",
  "id": 1,
  "active": false
}

Click Save to create the document.

Create an Eventing Function

To create a new Eventing Function:

Go to Data Tools Eventing.
Click Add Function.
In the Settings page, enter the following Function settings:
- external_rest_via_curl_get under Name.
- Use a recurring Timer to access an external REST endpoint through a GET operation. The initial fetch is 30 seconds into the future, with following fetches happening at the start of each subsequent day. under Description.
- The keyspace bulk.data.source under Listen to Location.
- The keyspace rr100.eventing.metadata under Eventing Storage.
Click Next.
In the Bindings page, click Add Binding and create two bindings.
- For the first binding:
  - Select Bucket.
  - Enter src_col as the Alias Name.
  - Enter the keyspace bulk.data.source under Bucket, Scope, and Collection.
  - Select Read and Write under Permission.
- For the second binding:
  - Select URL.
  - Enter exchangeRateApi as the Alias Name.
  - Enter https://api.frankfurter.app/latest as the URL.
  - Select No Auth under Authentication.
Click Next.

In the code editor, replace the placeholder JavaScript code with the following code sample:

function CreateRecurringTimer(context) {
    log('From CreateRecurringTimer: creating timer', context.mode, context.id);
    var nextSchedule = null;
    if (context.mode === "via_onupdate") {
        // Creates a timestamp 30 seconds from now for the initial Timer
        var thirtySecFromNow = new Date(); // Gets current time & adds 30 seconds to it
        thirtySecFromNow.setSeconds(thirtySecFromNow.getSeconds() + 30);
        nextSchedule = thirtySecFromNow;
    } else {
        // Must be: context.mode === "via_callback"
        // Creates UTC timestamp to fire a Timer for the next day
        // Does this for timers 2 to N
        var tomorrow = new Date();
        tomorrow.setHours(0,0,0,0);
        tomorrow.setDate(tomorrow.getDate() + 1);
        nextSchedule = tomorrow;
    }
    log("Finish CreateRecurringTimer (local time) nextSchedule", localISOTime(nextSchedule));
    createTimer(RecurringTimerCallback, nextSchedule, context.id, context);
}

function localISOTime (indate) {
    // JavaScript works with dates as UTC times, but sometimes you might prefer to see local time
    return new Date(indate.getTime() - indate.getTimezoneOffset() * 60 * 1000)
        .toISOString().split(/[TZ]/).slice(0, 2).join('T');
}

function RecurringTimerCallback(context) {
    log('From RecurringTimerCallback: timer fired', context);
    // Does any sort of recurring work here and updates a date_stamp in a document
    var now = new Date();
    var nowLoc = localISOTime(now);
    var dt_beg = now.getTime();
    // Generates a YYYY-MM-DD string in UTC for Yesterday
    var yesterday = new Date();
    yesterday.setHours(0,0,0,0);
    yesterday.setDate(yesterday.getDate() - 1);
    var apiReqDateUtc = yesterday.toISOString().substring(0, 10);
    // Generates a YYYY-MM-DD string in Local Time for Yesterday
    var apiReqDateLoc = localISOTime(yesterday).substring(0, 10);
    try {
        // ==============================
        // Performs a cURL GET here
        var request = {
            path: apiReqDateUtc
        };
        // Performs the cURL request using the URL alias form the settings
        var response = curl('GET', exchangeRateApi, request);
        var status = "OKAY";
        if (response.status != 200 && response.status != 302) {
            status = "FAIL";
        }
        // ==============================
        var curl_time_ms = new Date().getTime() - dt_beg;
        log('USER FUNCTION DONE ' + status +
            ' (curl ' + response.status + ' took ' + curl_time_ms + ' ms.)');
        if (response && response.body && response.body.date && response.body.base) {
            // Writes the exchange lookup table document
            // Does this 365 times a year
            src_col["exchange::" + response.body.date] = response.body;

            // Write status document of success
            src_col["cur_" + context.id] = {
                "last_update_loc": nowLoc,
                "last_update_utc": now, "apiReqDateUtc": apiReqDateUtc,
                "curl_success": true,  "valid": true, "curl_time_ms": curl_time_ms
            };

        } else {
            // Write status doc of failure
            src_col["cur_" + context.id] = {
                "last_update_loc": nowLoc,
                "last_update_utc": now, "apiReqDateUtc": apiReqDateUtc,
                "curl_success": true, "body_valid": false,  "curl_time_ms": curl_time_ms
            };
        }
    } catch (e) {
        var curl_time_ms = new Date().getTime() - dt_beg;
        log('USER FUNCTION DONE ' + status +
            ' (curl ERROR ' + e + ' took ' + curl_time_ms + ' ms.)');
        // Write status document of failure
        src_col["cur_" + context.id] = {
            "last_update_loc": nowLoc,
            "last_update_utc": now, "apiReqDateUtc": apiReqDateUtc,
            "curl_success": false, "body_valid": false, "curl_time_ms": curl_time_ms
        };
    }
    // Rearms the Timer
    CreateRecurringTimer({ "id": context.id, "mode": "via_callback" })
}

function OnUpdate(doc, meta) {
    // Filters mutations of interest
    if (doc.type !== 'recurring_timer') return;
    if (doc.active === false) {
        if (cancelTimer(RecurringTimerCallback, meta.id)) {
            log('From OnUpdate: canceled active Timer, doc.active', doc.active, meta.id);
        } else {
            log('From OnUpdate: no active Timer to cancel, doc.active', doc.active, meta.id);
        }
    } else {
        log('From OnUpdate: create/overwrite doc.active', doc.active, meta.id);
        CreateRecurringTimer({  "id": meta.id, "mode": "via_onupdate" });
    }
}

Click Create function to create your Eventing Function.

When a change happens to the data inside the source collection, the OnUpdate handler targets the control document by ignoring all documents that do not have a doc.type of recurring_timer. It then uses the field active to determine which action to take:

If active is true, the Eventing Function creates a series of daily Timers, with the first Timer firing 30 seconds into the future and subsequent Timers firing at the beginning of every following day.
If active is false, the Eventing Function cancels any existing Timers.

When a Timer created by the Eventing Function fires, the callback RecurringTimerCallback executes and writes a new document in the source collection with a similar key as another document in the source collection.

Deploy the Eventing Function

Deploy your Eventing Function:

Go to Data Tools Eventing.
Click More Options (⋮) next to recurring_timer.
Click Deploy to deploy your Function.

After it’s deployed, the Eventing Function executes on all existing documents and any documents you create in the future.

Example: Create a Recurring Timer and Allow the Timer to Fire and Rearm

This example walks you through how to create a Timer, have the Timer fire, and then have the Timer rearm.

Edit the Control Document

To edit the control document:

Go to Data Tools Documents.
Select the keyspace bulk.data.source in the Get documents from list.
Click the control document recurring_timer::1 to open the Edit Document dialog.

Change active to true:

{
  "type": "recurring_timer",
  "id": 1,
  "active": true
}

Click Save to create a mutation.

The document mutation causes the Eventing Function to create a Timer.

Check the Eventing Function Log

To check the Eventing Function log:

Go to Data Tools Eventing.

Click the Log icon next to the external_rest_via_curl_get Eventing Function. You should see the following in the debug log:

2021-07-18T14:37:25.136-07:00 [INFO] "Finish CreateRecurringTimer (local time) nextSchedule" "2021-07-18T14:37:55.134"
2021-07-18T14:37:25.134-07:00 [INFO] "From OnUpdate: create/overwrite doc.active" true "recurring_timer::1"
2021-07-18T14:37:25.134-07:00 [INFO] "From CreateRecurringTimer: creating timer" "via_onupdate" "recurring_timer::1"
2021-07-18T14:36:55.177-07:00 [INFO] "From OnUpdate: no active Timer to cancel, doc.active" false "recurring_timer::1"

Wait a few minutes and click the Log icon again. The Timer should have fired and executed the RecurringTimerCallback callback, and you should see the following in the debug log:

2021-07-18T14:37:59.164-07:00 [INFO] "From CreateRecurringTimer: creating timer" "via_callback" "recurring_timer::1"
2021-07-18T14:37:59.164-07:00 [INFO] "Finish CreateRecurringTimer (local time) nextSchedule" "2021-07-19T00:00:00.000"
2021-07-18T14:37:59.161-07:00 [INFO] "USER FUNCTION DONE OKAY (curl 200 took 443 ms.)"
2021-07-18T14:37:58.718-07:00 [INFO] "From RecurringTimerCallback: timer fired" {"id":"recurring_timer::1","mode":"via_onupdate"}

Check the Results in the Source Collection

To check that a new document has been created in the source collection:

Go to Data Tools Documents.
Select the keyspace bulk.data.source in the Get documents from list.

Click the new document cur_recurring_timer::1 to open the Edit Document dialog. The JSON document includes data written by the Timer’s callback.

{
  "last_update_loc": "2021-07-18T14:04:06.408",
  "last_update_utc": "2021-07-18T21:04:06.408Z",
  "apiReqDateUtc": "2021-07-17",
  "curl_success": true,
  "body_valid": false,
  "curl_time_ms": 442
}

Click Cancel to close the editor.

The Eventing Function you created writes a timestamp to the cur_recurring_timer::1 document at the beginning of every following day.

Example: Cancel the Recurring Timer

This example walks you through how to cancel the recurring Timer.

Edit the Control Document

To edit the control document:

Go to Data Tools Documents.
Select the keyspace bulk.data.source in the Get documents from list.
Click the control document recurring_timer::1 to open the Edit Document dialog.

Change active to false:

{
  "type": "recurring_timer",
  "id": 2,
  "active": false
}

Click Save to create a mutation.

The document mutation causes the Eventing Function to create a Timer.

Check the Eventing Function Log

To check the Eventing Function log:

Go to Data Tools Eventing.
Click the Log icon next to the external_rest_via_curl_get Eventing Function. You should see the line "From OnUpdate: canceled active Timer, doc.active" false "recurring_timer::1" in the debug log.

The recurring Timer has been cancelled.