Language Constructs

    +
    The language constructs are fundamental units of a language. This topic discusses the JavaScript constructs that have been removed and new constructs that have been added in order to support the requirements of Couchbase Functions.

    Using JavaScript, you can write your custom Functions. Couchbase Functions inherit support for most ECMAScript constructs by using Google v8 as the execution container. However, to support the ability to shard and scale Function-execution automatically, some capabilities have been removed. Additionally, to optimize language-utilization of the server environment, some new constructs have been added.

    While every effort is made to ensure the accuracy of the content of the Eventing documentation herein, it should be noted that the controlling technical document is the Couchbase 7.0 Eventing Specification available on GitHub.

    Removed Language Features

    The following JavaScript features have been removed and cannot be used in handlers:

    Global State

    Function handlers do not allow global variables. All state must be saved and retrieved from persistence providers. In Couchbase Server, the Data Service provider is used as a persistence provider. Therefore, all global states are contained in the Data Service bucket(s) made available to the Function handlers through bindings. This restriction is mandatory for the Function’s handler logic to remain agnostic of the rebalance operation.

    var count = 0;                         // Not allowed - global variable.
    function OnUpdate(doc, meta) {
      count++;
    }

    Note the use of 'Constant alias' bindings in the Function’s settings can be used to provide global constants accessible within a Function’s JavaScript. For example you might have a Constant alias of debug with a value of true (or false) to control verbose logging this would behaves just like adding a statement const debug = true; at the beginning of your JavaScript code.

    Asynchrony

    Asynchrony, particularly asynchronous callbacks, to be useful needs to retain access to their parent scope. As such asynchrony forms a node specific, long-running state that prevents the capture of the entire state in the persistence providers. Therefore, Function handlers are restricted to executing as short-running, straight-line code, without sleep and wakeups.

    function OnUpdate(doc, meta) {
      setTimeout(function(){}, 300);     // Not allowed - asynchronous flow.
    }

    Limited asynchrony is added back through time observers (or Timers). Time observers are designed specifically not to make the state node specific.

    Browser and other Extensions

    Function handlers execute as server-side code on Couchbase Server similar to the JavaScript code that is used in browsers.

    Because handlers do not execute in the context of a browser, the extensions that browsers add to the core language, such as window methods, DOM events etc. are not available. The Couchbase Server prevents these browser extensions from executing in a handler.  However a limited subset is added back (such as function timers in lieu of setTimeout, and curl calls in lieu of XHR).

    For example some code that runs in the browser is excluded from use in Eventing Functions. The ‘window’ term in the code window.XMLHttpRequest(), is not a server-side construct but is in the context of a browser and as such is not available to your Function handlers.

    function OnUpdate(doc, meta) {
      var rpc = window.XMLHttpRequest();  // Not allowed - browser extension.
    }

    Added Language Features

    The following constructs have been added:

    Basic Keyspace Accessors

    Couchbase buckets, when bound to a Function, appear as a global JavaScript map. The map operations such as get, set and delete are exposed to the Data Service providers get, set, and delete operations respectively.

    function OnUpdate(doc, meta) {
      // Assuming 'dest' is a bucket alias or binding to a keyspace
      var val = dest[meta.id];         // this is a bucket GET operation.
      dest[meta.id] = {"status":3};    // this is a bucket SET operation.
      delete dest[meta.id];            // this is a bucket DEL operation.
    }
    • The GET operation (operator [] applied on a bucket binding and used as a value expression)

      This fetches the corresponding object from the KV bucket the variable is bound to, and returns the parsed JSON value as a JavaScript object. Fetching a non-existent object from a bucket will return JavaScript undefined value. This operation throws an exception if the underlying bucket GET operation fails with an unexpected error.

    • The SET operation (operator [] appearing on left of = assignment statement)

      This sets the provided JavaScript value into the KV bucket the variable is bound to, replacing any existing value with the specified key. This operation throws an exception if the underlying bucket SET operation fails with an unexpected error.

    • The DELETE operation (operator [] appearing after JavaScript delete keyword)

      This deletes the provided key from the KV bucket the variable is bound to. If the object does not exist, this call is treated as a no-op. This operation throws an exception if the underlying bucket DELETE operation fails with an unexpected error.

    Advanced Keyspace Accessors

    The following advanced bucket accessors have been added to expose a richer set of options and operators. Unlike the basic bucket accessors these operations have non-trivial argument sets and return values for complete details refer to Advanced Keyspace Accessors Details.

    • The Advanced GET operation: result = couchbase.get(binding, meta)

      This operation allows reading a document from the bucket with an ability to specify the CAS value to be matched during the read. For more information see advanced GET operation.

    • The Advanced INSERT operation: result = couchbase.insert(binding, meta, doc)

      This operation allows creating a fresh document in the bucket. This operation will fail if the document with the specified key already exists. It allows specifying an expiration time (or TTL) to be set on the document. For more information see advanced INSERT operation.

    • The Advanced UPSERT operation: result = couchbase.upsert(binding, meta, doc)

      This operation allows updating an existing document in the bucket, or if absent, creating a fresh document with the specified key. The operation does not allow specifying CAS (it will silently ignore it). It also allows specifying an expiration time (or TTL) to be set on the document. For more information see advanced UPSERT operation.

    • The Advanced REPLACE operation: result = couchbase.replace(binding, meta, doc)

      This operation replaces an existing document in the bucket This operation will fail if the document with the specified key does not exist. This operation allows specifying a CAS value that must be matched as a pre-condition before proceeding with the operation. It also allows specifying an expiration time (or TTL) to be set on the document. For more information see advanced REPLACE operation.

    • The Advanced DELETE operation: result = couchbase.delete(binding, meta)

      This operation allows deleting a document in the bucket specified by key. Optionally, a CAS value may be specified which will be matched as a pre-condition to proceed with the operation. For more information see advanced DELETE operation.

    • The Advanced INCREMENT operation: result = couchbase.incrment(binding, meta)

      This operation atomically increments the field "count" in the specified document. For more information see advanced INCREMENT operation.

      The document must have the below structure:

      {"count": 23} // 23 is the current counter value

      The increment operation returns the post-increment value.

      If the specified counter document does not exist, one is created with count value as 0 and the structure noted above. And so, the first returned value will be 1.

      Due to limitations in KV engine API, this operation cannot currently manipulate full document counters.

    • The Advanced DECREMENT operation: result = couchbase.decrement(binding, meta)

      This operation atomically decrements the field "count" in the specified document. For more information see advanced DECREMENT operation.

      The document must have the below structure:

      {"count": 23} // 23 is the current counter value

      The decrement operation returns the post-decrement value.

      If the specified counter document does not exist, one is created with count value as 0 and the structure noted above. And so, the first returned value will be -1.

      Due to limitations in KV engine API, this operation cannot currently manipulate full document counters.

    Logging

    An additional function, log() has been introduced to the language, which allows handlers to log user defined messages. These log() statements will go the specific handler’s log file also known as the application log. The messages go files located in the Eventing data directory and do not contain any system log messages. The function takes a string to write to the file. If non-string types are passed, a best effort string representation will be logged, but the format of these may change over time. This function does not throw exceptions. For more information see application logs.

    function OnUpdate(doc, meta) {
      log("Now processing: " + meta.id);
    }

    The Eventing Service also creates a system log file named eventing.log common across all handlers to capture management and lifecycle information, however the end-user cannot write to this file. For more information see system log.

    N1QL Statements

    Top level N1QL keywords, such as SELECT, UPDATE, INSERT and DELETE, are available as inline keywords in handlers. Operations that return values such as SELECT are accessible through a returned iterable handle. N1QL Query results, via a SELECT, are streamed in batches to the iterable handle as the iteration progresses through the result set.

    N1QL DML statements cannot manipulate documents in the same bucket as the handler is listening for mutations on to avoid recursion. Workaround: use the exposed data service KV map in your Eventing function.

    JavaScript variables can be referred by N1QL statements using $<variable> syntax. Such parameters will be substituted with the corresponding JavaScript variable’s runtime value using N1QL named parameters substitution facility.

    When deploying the below Function with a feed boundary of "Everything" the same N1QL statement will execute 7,303 times. If the feed boundary is configured to "From now" and you then mutate just one (1) document in the keyspace beer-sample._default._default only one (1) query will be executed. Also keep in mind that adding an optimal index can speed up the query performance by 24X.

    function OnUpdate(doc, meta) {
        var strong = 70;
        var results =
            SELECT *                               /* N1QL queries are embedded directly.    */
            FROM `beer-sample`._default._default   /* Token escaping is standard N1QL style. */
            WHERE abv > $strong;                   // Local variable reference using $ syntax.
        for (var beer of results) {                // Stream results using 'for' iterator.
            log(beer);
            break;
        }
        results.close();                           // End the query and free resources held
    }

    The embedded N1QL call starts the query and returns a JavaScript Iterable object representing the result set of the query. The query is streamed in batches as the iteration proceeds. The returned handle can be iterated using any standard JavaScript mechanism including for…​of loops.

    In multiline N1QL statements (as above) you cannot use single line // end of line comments like this
    prior to the terminating semicolon as it will cause a syntax error in the transpilation of the N1QL statement, however multiline /* comments like this */ are allowed.

    The iterator is an input iterator (elements are read-only). The keyword this cannot be used in the body of the iterator. The variables created inside the iterator are local to the iterator.

    The returned handle must be closed using the close() method defined on it, which stops the underlying N1QL query and releases associated resources.

    When a handler completes for a given mutation and exits all resources will be freed even if you omit the close() statement for your result set(s). However in some complex use cases such as nested N1QL lookups a failure to explicitly call close() after each result set is no longer needed can tie up an excessive amount of N1QL resources and lead to poor performance.

    All three operations, i.e., the N1QL statement, iterating over the result set, and closing the Iterable handle can throw exceptions if unexpected error arises from the underlying N1QL query.

    As N1QL is not syntactically part of the JavaScript language, the handler code is transpiled to identify valid N1QL statements which are then converted to a standard JavaScript function call that returns an Iterable object with addition of a close() method.

    You must use $<variable>, as per N1QL specification, to use a JavaScript variable in the query statement. The object expressions for substitution are not supported and therefore you cannot use the meta.id expression in the query statement.

    Instead of meta.id expression, you can use var id = meta.id in an N1QL query.

    • Invalid N1QL Statement

      DELETE FROM mybucket.myscope.transactions WHERE username = $meta.id;
    • Valid N1QL Statement

      var id = meta.id;
      DELETE FROM mybucket.myscope.transactions WHERE username = $id;

    When you use a N1QL query inside a Function handler, remember to use an escaped identifier for keyspaces (bucket.scope.collection) with special characters (`bucket-name`). Escaped identifiers are surrounded by back ticks and support all identifiers in JSON

    For example:

    • If the bucket name is beer-sample and the scope and collection are both _default, then only the bucket in the N1QL needs to be escaped:

      SELECT * FROM `beer-sample`._default._default WHERE type ...
    • However if the bucket name was beersample, then the keyspace of the N1QL query needs no escaping:

      SELECT * FROM beersample._default._default WHERE type ...

    Built-in Functions

    The following built in functions have been added:

    The N1QL() Function Call

    The N1QL() function call is documented below for reference purposes but should not used directly as doing so would bypass the various semantic and syntactic checks of the transpiler (notably: recursive mutation checks will no longer function, and the statement will need to manual escaping of all N1QL special sequences and keywords).

    In addition the N1qlQuery() is now deprecated and has been replaced with the N1QL() call which has a different parameter format.
    • statement

      This is the identified N1QL statement. This will be passed to N1QL via SDK to run as a prepared statement. All referenced JS variables in the statement (using the $var notation) will be treated by N1QL as named parameters.

    • params

      This can be either a JavaScript array (for positional parameters) or a JavaScript map. When the N1QL statement utilizes positional parameters (i.e., $1, $2 …​), then params is expected to be a JavaScript array corresponding to the values to be bound to these positional parameters. When the N1QL statement utilizes named parameters (i.e., $name), then params is expected to be a JavaScript map object providing the name-value pairs corresponding to the variables used by the N1QL statement. Positional and named value parameters cannot be mixed.

      Note, adding an optimal index to the travel-sample._default.`_default keyspace for the below query can increase the performance by 57X.

      iterator using a positional params array

          // Using `travel-sample`._default._default to demonstrate params.
          // a) Positional param 1 is field 'iata' from the input doc
          // b) Positional param 2 from a Handler variable: max_dist
          // c) Will also prepare the statement for better performance
      
          if (doc.type !== "airline") return; // only process airline docs
      
          var max_dist = 120;
          var results = N1QL(
              "SELECT COUNT(*) AS cnt " +
              "FROM `travel-sample`._default._default " +
              "WHERE type = \"route\" " +
              "AND airline = $1 AND distance <= $2",
              [doc.iata,max_dist],
              { 'isPrepared': true }
          );

      Example iterator using a named params object

          // Using `travel-sample`._default._default to demonstrate named params.
          // a) Named param 1 '$mytype' is a hardcode
          // b) Named param 2 '$myairline' is field 'iata' from the input doc
          // c) Named param 3 '$mydistance' if from a Handler variable max_dist
          // d) Set the consistency in the options to none
      
          if (doc.type !== "airline") return; // only process airline docs
      
          var max_dist = 120;
          var results = N1QL("SELECT COUNT(*) AS cnt " +
              "FROM `travel-sample`._default._default " +
              "WHERE type = $mytype " +
              "AND airline = $myairline AND distance <= $mydistance",
              { '$mytype': 'route', '$mydistance': max_dist, '$myairline': doc.iata },
              { 'consistency': 'none' }
          );
    • options

      This is a JSON object having various query runtime options as keys. Currently, the following settings are recognized:

      • isPrepared

        This controls if the statement will be prepared. Normally, this defaults to false but can be set on a per statement basis to true for any N1QL query that needs increased performance.

      • consistency

        This controls the consistency level for the statement. Normally, this defaults to the consistency level specified in the overall handler settings but can be set on a per statement basis. The valid values are "none" and "request".

    • return value (handle)

      The call returns a JavaScript Iterable object representing the result set of the query. The query is streamed in batches as the iteration proceeds. The returned handle can be iterated using any standard JavaScript mechanism including for…​of loops.

      • close() Method on handle object (return value)

        This releases the resources held by the N1QL query. If the query is still streaming results, the query is cancelled.

    • Exceptions Thrown

      The N1QL() function throws an exception if the underlying N1QL query fails to parse or start executing. The returned Iterable handler throws an exception if the underlying N1QL query fails after starting. The close() method on the iterable handle can throw an exception if underlying N1QL query cancellation encounters an unexpected error.

    The crc64() Function Call

    crc64(): This function calculates the CRC64 hash of an object using the ISO polynomial. The function takes one parameter, the object to checksum, and this can be any JavaScript object that can be encoded to JSON. The hash is returned as a string (because JavaScript numeric types offers only 53-bit precision). Note that the hash is sensitive to ordering of parameters in case of map objects.

    function OnUpdate(doc, meta) {
        var crc_str = crc64(doc);
        /// code here ...
    }

    The crc64 function can be useful in cases like suppressing a duplicate mutation from the Sync Gateway (SG), when both the Sync Gateway & Eventing are leveraging the same bucket. Basically, Sync Gateway updates metadata of the document within the bucket, which in turn generates an event for Eventing to process. Eventing can’t differentiate between events from Sync Gateway and other events (doc updates via SDK, N1QL, and others). A workaround to this double mutation issue is possible via the crc64() function.

    function OnUpdate(doc, meta) {
        // Ignore documents created by Sync Gateway
        if(meta.id.startsWith("_sync") == true) return;
    
        // Ignore documents whose body has not changed since we last saw it
        var prev_crc = checksum_bucket[meta.id];
        var curr_crc = crc64(doc);
        if (prev_crc === curr_crc) return;
        checksum_bucket[meta.id] = curr_crc;
    
       // Business logic goes in here
    }

    Note that if multiple Eventing functions (or handlers) share the same Sync Gateway crc64() checksum documents, real mutations will be suppressed and missed. In this use case make the checksum documents unique to each handler, i.e. checksum_bucket["handler1:" + meta.id], checksum_bucket["handler2:" + meta.id], etc.

    Timers

    Timers are asynchronous compute, which offers Eventing Functions the ability to execute in reference to wall-clock events, refer to Timers.

    The createTimer() Function Call: createTimer(callback, date, reference, context)

    To create a timer a callback or JavaScript function will be executed at or close to the desired date. The reference is an identifier for the timer scoped to an Eventing function and callback. The context must be serializable data that is available to the callback when the timer is fired. For more information see createTimer function.

    The cancelTimer() Function Call: cancelTimer(callback, reference)

    To cancel a timer you can either by invoking createTimer() with the same reference of an existing timer or you can use the _cancelTimer() function. For more information see cancelTimer function.

    cURL

    The curl() Function Call: response_object = curl(method, binding, [request_object])

    The curl() function provides a way of interacting with external entities via a REST endpoint from Eventing Functions using either HTTP or HTTPS. For more information see curl function.

    Handler Signatures

    Eventing Service or framework calls the following JavaScript functions as entry points to the handler.

    OnUpdate Handler

    This handler gets called when a document is created or modified, e.g. Insert/Update. The handler listens to mutations (the creation or modification of documents) in the associated source Bucket.

    In this handler the following limitations exist, both limitations arise due to KV engine design choices and may be revisited in the future:

    • If a document is modified several times in a short duration, the calls may be coalesced into a single event due to deduplication.

    • It is not possible to distinguish between a Create and an Update operation.

    A sample OnUpdate handler is displayed below:

    function OnUpdate(doc, meta) {
      if (doc.type === 'order' && doc.value > 5000) {
        // ‘phonverify’ is a bucket alias or binding to a keyspace.
        phoneverify[meta.id] = doc.customer;
      }
    }

    OnDelete Handler

    This handler gets called when a document is deleted or removed due to an expiry. The handler listens to mutations (deletions or expirations) in the associated source Bucket. It became possible to determine if the document was deleted or expired. This is accomplished via the optional argument "options" — a JavaScript map object with a boolean property named 'expired'.

    In this handler the following limitation exists. This limitation arises due to KV engine design choices and may be revisited in the future:

    • It is not possible to distinguish if the document was removed is a Delete operation or removed as a result of a document expiration.

    A sample OnDelete handler is displayed below:

    function OnDelete(meta,options) {
        if (options.expired) log("Document expired", meta.id);
        var addr = meta.id;
        var res = SELECT id from mybucket.myscope.orders WHERE shipaddr = $addr;
        for (var id of res) {
            log("Address invalidated for pending order: " + id);
        }
    }

    Note that the pre-6.6.0 argument syntax is still fully supported, but you will not be able to differentiate deletion from expiration.

    function OnDelete(meta) {
        log("Document deleted or expired", meta.id);
    }

    Reserved Words

    Reserved words are words that cannot be used in a handler as a variable name, function name, or as a property in the Function handler code. The following table lists the reserved words that you must refrain from using as they are used by the transpiler to integrate Couchbase’s query language, N1QL with Eventing.

    N1QL Keywords

    ALTER

    EXECUTE

    MERGE

    UPDATE

    BUILD

    EXPLAIN

    PREPARE

    UPSERT

    CREATE

    GRANT

    RENAME

    DELETE

    INFER

    REVOKE

    DROP

    INSERT

    SELECT

    What Happens If You Use a Reserved Word?

    Let’s say you try to create a new Function handler code using a reserved word for variable names, for function names, and as a property binding value. All three cases generate a deployment error.

    Reserved words as a variable name:

    function get_numip_first_3_octets(ip) {
        var grant = 0;
        if (ip) {
            var parts = ip.split('.');
        }
    }

    Reserved words as a function name:

    function grant(ip) {
        var return_val = 0;
        if (ip) {
            var parts = ip.split('.');
        }
    }

    During the Function deployment step, when the system validates the handler code, it displays an error message such as the following:

    + Sample Error Message - Deployment failed: Syntax error (<line and column numbers>) - grant is a reserved name in N1QLJs

    Reserved words as a property bindings value

    reserved words 7 0