Language Constructs

    +
    The language constructs are fundamental units of a language. This topic discusses the JavaScript constructs that have been removed and new constructs that have been added in order to support the requirements of Couchbase Functions.

    Using JavaScript, you can write your custom Functions. Couchbase Functions inherit support for most ECMAScript constructs by using Google v8 as the execution container. However, to support the ability to shard and scale Function-execution automatically, some capabilities have been removed. Additionally, to optimize language-utilization of the server environment, some new constructs have been added.

    While every effort is made to ensure the accuracy of the content of the Eventing documentation herein, it should be noted that the controlling technical document is the Couchbase 6.5 Eventing Specification available on GitHub.

    Removed Language Features

    The following JavaScript features have been removed and cannot be used in handlers:

    Global State:

    Function handlers do not allow global variables. All state must be saved and retrieved from persistence providers. In Couchbase Server, the Data Service provider is used as a persistence provider. Therefore, all global states are contained in the Data Service bucket(s) made available to the Function handlers through bindings. This restriction is mandatory for the Function’s handler logic to remain agnostic of the rebalance operation.

    var count = 0;                         // Not allowed - global variable.
    function OnUpdate(doc, meta) {
      count++;
    }

    Asynchrony:

    Asynchrony, particularly asynchronous callbacks, to be useful needs to retain access to their parent scope. As such asynchrony forms a node specific, long-running state that prevents the capture of the entire state in the persistence providers. Therefore, Function handlers are restricted to executing as short-running, straight-line code, without sleep and wakeups. Limited asynchrony is added back through time observers. Time observers are designed specifically not to make the state node specific.

    function OnUpdate(doc, meta) {
      setTimeout(function(){}, 300);     // Not allowed - asynchronous flow.
    }

    Browser and other Extensions:

    Function handlers execute as server-side code on Couchbase Server similar to the JavaScript code that is used in browsers.

    Because handlers do not execute in the context of a browser, the extensions that browsers add to the core language, such as window methods, DOM events etc. are not available. The Couchbase Server prevents these browser extensions from executing in a handler.  However a limited subset is added back (such as function timers in lieu of setTimeout, and curl calls in lieu of XHR).

    For example that the code displayed in the below example runs in the browser. The ‘window’ term in the code window.XMLHttpRequest(), is not a server-side construct but is in the context of a browser and as such is not available to your Function handlers.

    function OnUpdate(doc, meta) {
      var rpc = window.XMLHttpRequest();  // Not allowed - browser extension.
    }

    Added Language Features

    The following constructs have been added:

    Bucket Accessors:

    Couchbase buckets, when bound to a Function, appear as a global JavaScript map. The map operations such as get, set and delete are exposed to the Data Service providers get, set, and delete operations respectively.

    function OnUpdate(doc, meta) {
      // Assuming 'dest' is a bucket alias binding
      var val = dest[meta.id];         // this is a bucket GET operation.
      dest[meta.id] = {"status":3};    // this is a bucket SET operation.
      delete dest[meta.id];            // this is a bucket DEL operation.
    }
    • The GET operation (operator [] applied on a bucket binding and used as a value expression)

      This fetches the corresponding object from the KV bucket the variable is bound to, and returns the parsed JSON value as a JavaScript object. Fetching a non-existent object from a bucket will return JavaScript undefined value. This operation throws an exception if the underlying bucket GET operation fails with an unexpected error.

    • The SET operation (operator [] appearing on left of = assignment statement)

      This sets the provided JavaScript value into the KV bucket the variable is bound to, replacing any existing value with the specified key. This operation throws an exception if the underlying bucket SET operation fails with an unexpected error.

    • The DELETE operation (operator [] appearing after JavaScript delete keyword)

      This deletes the provided key from the KV bucket the variable is bound to. If the object does not exist, this call is treated as a no-op. This operation throws an exception if the underlying bucket DELETE operation fails with an unexpected error.

    Logging:

    An additional function, log() has been introduced to the language, which allows handlers to log user defined messages. These messages go into the eventing data directory and do not contain any system log messages. The function takes a string to write to the file. If non-string types are passed, a best effort string representation will be logged, but the format of these may change over time. This function does not throw exceptions.

    function OnUpdate(doc, meta) {
      log("Now processing: " + meta.id);
    }

    N1QL Statements:

    Top level N1QL keywords, such as SELECT, UPDATE, INSERT and DELETE, are available as inline keywords in handlers. Operations that return values such as SELECT are accessible through a returned Iterable handle. N1QL Query results, via a SELECT, are streamed in batches to the Iterable handle as the iteration progresses through the result set.

    N1QL DML statements cannot manipulate documents in the same bucket as the handler is listening for mutations on to avoid recursion. Workaround: use the exposed data service KV map in your Eventing function.

    JavaScript variables can be referred by N1QL statements using $<variable> syntax. Such parameters will be substituted with the corresponding JavaScript variable’s runtime value using N1QL named parameters substitution facility.

    function OnUpdate(doc, meta) {
        var strong = 70;
        var results =
            SELECT *                  /* N1QL queries are embedded directly.    */
            FROM `beer-sample`       /* Token escaping is standard N1QL style. */
            WHERE abv > $strong;      // Local variable reference using $ syntax.
        for (var beer of results) {   // Stream results using 'for' iterator.
            log(beer);
            break;
        }
        results.close();              // End the query and free resources held
    }

    The call starts the query and returns a JavaScript Iterable object representing the result set of the query. The query is streamed in batches as the iteration proceeds. The returned handle can be iterated using any standard JavaScript mechanism including for…​of loops.

    In multiline N1QL statements (as above) you cannot use single line // end of line comments like this
    prior to the terminating semicolon as it will cause a syntax error in the transpilation of the N1QL statement, however muliti-line /* comments like this */ are allowed.

    The iterator is an input iterator (elements are read-only). The keyword this cannot be used in the body of the iterator. The variables created inside the iterator are local to the iterator.

    The returned handle must be closed using the close() method defined on it, which stops the underlying N1QL query and releases associated resources.

    When a handler completes for a given mutation and exits all resources will be freed even if you omit the close() statement for your result set(s). However in some complex use cases such as nested N1QL lookups a failure to explicitly call close() after each result set is no longer needed can tie up an excessive amount of N1QL resources and lead to poor performance.

    All three operations, i.e., the N1QL statement, iterating over the result set, and closing the Iterable handle can throw exceptions if unexpected error arises from the underlying N1QL query.

    As N1QL is not syntactically part of the JavaScript language, the handler code is transpiled to identify valid N1QL statements which are then converted to a standard JavaScript function call that returns an Iterable object with addition of a close() method.

    You must use $<variable>, as per N1QL specification, to use a JavaScript variable in the query statement. The object expressions for substitution are not supported and therefore you cannot use the meta.id expression in the query statement.

    Instead of meta.id expression, you can use var id = meta.id in an N1QL query.

    • Invalid N1QL Statement

      DELETE FROM `transactions` WHERE username = $meta.id;
    • Valid N1QL Statement

      var id = meta.id;
      DELETE FROM `transactions` WHERE username = $id;

    When you use a N1QL query inside a Function handler, remember to use an escaped identifier for bucket names with special characters (`bucket-name`). Escaped identifiers are surrounded by backticks and support all identifiers in JSON

    For example:

    • If the bucket name is beer-sample, then use the N1QL query such as:

      SELECT * FROM `beer-sample` WHERE type...
    • If bucket name is beersample, then use the N1QL query such as:

      SELECT * FROM beersample WHERE type ...

    Built-in Functions

    The following built in functions have been added:

    The N1QL() function call:

    The N1QL() function call is documented below for reference purposes but should not used directly as doing so would bypass the various semantic and syntactic checks of the transpiler (notably: recursive mutation checks will no longer function, and the statement will need to manual escaping of all N1QL special sequences and keywords).

    In addition the N1qlQuery() is now deprecated and has been replaced with the N1QL() call which has a different parameter format.
    • statement

      This is the identified N1QL statement. This will be passed to N1QL via SDK to run as a prepared statement. All referenced JS variables in the statement (using the $var notation) will be treated by N1QL as named parameters.

    • params

      This can be either a JavaScript array (for positional parameters) or a JavaScript map. When the N1QL statement utilizes positional parameters (i.e., $1, $2 …​), then params is expected to be a JavaScript array corresponding to the values to be bound to these positional parameters. When the N1QL statement utilizes named parameters (i.e., $name), then params is expected to be a JavaScript map object providing the name-value pairs corresponding to the variables used by the N1QL statement. Positional and named value parameters cannot be mixed.

    • options

      This is a JSON object having various query runtime options as keys. Currently, the following settings are recognized:

      • consistency

        This controls the consistency level for the statement. Normally, this defaults to the consistency level specified in the overall handler settings but can be set on a per statement basis. The valid values are "none" and "request".

    • return value (handle)

      The call returns a JavaScript Iterable object representing the result set of the query. The query is streamed in batches as the iteration proceeds. The returned handle can be iterated using any standard JavaScript mechanism including for…​of loops.

      • close() Method on handle object (return value)

        This releases the resources held by the N1QL query. If the query is still streaming results, the query is cancelled.

    • Exceptions Thrown

      The N1QL() function throws an exception if the underlying N1QL query fails to parse or start executing. The returned Iterable handler throws an exception if the underlying N1QL query fails after starting. The close() method on the iterable handle can throw an exception if underlying N1QL query cancellation encounters an unexpected error.

    The crc64() function call:

    crc64(): This function calculates the CRC64 hash of an object using the ISO polynomial. The function takes one parameter, the object to checksum, and this can be any JavaScript object that can be encoded to JSON. The hash is returned as a string (because JavaScript numeric types offers only 53-bit precision). Note that the hash is sensitive to ordering of parameters in case of map objects.

    function OnUpdate(doc, meta) {
        var crc_str = crc64(doc);
        /// code here ...
    }

    The crc64 function can be useful in case like suppressing a duplicate mutation from the Sync Gateway (or SG), when both the Sync Gateway & Eventing are leveraging same bucket. Basically, Sync gateway would update metadata of the document within the bucket - which in turn generates an event for Eventing to process. Eventing can’t differentiate between events from Sync Gateway vs other events(doc updates via SDK, N1QL and others). A work around to this double mutation issue is possible via the crc64() function

    function OnUpdate(doc, meta) {
        // Ignore documents created by sync gateway
        if(meta.id.startsWith("_sync") == true) return;
    
        // Ignore documents whose body has not changed since we last saw it
        var prev_crc = checksum_bucket[meta.id];
        var curr_crc = crc64(doc);
        if (prev_crc === curr_crc) return;
        checksum_bucket[meta.id] = curr_crc;
    
       // Business logic goes in here
    }

    Handler Signatures

    Eventing Service or framework calls the following JavaScript functions as entry points to the handler.

    OnUpdate Handler:

    This handler gets called when a document is created or modified, e.g. Insert/Update. The handler listens to mutations (the creation or modification of documents) in the associated source Bucket.

    A sample OnUpdate handler is displayed below:

    function OnUpdate(doc, meta) {
      if (doc.type === 'order' && doc.value > 5000) {
        // ‘phonverify’ is a bucket alias that is specified as a Bucket binding.
        phoneverify[meta.id] = doc.customer;
      }
    }

    In this handler following limitations exist:

    • If a document is modified several times in a short duration, the calls may be coalesced into a single event due to deduplication.

    • It is not possible to distinguish between a Create and an Update operation.

    OnDelete Handler:

    This handler gets called when a document is deleted or removed due to an expiry. The handler listens to mutations (deletions or expirations) in the associated source Bucket.

    A sample OnDelete handler is displayed below:

    function OnDelete(meta) {
        var addr = meta.id;
        var res = SELECT id from orders WHERE shipaddr = $addr;
        for (var id of res) {
          log("Address invalidated for pending order: " + id);
        }
      }

    In this handler the following limitations exist:

    • It is not possible to distinguish if the document was removed is a Delete operation or removed as a result of a document expiration.

    • It is not possible to get the value of the document that was just deleted or expired.

    Reserved Words

    Reserved words are words that cannot be used in a handler as a variable name, function name, or as a property in the Function handler code. The following table lists the reserved words that you must refrain from using as they are used by the transpiler to integrate Couchbase’s query language, N1QL with Eventing.

    N1QL Keywords

    ALTER

    EXECUTE

    MERGE

    UPDATE

    BUILD

    EXPLAIN

    PREPARE

    UPSERT

    CREATE

    GRANT

    RENAME

    DELETE

    INFER

    REVOKE

    DROP

    INSERT

    SELECT

    What Happens If You Use a Reserved Word?

    Let’s say you try to create a new Function handler code using a reserved word for variable names, for function names, and as a property bindings value. All three cases generate a deployment error.

    Reserved words as a variable name:

    function get_numip_first_3_octets(ip) {
        var grant = 0;
        if (ip) {
            var parts = ip.split('.');
        }
    }

    Reserved words as a function name:

    function grant(ip) {
        var return_val = 0;
        if (ip) {
            var parts = ip.split('.');
        }
    }

    During the Function deployment step, when the system validates the handler code, it displays an error message such as the following:

    + Sample Error Message - Deployment failed: Syntax error (<line and column numbers>) - grant is a reserved name in N1QLJs

    Reserved words as a property bindings value

    reserved words 6 5