Function: Document Controlled Expiry

    +

    Goal: Purge a document automatically based on self-contained start and duration fields.

    • This function docControlledSelfExpiry demonstrates self-expiry of a document for example a user trial.

    • Requires a metadata bucket and a source bucket.

    • The source bucket must be named "source" and the Function needs a bucket alias of "src_bkt" in mode read+write.

    • When documents are created, they will have no expiration value. This function processes the initial mutation to calculate and set the proper TTL.

    • In Couchbase, when using a simple integer expiry value (as opposed to a proper date or time object), the expiration can be specified in two ways:

      • As an offset from the current time. If the absolute value of the expiry is less than 30 days (60 * 60 * 24 * 30 seconds), it is considered an offset.

      • As an absolute Unix time stamp. If the value is greater than 30 days (60 * 60 * 24 * 30 seconds), it is considered an absolute time stamp.

    • Will operate on any document with type == "trial_customers".

    • Will ignore any doc with a non-zero TTL (prevents infinite recursion)

    • Uses the N1QL(…​) function to update the source bucket instead of an inline N1QL statement because inline N1QL is prohibited from updating the source bucket of an Eventing handler to prevent infinite recursion scenarios.

    • The recursion from the N1QL(…​) statement is ignored via the if (meta.expiration !== 0) { …​ } filter.

    • This is different than setting a TTL on a bucket which will typically update (or extend) the TTL of a document on each mutation.

    You must use the function N1QL(…​) with great caution when updating the source bucket of your Eventing handler as you can easily create infinite recursion which may crash your server.
    • docControlledSelfExpiry

    • Input Data/Mutation

    • Output Data/Mutation

    function OnUpdate(doc, meta) {
        // Filter items that don't have been updated, this also stops
        // any recursion when we update meta.expiration via N1QL(...)
        if (meta.expiration !== 0) {
            log(meta.id, "IGNORE expiration "+meta.expiration+" !== 0 or "+
                new Date(meta.expiration).toString());
            return;
        }
    
        // Optional filter to a specic field like 'type'
        if (doc.type !== 'trial_customers') return;
    
        // Our expiry is based on a JavaScript date parsable field, it must exist
        if (!doc.trialStartDate || !doc.trialDurationDays) return;
    
        // Convert the doc's field timeStamp and convert to unix epoch time (in ms.).
        var docTimeStampMs = Date.parse(doc.trialStartDate);
    
        var keepDocForMs = doc.trialDurationDays * 1000 * 60 * 60 * 24 ;
        var nowMs = Date.now();  // get current unix time (in ms.).
    
        // Archive if we have kept it for too long no need to set the expiration
        if( nowMs >= (docTimeStampMs + keepDocForMs) ) {
    
            // Delete the document form the source bucket via the map alias
            delete src_bkt[meta.id];
    
            log(meta.id, "DELETE from src_bkt to dst_bkt alias as our expiration " +
                new Date(docTimeStampMs + keepDocForMs).toString()) + " is already past";
        } else {
            var key = meta.id;
            //set the meta.expiration=ttlMs
            var	ttlMs = docTimeStampMs + keepDocForMs;
    
            // Use N1QL to write back a non-zero TTL to the document hear we actually
            // have to use the real bucket name "source" instead of the alias src_bkt
            // as we are using N1QL. This will cause recursion but it will be ignored
            // since we ignore all non-zero TTLs
    
            if (ttlMs !== 0) {
                log(meta.id, "UPDATE expiration "+meta.expiration+" === 0 set to "+
                    ttlMs+" or " + new Date(ttlMs).toString());
    
                // Ensure non-zero, just be safe just in case somehow 1) doc.timeStamp
                // evals to 0, and 2) keepDocForMs is set to 0
                var stmt = "UPDATE `source` USE KEYS \""+key+
                           "\" SET meta().expiration = " + Math.floor(ttlMs/1000);
                N1QL(stmt);
    
                // Future in 6.6.1+ we can avoid N1QL via Eventing's new Advanced Bucket Ops
                // couchbase.replace(src,{"id":meta.id,"expiration":Math.floor(ttlMs/1000)},doc);
            }
        }
    }

    We want to create a test set of four (4) documents, use the Query Editor to insert the the data items (you do not need an index).

    Note, if the today is past 2020-09-24 just change the trialStartDate for the last two records to at least 90 days from now.

      INSERT INTO `source` (KEY,VALUE)
      VALUES ( "trial_customers::0", {
        "type": "trial_customers",
        "id": 0,
        "trialStartDate": "08-25-2019",
        "trialDurationDays": 30,
        "note": "this is old will get immeadiately deleted"
      } ),
      VALUES ( "trial_customers::1",
      {
        "type": "trial_customers",
        "id": 1,
        "trialStartDate": "01-27-2020",
        "trialDurationDays": 30,
        "note": "this is old will get immeadiately deleted"
      } ),
      VALUES ( "trial_customers::2",
      {
        "type": "trial_customers",
        "id": 2,
        "trialStartDate": "08-25-2020",
        "trialDurationDays": 30,
        "note": "this will get an exiration set"
      } ),
      VALUES ( "trial_customers::3",
      {
        "type": "trial_customers",
        "id": 3,
        "trialStartDate": "08-26-2020",
        "trialDurationDays": 60,
        "note": "this will get an exiration set"
      } );
    NEW/OUTPUT: KEY trial_customers::2
    
    {
      "id": 2,
      "note": "this will get an exiration set",
      "trialDurationDays": 30,
      "trialStartDate": "08-25-2020",
      "type": "trial_customers"
    }
    
    NEW/OUTPUT: KEY trial_customers::3
    
    {
      "id": 3,
      "note": "this will get an exiration set",
      "trialDurationDays": 60,
      "trialStartDate": "08-26-2020",
      "type": "trial_customers"
    }
    
    We end up with two (2) of the four documents (obviously you may need to adjust the N1QL INSERT in a few months as all the document would be immediately deleted).
    
    * "trial_customers::0" was deleted
    * "trial_customers::1" was deleted
    * "trial_customers::2" has an meta.expiration set for 1600930800 (or 2020-09-24 07:00:00 UTC) in it's metadata
    * "trial_customers::3" has an meta.expiration set for 1603609200 (or 2020-10-25 07:00:00 UTC) in it's metadata