Function: Advanced Document Controlled Expiry

    +

    Goal: Purge a document automatically based on self-contained start and duration fields.

    • This function advancedDocControlledSelfExpiry demonstrates self-expiry of a document for example a user trial.

    • Requires a metadata bucket and a source bucket.

    • The source bucket must be named "source" and the Function needs a bucket alias of "src_bkt" in mode read+write.

    • When documents are created, they will have no expiration value. This function processes the initial mutation to calculate and set the proper TTL.

    • As we are using Advanced Bucket Accessors setting document expirations (or TTLs) we use a JavaScript Data object.

    • Will operate on any document with type == "trial_customers".

    • Will ignore any doc with a non-zero TTL

    • This is different than setting a TTL on a bucket which will typically update (or extend) the TTL of a document on each mutation.

    • advancedDocControlledSelfExpiry

    • Input Data/Mutation

    • Output Data/Mutation

    Two variants of this function are available - a 6.6 version that relies on N1QL and a 6.6.1 version (this Function) that directly sets the expiration. You can completely avoid N1QL(…​) and use couchbase.replace(bucket_binding, meta, doc) as the advancedDocControlledSelfExpiry variant is much faster.

    advancedDocControlledSelfExpiry (direct TTL) Couchbase Server 6.6.1

    function OnUpdate(doc, meta) {
        // Filter items that don't have been updated
        if (meta.expiration !== 0) {
            log(meta.id, "IGNORE expiration "+meta.expiration+" !== 0 or "+
                new Date(meta.expiration).toString());
            return;
        }
    
        // Optional filter to a specic field like 'type'
        if (doc.type !== 'trial_customers') return;
    
        // Our expiry is based on a JavaScript date parsable field, it must exist
        if (!doc.trialStartDate || !doc.trialDurationDays) return;
    
        // Convert the doc's field timeStamp and convert to unix epoch time (in ms.).
        var docTimeStampMs = Date.parse(doc.trialStartDate);
    
        var keepDocForMs = doc.trialDurationDays * 1000 * 60 * 60 * 24 ;
        var nowMs = Date.now();  // get current unix time (in ms.).
    
        // Archive if we have kept it for too long no need to set the expiration
        if( nowMs >= (docTimeStampMs + keepDocForMs) ) {
    
            // Delete the document form the source bucket via the map alias
            delete src_bkt[meta.id];
    
            log(meta.id, "DELETE from src_bkt to dst_bkt alias as our expiration " +
                new Date(docTimeStampMs + keepDocForMs).toString()) + " is already past";
        } else {
            var key = meta.id;
            //set the meta.expiration=ttlMs
            var	ttlMs = docTimeStampMs + keepDocForMs;
    
            if (ttlMs !== 0) {
                log(meta.id, "UPDATE expiration "+meta.expiration+" === 0 set to "+
                    ttlMs+" or " + new Date(ttlMs).toString());
                // Advanced Bucket Accessors use JavaScript Date objects
                var expiryDate = new Date(ttlMs);
                // This is 4X to 5X faster than using N1QL(...) and no need to worry about recursion.
                var res = couchbase.replace(src_bkt,{"id":meta.id,"expiry_date":expiryDate},doc);
                if (!res.success) {
                    log(meta.id,'Setting TTL to',expiryDate,'failed',res);
                }
            }
        }
    }

    We want to create a test set of four (4) documents, use the Query Editor to insert the the data items (you do not need an index).

    Note, if the today is past 01-06-2020 (MM-DD-YYYY) just change the trialStartDate for the last two records to at least 90 days from now.

      UPSERT INTO `source` (KEY,VALUE)
      VALUES ( "trial_customers::0", {
        "type": "trial_customers",
        "id": 0,
        "trialStartDate": "08-25-2019",
        "trialDurationDays": 30,
        "note": "this is old will get immeadiately deleted"
      } ),
      VALUES ( "trial_customers::1",
      {
        "type": "trial_customers",
        "id": 1,
        "trialStartDate": "01-27-2020",
        "trialDurationDays": 30,
        "note": "this is old will get immeadiately deleted"
      } ),
      VALUES ( "trial_customers::2",
      {
        "type": "trial_customers",
        "id": 2,
        "trialStartDate": "08-25-2021",
        "trialDurationDays": 30,
        "note": "this will get an exiration set"
      } ),
      VALUES ( "trial_customers::3",
      {
        "type": "trial_customers",
        "id": 3,
        "trialStartDate": "08-26-2021",
        "trialDurationDays": 60,
        "note": "this will get an exiration set"
      } );
    NEW/OUTPUT: KEY trial_customers::2
    
    {
      "id": 2,
      "note": "this will get an exiration set",
      "trialDurationDays": 30,
      "trialStartDate": "08-25-2021",
      "type": "trial_customers"
    }
    
    NEW/OUTPUT: KEY trial_customers::3
    
    {
      "id": 3,
      "note": "this will get an exiration set",
      "trialDurationDays": 60,
      "trialStartDate": "08-26-2021",
      "type": "trial_customers"
    }
    
    We end up with two (2) of the four documents (obviously you may need to adjust the N1QL INSERT in a few months as all the document would be immediately deleted).
    
    * "trial_customers::0" was deleted
    * "trial_customers::1" was deleted
    * "trial_customers::2" has an meta.expiration set for 1632466800 (or 2021-09-24 07:00:00 UTC) in it's metadata
    * "trial_customers::3" has an meta.expiration set for 1635145200 (or 2021-10-25 07:00:00 UTC) in it's metadata