A newer version of this documentation is available.

View Latest

Document Expiry and Archival

    +

    Goal: When a document in an existing bucket is about to expire, a new document is created in a newly created bucket.

    Implementation: Write an OnUpdate handler, which runs whenever a document is created or mutated. The handler calls a timer routine, which executes a callback function, two minutes prior to any document’s established expiration. This function retrieves a specified value from the document, and stores it in a document of the same name, in a specified target bucket. The original document in the source bucket is not changed..

    For this example, the buckets created such as source, target, and metadata buckets, are used. A new document is created within the source bucket, and this document has its expiration — or Time To Live (TTL) — set to occur ten minutes after the document’s creation.

    Python script for this Example is provided for reference. Using the Couchbase SDK, you can create or modify the document expiration. In this example, the Couchbase SDK Python client creates a document and sets the document’s expiration.

    from couchbase.cluster import Cluster
    from couchbase.cluster import PasswordAuthenticator
    import time
    cluster = Cluster('couchbase://localhost:8091')
    authenticator = PasswordAuthenticator('Administrator', 'password')
    cluster.authenticate(authenticator)
    
    cb = cluster.open_bucket('source')
    cb.upsert('SampleDocument2', {'a_key': 'a_value'})
    cb.touch('SampleDocument2', ttl=10*60)

    The script imports a Couchbase cluster object, and authenticates against it, using (for demonstration purposes) the Full Administrator username and password (the cluster is assumed to be accessible on localhost). The script then opens the existing source bucket, and inserts a new document, named SampleDocument2, whose body is {'a_key': 'a_value'}.

    For information on installing the Couchbase Python SDK, refer to Start Using the SDK. For information on using the Couchbase Python SDK to establish bucket-expiration, refer to Document Operations.

    Procedure

    Proceed as follows:

    1. Install the Couchbase SDK Python client and from the appropriate folder, start Python.

      ./python
    2. On the Python prompt, enter the provided code.

      >>> from couchbase.cluster import Cluster
      >>> from couchbase.cluster import PasswordAuthenticator
      >>> import time
      >>> cluster = Cluster('couchbase://localhost:8091')
      >>> authenticator = PasswordAuthenticator('Administrator', 'password')
      >>> cluster.authenticate(authenticator)
      >>> cb = cluster.open_bucket('source')
      >>> cb.upsert('SampleDocument2', {'a_key': 'a_value'})
      OperationResult<rc=0x0, key='SampleDocument2', cas=0x1519ec8cdee90000>
      >>> cb.touch('SampleDocument2', ttl=10*60)
      OperationResult<rc=0x0, key='SampleDocument2', cas=0x1519ec8e686c0000>
      >>>
    3. To verify bucket creation, access the Buckets screen from the Couchbase Web Console and click the Document tab of the Source bucket. The new document gets displayed.

    4. [Optional Step] Click on a document’s id to view the metadata information.

    5. From the Couchbase Web Console > Eventing page, click ADD FUNCTION, to add a new Function. The ADD FUNCTION dialog appears.

    6. In the ADD FUNCTION dialog, for individual Function elements provide the below information:

      • For the Source Bucket drop-down, select source.

      • For the Metadata Bucket drop-down, select metadata.

      • Enter add_timer_before_expiry as the name of the Function you are creating in the FunctionName text-box.

      • Enter text Function that adds timer before document expiry, in the Description text-box.

      • For the Settings option, use the default values.

      • For the Bindings option, add two bindings. For the first binding specify source as the name of the bucket, and specify src as the associated value. For the second binding, specify target as the name of the bucket, and specify tgt as the associated value.

    7. After providing all the required information in the ADD FUNCTION dialog, click Next: Add Code. The add_timer_before_expiry dialog appears.

    8. The add_timer_before_expiry dialog initially contains a placeholder code block. You will substitute your actual add_timer_before_expiry code in this block.

      docexpiry 02 editor with default 60
    9. Copy the following Function, and paste it in the placeholder code block of add_timer_before_expiry dialog.

      function OnUpdate(doc, meta) {
          // Only process for those documents that have a non-zero TTL
          if (meta.expiration == 0 ) return;
          // Get the TTL and compute 2 minutes prior to the TTL, note JavaScript Date() takes msec.
          var twoMinsPrior = new Date((meta.expiration - 2*60) * 1000);
          // Create a context and then create a timer with our context
          var context = { docID : meta.id, expiration : meta.expiration };
          createTimer(DocTimerCallback, twoMinsPrior , meta.id, context);
          log('OnUpdate add Timer 2 min. prior to TTL to DocId:',  meta.id);
      }
      function DocTimerCallback(context) {
          log('DocTimerCallback 1 on DocId:', String(context.docID));
          // create a new document with the same ID but in the target bucket
          tgt[context.docID] = "To Be Expired in 2 min., Key's Value is:" + JSON.stringify(src[context.docID]);
          log('DocTimerCallback 2 src expiry:', new Date(context.expiration  * 1000));
          log('DocTimerCallback 3 tgt archive via Key:', String(context.docID));
      }

      After pasting, the screen appears as displayed below:

      docexpiry 03 editor with code 60
    10. Click Save.

    11. To return to the Eventing screen, click Eventing tab.

    12. From the Eventing screen, click Deploy.

    13. In the Confirm Deploy Function dialog, select Everything from the Feed boundary option.

    14. Click Deploy. The function is deployed and starts running within a few seconds.

      docexpiry 04 buckets

      As a result, a new document — like the original, named SourceDocument2 — is created, with a value based on that of the original. After two minutes has elapsed, check the documents within the source bucket: the original SourceDocument2 is no longer visible, having been removed at its defined expiration-time.