Purge a document automatically based on the document’s self-contained start and duration fields.
The advancedDocControlledSelfExpiry function:
-
Demonstrates the self-expiry of a document (for example, a user trial)
-
Requires Eventing Storage (or a metadata collection) and a source collection
-
Requires a binding of type
bucket alias -
Processes the initial mutation to calculate and set the TTL of a newly-created document
-
Uses a JavaScript data object to set document expiration
-
Operates on any document where
type == "trial_custoimers" -
Ignores any document with a TTL that is not zero
When you use a simple integer instead of a proper date or time object for your document’s expiration value, the expiration value is specified in one of the following ways:
-
As an offset from the current time if the value is less than 30 days (60 * 60 * 24 * 30 seconds).
-
As an absolute Unix time stamp if the value is greater than 30 days (60 * 60 * 24 * 30 seconds).
If a Bucket Max Time-to-Live is set and specified in seconds, it’s enforced as a hard upper limit.
Any subsequent document mutation, whether by SQL++, Eventing, or a Couchbase SDK, results in the document having its expiration adjusted and set to the bucket’s maximum TTL if the operation has:
-
No TTL
-
A TTL of zero
-
A TTL greater than the bucket’s TTL
-
advancedDocControlledSelfExpiry
-
Input data
-
Output data
There are two variants of this function available: a Couchbase Server version 6.6 that relies on SQL++, and a Couchbase Server version 6.6.1+/7.0.0+ that directly sets the expiration.
You can improve your function’s performance by avoiding N1QL() and using couchbase.replace(bucket_binding, meta, doc) instead.
The following example directly sets the expiration.
// Configure the settings for the advancedDocControlledSelfExpiry function as follows:
//
// Version 7.1+
// "Function Scope"
// *.* (or try bulk.data if non-privileged)
// Version 7.0+
// "Listen to Location"
// bulk.data.source
// "Eventing Storage"
// rr100.eventing.metadata
// Binding(s)
// 1. "binding type", "alias name...", "bucket.scope.collection", "Access"
// "bucket alias", "src_col", "bulk.data.source", "read and write"
//
// Version 6.6.1
// "Source Bucket"
// source
// "MetaData Bucket"
// metadata
// Binding(s)
// 1. "binding type", "alias name...", "bucket", "Access"
// "bucket alias", "src_col", "source", "read and write"
function OnUpdate(doc, meta) {
// Filter items that have not been updated
if (meta.expiration !== 0) {
log(meta.id, "IGNORE expiration "+meta.expiration+" !== 0 or "+
new Date(meta.expiration).toString());
return;
}
// Optional filter to a specific field like 'type'
if (doc.type !== 'trial_customers') return;
// The expiry is based on a JavaScript date parsable field
if (!doc.trialStartDate || !doc.trialDurationDays) return;
// Convert the doc field timeStamp to Unix epoch time in milliseconds
var docTimeStampMs = Date.parse(doc.trialStartDate);
var keepDocForMs = doc.trialDurationDays * 1000 * 60 * 60 * 24 ;
var nowMs = Date.now(); // Get current Unix time in milliseconds
// Archive if it has been kept for too long; you do not need to set an expiration
if( nowMs >= (docTimeStampMs + keepDocForMs) ) {
// Delete the document from the source collection through the map alias
delete src_col[meta.id];
log(meta.id, "DELETE from src_col to dst_bkt alias as our expiration " +
new Date(docTimeStampMs + keepDocForMs).toString()) + " is already past";
} else {
var key = meta.id;
// Set the meta.expiration=ttlMs
var ttlMs = docTimeStampMs + keepDocForMs;
if (ttlMs !== 0) {
log(meta.id, "UPDATE expiration "+meta.expiration+" === 0 set to "+
ttlMs+" or " + new Date(ttlMs).toString());
// Advanced Bucket Accessors use JavaScript Date objects
var expiryDate = new Date(ttlMs);
// This is 4X to 5X faster than using N1QL(...) and you do not need to worry about recursion
var res = couchbase.replace(src_col,{"id":meta.id,"expiry_date":expiryDate},doc);
if (!res.success) {
log(meta.id,'Setting TTL to',expiryDate,'failed',res);
}
}
}
}Create a test set of 4 documents using the Query Editor to insert the data items. You do not need an Index.
If today’s date is past 08-25-2021 (MM-DD-YYYY), you can change the trialStartDate for the last two records to at least 90 days from today.
UPSERT INTO `bulk`.`data`.`source` (KEY,VALUE)
VALUES ( "trial_customers::0", {
"type": "trial_customers",
"id": 0,
"trialStartDate": "08-25-2019",
"trialDurationDays": 30,
"note": "this is old will get immeadiately deleted"
} ),
VALUES ( "trial_customers::1",
{
"type": "trial_customers",
"id": 1,
"trialStartDate": "01-27-2020",
"trialDurationDays": 30,
"note": "this is old will get immeadiately deleted"
} ),
VALUES ( "trial_customers::2",
{
"type": "trial_customers",
"id": 2,
"trialStartDate": "08-25-2021",
"trialDurationDays": 30,
"note": "this will get an exiration set"
} ),
VALUES ( "trial_customers::3",
{
"type": "trial_customers",
"id": 3,
"trialStartDate": "08-26-2021",
"trialDurationDays": 60,
"note": "this will get an exiration set"
} );NEW/OUTPUT: KEY trial_customers::2
{
"id": 2,
"note": "this will get an exiration set",
"trialDurationDays": 30,
"trialStartDate": "08-25-2021",
"type": "trial_customers"
}
NEW/OUTPUT: KEY trial_customers::3
{
"id": 3,
"note": "this will get an exiration set",
"trialDurationDays": 60,
"trialStartDate": "08-26-2021",
"type": "trial_customers"
}
Returns 2 of the 4 documnents.
* "trial_customers::0" was deleted
* "trial_customers::1" was deleted
* "trial_customers::2" has an meta.expiration set for 1632466800 (or 2021-09-24 07:00:00 UTC) in it's metadata
* "trial_customers::3" has an meta.expiration set for 1635145200 (or 2021-10-25 07:00:00 UTC) in it's metadata