Function: Shipping Notifier

      +

      Goal: Send notifications when an order scheduled to arrive, when it is shipped, and when it is delivered.

      • This function shippingNotifier demonstrates a shipping workflow.

      • Requires Eventing Storage (or metadata collection), and "active", "archive", "notify" collections.

      • Requires four buckets metadata, active (the source), archive, and a notify.

        • The "notify"collection can be used to integrate with SDK or Kafka to send notifications.

      • Will operate on any doc with type === "ship".

      • Will update the source document with key information on each notify.

      • Delivered the shipping recored is archived.

      • On each notify data is read from the "active" bucket for type === "cust" and type === "order" as needed to build the notification.

      Other:

      • Note we could have used curl() to send the notify messages instead of using our staging bucket called "notify".

      • There are no try catch blocks and only limited error checking to highlight the functionality.

      • It is expected that the application that processes the "notify" bucket will purge the notification documents.

      • The notification history is stored in the shipping document and archived for all time.

      • shippingNotifier

      • Input Data/Mutation

      • Output Data/Mutation

      // To run configure the settings for this Function, shippingNotifier, as follows:
      //
      // Version 7.1+
      //   "Function Scope"
      //     *.* (or try bulk.data if non-privileged)
      // Version 7.0+
      //   "Listen to Location"
      //     bulk.data.active
      //   "Eventing Storage"
      //     rr100.eventing.metadata
      //   Binding(s)
      //    1. "binding type", "alias name...", "bucket.scope.collection", "Access"
      //       "bucket alias", "act_col",       "bulk.data.active",        "read and write"
      //       "bucket alias", "arc_col",       "bulk.data.archive",       "read and write"
      //       "bucket alias", "snd_col",       "bulk.data.notify",        "read and write"
      //
      // Version 6.X
      //   "Source Bucket"
      //     source
      //   "MetaData Bucket"
      //     metadata
      //   Binding(s)
      //    1. "binding type", "alias name...", "bucket",     "Access"
      //       "bucket alias", "act_col",       "active",     "read and write"
      //       "bucket alias", "arc_col",       "archive",    "read and write"
      //       "bucket alias", "snd_col",       "notify",     "read and write"
      
      function sendNotifySchedDelivCallback(context) {
          // This is a normal sceduled delivery notificaton
      
          // Look up the controlling ship: doc
          var shipkey = context.id;
          var shipdoc = act_col[shipkey];
          if (shipdoc === null) {
              // stale timer
              return;
          }
      
          // Make sure we are active and still need to send
          if (shipdoc.type != "ship" || !shipdoc.active || shipdoc.notifys[context.idx].notifySent) return;
      
          // Look up the realted order: doc
          var orderkey = "order" + ":" + shipdoc.id;
          var orderdoc = act_col[orderkey];
      
          // Look up the realted cusomer: doc
          var custkey = "cust" + ":" + orderdoc.cust_id;
          var custdoc = act_col[custkey];
          var notifyId = "ntfy" + ":" + context.idx + ":" + shipkey;
      
          // log('shipdoc',  shipdoc);
          // log('orderdoc', orderdoc);
          // log('custdoc',  custdoc);
          // log("notifyId",notifyId);
      
          var senddoc = {
              "notifyReason": context.item.notifyReason,
              "first_name": custdoc.first_name,
              "last_name": custdoc.last_name,
              "email": custdoc.email,
              "phone": custdoc.phone,
              "items": orderdoc.items,
              "utcOffset": shipdoc.utcOffset
          };
      
          // Add any special details
          if (context.item.notifyReason === "scheduled delivery") {
              senddoc["schedDelivTs"] = shipdoc.schedDelivTs;
          } else
          if (context.item.notifyReason === "delivered") {
              senddoc["deliveredTs"] = shipdoc.deliveredTs;
          } else
          if (context.item.notifyReason === "shipped") {
              senddoc["shippedTs"] = shipdoc.shippedTs;
          }
      
          // Write to send bucket -or- emit via cURL
          snd_col[notifyId] = senddoc;
      
          // Mark as sent
          shipdoc.notifys[context.idx].notifySent = true;
      
          // See if we are done and can archive this
          if (shipdoc.delivered && context.item.notifyReason === "delivered") {
              shipdoc.active = false;
              // Yes we can archive write to archive bucket
              arc_col[shipkey] = shipdoc;
              // and remove
              delete act_col[shipkey];
          } else {
              // No just update in the source bucket
              act_col[shipkey] = shipdoc;
          }
      
          log("senddoc", senddoc);
      }
      
      function OnUpdate(doc, meta) {
          // Filter out non interesting items
          if (doc.type != "ship" || !meta.id.startsWith("ship:") || !doc.active) return;
      
          var nowMs = Date.now();                // this instant or now in ms.
          var nowSec = Math.trunc(nowMs / 1000); // this instant or now in sec.
      
          if (doc.shipped || doc.delivered) {
              // these are events they do not need to be scheduled via a Timer
              if (doc.shipped) {
                  if (doc.shippedTs === null) {
                      doc.shippedTs = nowSec;
                  }
                  var item = {
                      "notifyReason": 'shipped',
                      "notifyTs": nowSec,
                      "notifySent": false
                  };
              }
              if (doc.delivered) {
                  if (doc.deliveredTs === null) {
                      doc.deliveredTs = nowSec;
                  }
                  var item = {
                      "notifyReason": 'delivered',
                      "notifyTs": nowSec,
                      "notifySent": false
                  };
              }
              // Add to the notification array or history
              doc.notifys.push(item);
              // Write the source doc since we will sending an immediate notification
              act_col[meta.id] = doc;
              var context = {
                  "item": item,
                  "idx": doc.notifys.length - 1,
                  "id": meta.id
              };
              // There no need for a timer we can do this now since it is an event
              sendNotifySchedDelivCallback(context);
              return;
          }
      
          // Look for any needed notifications in the future
          for (var idx = 0; idx < doc.notifys.length; idx++) {
              var item = doc.notifys[idx];
              if (!item.notifySent) {
      
                  // JavaScript works in ms. BUT the doc's fields are in sec. - so convert and make a Date()
                  var fireAt = new Date(item.notifyTs * 1000);
      
                  // Make unique ref for this notification can overwrite/adjust or cancel
                  var notifyId = "ntfy" + ":" + idx + ":" + meta.id;
      
                  // Pass minimal data in our context, the callback will look everything else up.
                  var context = {
                      "item": item,
                      "idx": idx,
                      "id": meta.id
                  };
      
                  // We will always 'overwrite' this timer(s) notification by the Timer's
                  // reference_id (6.6.0+ required for this) on every mutation
                  // log("create/overwrite notification "+ notifyId, item);
                  createTimer(sendNotifySchedDelivCallback, fireAt, notifyId, context);
              }
          }
      }

      We want to create a test set of three (3) documents. Use the Query Editor to insert the the data items (you do not need an index).

      For key "ship:dea0fca2-e7b7-11ea-adc1-0242ac120002", you may want to adjust the timestamps as the times are in seconds since Unix epoch. Use a tool like https://www.dcode.fr/timestamp-converter or https://www.epochconverter.com/ .
        UPSERT INTO `bulk`.`data`.`active` (KEY,VALUE)
        VALUES ( "order:dea0fca2-e7b7-11ea-adc1-0242ac120002", {
          "type": "order",
          "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
          "cust_id": 108998,
          "items": [
            {
                "sku": "SK18768",
                "descr": "Ticondorna pencils 12 pack",
                "qty": 3
            },
            {
                "sku": "SK89736",
                "descr": "Sharpie large marker",
                "qty": 1
            }
          ]
        }),
        VALUES ( "cust:108998", {
          "type": "cust",
          "id": 108998,
          "first_name": "John",
          "last_name":  "Smith",
          "email": "jon.smith@gmail.com",
          "addr1": "1010 E. 100th Ave.",
          "addr2": "Apt 101B",
          "city": "New York",
          "state": "NY",
          "zip": 10000,
          "phone": "+1 714-222-2222"
        }),
        VALUES ( "ship:dea0fca2-e7b7-11ea-adc1-0242ac120002", {
          "type": "ship",
          "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
          "utcOffset": -420,
          "orderTs": 1598214610,
          "schedDelivTs": 1598486400,
          "shippedTs": null,
          "deliveredTs": null,
          "notifys": [
            {
              "notifyTs": 1598450400,
              "notifyReason": "scheduled delivery",
              "notifySent": false
            }
          ],
          "exceptions": [],
          "shipped": false,
          "delivered": false,
          "active": true
        });

      To fully exercise the logic, run the following steps(to re-run flush the 'active', 'archive' and 'notify' collections and redo the UPSERT the data):

      • Deploy the Function with a Feed Boundary from "Everything".

        • Wait for about 7-14 seconds (timers are high volume not wall clock accurate) and notice collection "notify" has our first notification (the timer was scheduled in the past).

        • The shipping document will be modified in collection 'active' as follows:

          UPDATED/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
          {
            "active": true,
            "delivered": false,
            "deliveredTs": null,
            "exceptions": [],
            "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
            "notifys": [
              {
                "notifyReason": "scheduled delivery",
                "notifySent": true,
                "notifyTs": 1598450400
              }
            ],
            "orderTs": 1598214610,
            "schedDelivTs": 1598486400,
            "shipped": false,
            "shippedTs": null,
            "type": "ship",
            "utcOffset": -420
          }
        • You will now have the first notificaton document in collection 'notify' as follows:

          NEW/OUTPUT: KEY ntfy:0:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
          {
            "notifyReason": "scheduled delivery",
            "first_name": "John",
            "last_name": "Smith",
            "email": "jon.smith@gmail.com",
            "phone": "+1 714-222-2222",
            "items": [
              {
                "descr": "Ticondorna pencils 12 pack",
                "qty": 3,
                "sku": "SK18768"
              },
              {
                "descr": "Sharpie large marker",
                "qty": 1,
                "sku": "SK89736"
              }
            ],
            "utcOffset": -420,
            "schedDelivTs": 1598486400
          }
        • The application log for the Eventing handler will show something like the following:

          2021-07-18T21:17:51.715-07:00 [INFO] "senddoc" {"notifyReason":"scheduled delivery","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"schedDelivTs":1598486400}

      • In collection "active" mutate ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 by setting "shipped" to true.

        • The shiping document will be automatically modified in collection 'active' as follows:

          UPDATED/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
          {
            "active": true,
            "delivered": false,
            "deliveredTs": null,
            "exceptions": [],
            "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
            "notifys": [
              {
                "notifyReason": "scheduled delivery",
                "notifySent": true,
                "notifyTs": 1598450400
              },
              {
                "notifyReason": "shipped",
                "notifyTs": 1626668498,
                "notifySent": true
              }
            ],
            "orderTs": 1598214610,
            "schedDelivTs": 1598486400,
            "shipped": true,
            "shippedTs": 1626668498,
            "type": "ship",
            "utcOffset": -420
          }
        • You will now have the second notificaton document in collection 'notify' as follows:

          NEW/OUTPUT: KEY ntfy:1:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
          {
            "notifyReason": "shipped",
            "first_name": "John",
            "last_name": "Smith",
            "email": "jon.smith@gmail.com",
            "phone": "+1 714-222-2222",
            "items": [
              {
                "descr": "Ticondorna pencils 12 pack",
                "qty": 3,
                "sku": "SK18768"
              },
              {
                "descr": "Sharpie large marker",
                "qty": 1,
                "sku": "SK89736"
              }
            ],
            "utcOffset": -420,
            "shippedTs": 1626668498
          }
        • The Application log for the Eventing handler will show something like the following

          2021-07-18T21:21:38.547-07:00 [INFO] "senddoc" {"notifyReason":"shipped","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"shippedTs":1626668498}

      • In collection "active", mutate ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 again by setting "delivered" to true.

        • The shiping document will be removed from collections 'active' and archived to the collection 'archive' as follows:

          DELETE/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "active"
          
          NEW/OUTPUT: KEY ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "archive"
          {
            "active": false,
            "delivered": true,
            "deliveredTs": 1626668622,
            "exceptions": [],
            "id": "dea0fca2-e7b7-11ea-adc1-0242ac120002",
            "notifys": [
              {
                "notifyReason": "scheduled delivery",
                "notifySent": true,
                "notifyTs": 1598450400
              },
              {
                "notifyReason": "shipped",
                "notifyTs": 1626668498,
                "notifySent": true
              },
              {
                "notifyReason": "delivered",
                "notifyTs": 1626668622,
                "notifySent": true
              }
            ],
            "orderTs": 1598214610,
            "schedDelivTs": 1598486400,
            "shipped": true,
            "shippedTs": 1626668498,
            "type": "ship",
            "utcOffset": -420
          }
        • You will now have the third and final notificaton document in collection 'notify' as follows:

          NEW/OUTPUT: KEY ntfy:2:ship:dea0fca2-e7b7-11ea-adc1-0242ac120002 in collection "notify"
          {
            "notifyReason": "delivered",
            "first_name": "John",
            "last_name": "Smith",
            "email": "jon.smith@gmail.com",
            "phone": "+1 714-222-2222",
            "items": [
              {
                "descr": "Ticondorna pencils 12 pack",
                "qty": 3,
                "sku": "SK18768"
              },
              {
                "descr": "Sharpie large marker",
                "qty": 1,
                "sku": "SK89736"
              }
            ],
            "utcOffset": -420,
            "deliveredTs": 1626668622
          }
        • The Application log for the Eventing handler will show something like the following

          2021-07-18T21:23:42.248-07:00 [INFO] "senddoc" {"notifyReason":"delivered","first_name":"John","last_name":"Smith","email":"jon.smith@gmail.com","phone":"+1 714-222-2222","items":[{"descr":"Ticondorna pencils 12 pack","qty":3,"sku":"SK18768"},{"descr":"Sharpie large marker","qty":1,"sku":"SK89736"}],"utcOffset":-420,"deliveredTs":1626668622}

      Note that with respect to the notifications that were created:

      • index 0 created a Timer that was fired immediately as it used a timer and was in the past.

        notifyTs = 2020-08-26T14:00:00.000Z or Wed Aug 26 2020 07:00:00 GMT-0700 (Pacific Daylight Time)

      • index 1 was an event e.g. shipped was mutated to true (it didn’t need a Timer) and fired instantly.

        shippedTs = 2021-07-19T04:21:38.000Z or Sun Jul 18 2021 21:21:38 GMT-0700 (Pacific Daylight Time)

      • index 2 was an event e.g. delivered was mutated to true (it didn’t need a Timer) and fired instantly.

        deliveredTs = 2021-07-19T04:23:42.000Z or Sun Jul 18 2021 21:23:42 GMT-0700 (Pacific Daylight Time)