Mobile-Server Data Sync

Shared Bucket Access

Sync Gateway uses the Shared Bucket Access feature to synchronize document changes made to Couchbase Server bucket through the Couchbase SDKs or through the Couchbase Server Admin UI to Couchbase Lite clients. Changes made on Couchbase Lite clients are synchronized to the server. The following diagram shows different clients (mobile, web, desktop) writing data to the same bucket in Couchbase Server.

Prior to Sync Gateway 1.5, you either had to ensure that all writes happened through the Sync Gateway REST API, or had to set up bucket shadowing to ensure that the security and replication metadata needed by mobile applications was preserved.

Bucket shadowing is deprecated and will be removed in a future release. So applications using bucket shadowing must start migrating to using the shared-bucket-access capability

With "shared-bucket-access" capability, the sync metadata created by the Sync Gateway is abstracted from applications reading and writing data directly to Couchbase Server. Sync Gateway utilizes a feature of Couchbase Server 5.0 called XATTRs (eXtended ATTRibutes) to store that metadata into an external document fragment instead of inline within the document body.

How to enable it

This new feature was made opt-in primarily out of consideration for existing customers upgrading from Sync Gateway 1.4. It ensures that their existing configs will continue to work as-is, and supports upgrade without bringing down the entire Sync Gateway cluster.

The getting started page includes step-by-step instructions to:

Install Couchbase Server 5.0 or above.
Configure Couchbase Server with RBAC.

Install Sync Gateway with a configuration file that enables Mobile-Web data synchronization. Another example is presented below.

{
    "databases": {
        "db": {
            "bucket": "my-bucket",
            "username": "my-user",
            "password": "my-password",
            "server": "http://localhost:8091",
            "enable_shared_bucket_access": true, (1)
            "import_docs": true (2)
        }
    }
}

1	The `enable_shared_bucket_access` property is used to enable Web-Server data sync and must be `true` on all nodes participating in such a configuration.
2	The `import_docs` property is used to specify that a Sync Gateway node should (only) import document changes using the Import process. But note that it can only be set to `true` on a single node.

From then on, documents can be inserted on the Server directly (with N1QL or SDKs) or through the Sync Gateway REST API. The mobile metadata is no longer kept in the document, but in a system extended attribute in Couchbase Server.

Import process

The import process is a key part of mobile convergence. It is the means by which Sync Gateway becomes aware of non-Sync Gateway data changes and obtains the mobile metadata it requires to replicate changes. Any non-Sync Gateway change is eligible for import.

By default import processing is disabled in version 2.6 and earlier and should only be enabled on one node

The document is first run through the Sync Function to compute read security and routing, with the following differences:

The import is processed with an admin user context in the Sync Function, similar to writes made through the Sync Gateway Admin API. This means that requireAccess, requireUser and requireRole calls in the Sync Function are treated as no-ops.
During import, oldDoc is nil when the Sync Function is executed.

Only one node in a Sync Gateway cluster should be configured for feed-based import - it processes the full DCP stream.

You can specify a filter function using the $dbname.import_filter property, which will only import specific documents.

Use logging.console.log keys to troubleshoot import processing issues in the logs.

Reference

The reference to the configuration properties can be found below.

$dbname.enable_shared_bucket_access to enable convergence for a given database.
$dbname.import_docs to give a particular Sync Gateway node the role of importing the documents.
$dbname.import_filter to select which document(s) to make aware to mobile clients.

Metadata Purge Interval

When $dbname.enable_shared_bucket_access is enabled, mobile tombstones are not retained indefinitely. They are purged based on the server’s metadata purge interval. To ensure tombstones are replicated to clients, you should set the server’s metadata purge interval based on your expected replication frequency (see the $dbname.enable_shared_bucket_access reference).

FAQ

How do I query the document’s sync metadata?

Starting in Couchbase Server 5.5, the N1QL query language supports the ability to select extended attributes (xattrs) where the document’s sync metadata is stored. The following query shows an example of that feature.

SELECT meta().xattrs._sync FROM `travel-sample` WHERE meta().id = "mydocId"

Prior to 5.5, there is no way to query the mobile metadata with shared_bucket_access_enabled: true.

The sync metadata is maintained internally by Sync Gateway and its structure can change at any time. It should not be used to drive business logic of applications. The direct use of the N1QL query is unsupported and must not be used in production environments.

How do I access a blob stored in Couchbase Lite?

See the Couchbase Lite documentation on Blobs.

How do I access an attachment from a WebApp?

Attachments can be accessed through Sync Gateway’s REST API using the /{db}/{doc}/{attachment} endpoint.

Migrating from Bucket Shadowing

As of Sync Gateway 1.5, the Bucket Shadowing feature is deprecated and no longer supported. The following steps outline a recommended method for migrating from Bucket Shadowing to the latest version with interoperability between Couchbase Server SDKs and Couchbase Mobile.

Follow the recommendations in the Couchbase Server documentation to upgrade all instances to 5.0.
Create a new bucket on Couchbase Server (bucket 2).
Install Sync Gateway 1.5 on a separate node with shared access enabled and connect it to the new bucket (bucket 2).
Setup a push replication from the Sync Gateway instance used for Bucket Shadowing to the Sync Gateway 1.5 instance.
Once the replication has completed, test your application is performing as expected.
Update the load balancer to direct incoming traffic to the Sync Gateway 1.5 instance when you are ready to upgrade.
Delete the first bucket (bucket 1).

bucket shadowing migration