User Profile Sample: Couchbase Lite Data Sync

Introduction

Couchbase Sync Gateway is a key component of the Couchbase Mobile stack. It is an Internet-facing synchronization mechanism that securely syncs data across devices as well as between devices and the cloud. Couchbase Mobile 3.0 introduces centralized persistent module configuration of synchronization, which simplifies the administration of Sync Gateway clusters — see Sync Gateway Configuration

The core functions of the Sync Gateway include:

Data Synchronization across devices and the cloud
Authorization & Access Control
Data Validation

What You Will Learn

In this tutorial you will learn how to:

Setup a basic Couchbase Sync Gateway configuration to sync content between multiple Couchbase Lite enabled clients — see: Sync Gateway.
We will will cover the basics of the Sync Gateway Configuration
Configure your Sync Gateway to enforce data routing, access control and authorization — see: Sync Function.
We will cover the basics of the Sync Function API.
Configure your Couchbase Lite clients for replication with the Sync Gateway
Use "Live Queries" or Query events within your Couchbase Lite clients to be asynchronously notified of changes — see: Query Events / Live Queries

We will be using a Swift App as an example of a Couchbase Lite enabled client.

You can learn more about the Sync Gateway here in the Sync Gateway Documentation.

Prerequisites

This tutorial assumes familiarity with building Swift apps with Xcode, XCFrameworks and with Couchbase Lite.

If you are unfamiliar with the basics of Couchbase Lite, it is recommended that you walk through the following tutorials
- Fundamentals of using Couchbase Lite as a standalone database — see: Standalone tutorial.
- Using queries with a prebuilt version of Couchbase Lite database — see: Query tutorial.

iOS (Xcode 12/13)
Download the latest version from the Mac App Store

If you are on an older version of Xcode, which you need to retain for other development needs, make a copy of your existing version of Xcode and install the latest Xcode version. That way you can have multiple versions of Xcode on your Mac.

git (Optional)
This is required if you would prefer to pull the source code from GitHub repo.
- Create a free github account if you don’t already have one
- git can be downloaded from git-scm.org
curl HTTP client
You can use any HTTP client of your choice. But we will use curl in our tutorial. Download latest version from the curl website
Docker
We will be using Docker to run images of both Couchbase Server and the Sync Gateway — to download Docker, or for more information, see: Get Docker

System Overview

We will be working with the simple "User Profile" app which we introduced in the Standalone tutorial and extended in the Query tutorial.

In this tutorial, we will be extending that app to support data sync. It will do the following

Allows users to log in and create or update their user profile information. The user profile view is automatically updated every time the profile information changes in the underlying database
The user profile information is synced with a remote Sync Gateway which then syncs it to other devices (subject to access control and routing configurations specified in the sync function)

Figure 1. The sample user profile application running in a simulator

App Installation

Fetching App Source Code

Clone the sync branch of the User Profile Demo project from GitHub. Type the following command in your terminal

git clone -b sync https://github.com/couchbaselabs/userprofile-couchbase-mobile.git

Installing Couchbase Lite XCFramework

Next, we will download the Couchbase Lite 3.0 XCFramework.

The Couchbase Lite iOS XCFramework is distributed via SPM, CocoaPods, Carthage or you can download the pre-built XCFramework — see the Get Started - Install documentation for more information.

In our example, we will download the pre-built version of the XCFramework, using a script. To do this, type the following in a command terminal:

cd /path/to/UserProfileDemo/content/modules/userprofile/examples

sh install_tutorial.sh 3.0.0 (1)

1	Where 3.0.0 is the required Couchbase Lite release number.

Next, let’s verify the installation.

Try it Out

Open the UserProfileDemo.xcodeproj Xcode project file, located at
/path/to/UserProfileDemo/content/modules/userprofile/examples
```
open UserProfileDemo.xcodeproj
```
Use Xcode to build and run the project in two simulators
Verify that you see the login screen on both the simulators

Figure 2. User Profile Login Screen Image

Data Model

If you followed along with the Query tutorial, you can skip this section and proceed to the Backend Installation section. We have not changed the Data model for this tutorial.

Couchbase Lite is a JSON Document Store. A Document is a logical collection of named fields and values. The values are any valid JSON types. In addition to the standard JSON types, Couchbase Lite supports some special types like Date and Blob. While it is not required or enforced, it is a recommended practice to include a "type" property that can serve as a namespace for related.

The User Profile Document

The app deals with a single Document with a "type" property of "user" as shown in Example 1. The document ID is of the form "user::demo@example.com".

Example 1. A user profile document

{
    "type":"user",
    "name":"Jane Doe",
    "email":"jame.doe@earth.org",
    "address":"101 Main Street",
    "image":CBLBlob (image/jpg),
    "university":"Rensselaer Polytechnic"
}

The User Record

The "user" Document is encoded to a native struct named UserRecord as shown in Example 2

Example 2. The encoding of a UserRecord to a native structure

let kUserRecordDocumentType = "user"
typealias ExtendedData = [[String:Any]]
struct UserRecord : CustomStringConvertible{
    let type = kUserRecordDocumentType
    var name:String?
    var email:String?
    var address:String?
    var imageData:Data?
    var university:String?
    var extended:ExtendedData? // future
  
    var description: String {
        return "name = \(String(describing: name)), email = \(String(describing: email)), address = \(String(describing: address)), imageData = \(imageData)"
    }
    

}

The University Document

The app comes bundled with a collection of Documents of type "university". Each Document represents a university — see Example 3

Example 3. A university document

{
    "type":"university","web_pages": [
      "http://www.rpi.edu/"
    ],
    "name": "Rensselaer Polytechnic Institute",
    "alpha_two_code": "US",
    "state-province": null,
    "domains": [
      "rpi.edu"
    ],
    "country": "United States"
}

UniversityRecord

The "university" Document is encoded to a native struct named UniversityRecord — as shown in Example 4

Example 4. The encoding of a UniversityRecord to a native structure

typealias Universities = [UniversityRecord]
// Native object
struct UniversityRecord : CustomStringConvertible{
    
    var alphaTwoCode:String?
    var country:String?
    var domains:[String]?
    var name:String?
    var webPages:[String]?
    
    var description: String {
        return "name = \(String(describing: name)), country = \(String(describing: country)), domains = \(String(describing: domains)), webPages = \(webPages), alphaTwoCode = \(String(describing: alphaTwoCode)) "
    }
    
}

Backend Installation

We will install Couchbase Server and Sync Gateway using Docker.

Prerequisites

You must have Docker installed on your laptop. For more on Docker — see: Get Docker
On Windows, you may need admin privileges.
Ensure that you have sufficient memory and cores allocated to docker. At Least 3GB of RAM is recommended.

Docker Network

Create a docker network named “workshop”

docker network ls

docker network create -d bridge workshop

Couchbase Server

Install

We have a custom docker image priyacouch/couchbase-server-userprofile:7.0.0-dev of Couchbase Server, which creates an empty bucket named “userprofile” and an RBAC user “admin” with “sync gateway” role.

Alternatively, you can follow the instructions in our documentation — see: Get Started - Prepare, to install Couchbase Server and configure it with the relevant bucket.

Optionally, remove any existing Docker container
```
docker stop cb-server && docker rm cb-server
```

Start Couchbase Server in a Docker container

docker run -d --name cb-server \
--network workshop \
-p 8091-8094:8091-8094 -p 11210:11210 \
priyacouch/couchbase-server-userprofile:7.0.0-dev

Test Server Install

The server could take a few minutes to deploy and fully initialize; so be patient.

Check the Docker logs using the command:
```
docker logs -f cb-server
```
When the setup is completed, you should see output similar to that shown in Figure 3.

Figure 3. Server set-up output
Now check the required data is in place:
1. Open up http://localhost:8091 in a browser
2. Sign in as “Administrator” and “password” in login page
3. Go to “buckets” menu and confirm “userprofile” bucket is created
4. Go to “security” menu and confirm “admin” user is created

Sync Gateway

Now we will install, configure and run Sync Gateway.

Configuration

When using Sync Gateway 3.0, we can opt to provide a bootstrap configuration — see: Sync Gateway Configuration. We would then provision database, sync and other configuration using the Admin REST endpoints Alternatively, we can continue to run in legacy-mode, using the Pre-3.0 configuration.

In this tutorial — for the purposes of backward compatibility — we will run 3.x using its legacy configuration option. That is, we will be running with the disable_persistent_config option in the configuration file set to true. You can, if you wish, run a 2.8 version of Sync Gateway instead.

The configuration files corresponding to this sample application are shown in Table 1. They are available in the "sync" branch of the github repo hosting the app, which you cloned — look in:
/path/to/cloned/repo/userprofile-couchbase-mobile/content/modules/userprofile-sync/examples/

Table 1. Available configuration files
Release	Filename
3.x	sync-gateway-config-userprofile-demo-3-x-legacy.json
2.x	sync-gateway-config-userprofile-demo-2-x.json

Deploy

Let us configure and launch Sync Gateway in a Docker container.

Switch to the the folder containing the cloned configuration files, using:

cd /path/to/cloned/repo/userprofile-couchbase-mobile/content/modules/userprofile-sync/examples

Make sure no Sync Gateway container exists, using:

docker stop sync-gateway && docker rm sync-gateway

Launch Sync Gateway in a Docker container

You should see configuration files for the latest major version and the previous major version in this folder — see: Table 1. Choose an appropriate version.

For non-Windows Systems

Sync Gateway 3.0
Sync Gateway 2.x

Configuring and running Sync Gateway 3.0 in Docker using the configuration in sync-gateway-config-userprofile-demo-3-x-legacy.json.

Note the use of disable_persistent_config in the configuration file to force legacy configuration mode. ^[1]

docker run -p 4984-4986:4984-4986 \
--network workshop \
--name sync-gateway \
-d \
-v `pwd`/sync-gateway-config-userprofile-demo-3-x-legacy.json:\
/etc/sync_gateway/sync_gateway.json \
couchbase/sync-gateway:3.0-enterprise \
/etc/sync_gateway/sync_gateway.json

Configuring and running Sync Gateway 2.8 in Docker

docker run -p 4984-4986:4984-4986 \
--network workshop \
--name sync-gateway \
-d \
-v `pwd`/sync-gateway-config-userprofile-demo-2-x.json:\
/etc/sync_gateway/sync_gateway.json \
couchbase/sync-gateway:2.8.4-enterprise \
/etc/sync_gateway/sync_gateway.json

For Windows Systems

Sync Gateway 3.0
Sync Gateway 2.x

Configuring and running Sync Gateway 3.0 in legacy mode

docker run -p 4984-4986:4984-4986 ^
--network workshop ^
--name sync-gateway ^
-d -v %cd%sync-gateway-config-userprofile-demo-3-x-legacy.json:^
/etc/sync_gateway/sync_gateway.json ^
couchbase/sync-gateway:3.0.0-enterprise ^
/etc/sync_gateway/sync_gateway.json

Configuring and running Sync Gateway 2.8

docker run -p 4984-4986:4984-4986 ^
--network workshop ^
--name sync-gateway ^\
-d ^
-v %cd%/sync-gateway-config-userprofile-demo-2-x.json:^
etc/sync_gateway/sync_gateway.json ^
couchbase/sync-gateway:2.8.4-enterprise ^
/etc/sync_gateway/sync_gateway.json

Test the Installation

Now we can confirm that the Sync Gateway is up and running.

Check the log messages
```
docker logs -f sync-gateway
```
You will see a series of log messages. Make sure there are no errors.

Then open up http://localhost:4984 in browser.
You should see equivalent of the following message

{"couchdb":"Welcome","vendor":{"name":"Couchbase Sync Gateway","version":"3.0"},"version":"Couchbase Sync Gateway/3.0.0(145;e3f46be) EE"}

Now that we have the server and the sync gateway installed, we can verify data sync between Couchbase Lite enabled apps.

Sync Function

A key component of the sync process is the Sync Function and we will first look at how that can be set-up to control how data sync works.

The Sync Function is a Javascript-function that is specified as part of the Sync Gateway Configuration It handles Authorization , Data Validation, Data Routing and Access Control.

To get started learning about this function:

Open the your configuration file using a text editor of your choice. It will be located in the app bundle at
/path/to/cloned/repo/UserProfileDemo/content/modules/userprofile/examples.
Locate the sync setting in the file

Now you can follow along with the rest of the sections below.

Authorization

We use Basic Authentication in our application. The Id of the user making the request is specified in the Authorization header.

Locate the // Authorization section of the Sync Function. You will see that we are using the Sync function’s requireUser() API to verify that the email property specified in the Document matches the Id of the user making the request — see Example 5.

Example 5. Sync function — Authorization

function sync(doc, oldDoc) {

/* Authorization */

// Verify the user making the request is the same as the one in doc's email
requireUser(doc.email);


}

Data Validation

In the sync function we also do some basic validation of the contents of the Document — as shown in Example 6.

Example 6. Sync function — Data Validation

/* Data Validation */

// Validate the presence of email field.
// This is the "username" (1)
validateNotEmpty("email", doc.email);

// Validate that the document Id _id is prefixed by owner (2)
var expectedDocId = "user" + "::" + doc.email;

(3)
if (expectedDocId != doc._id) {
  // reject document
  throw({forbidden: "user doc Id must be of form user::email"});
}

1	Verify that the `email` property is not null. If it’s null, we throw a JS exception (see `validateNotEmpty()` function)
2	If this a new document, then verify that the `Id` of the Document is of the required format (i.e. "user::demo@example.com"). We throw an exception if that’s not the case.
3	If this is a document update, then verify that the `email` property value has not changed. Again, we throw an exception if that’s not the case.

You can learn more about the Sync Function in the documentation here: Sync Function API

Data Routing

Channels provide a mechanism to "tag" documents. They are typically used to route/segregate documents based on the contents of those documents — as shown in: Example 7.

When combined with the access() and requireAccess() API, the channel() API can also be used to enforce Access Control.

As we shall see in a later section, clients can use channels to pull just a subset of documents.

Example 7. Using channel() to tag/route documents

/* Routing */

// Add doc to the user's channel.
var username = getEmail(); (1)

var channelId = "channel."+ username; (2)

channel(channelId); (3)

1	Retrieve the the `email` property specified in the document. We will uses this as our user and channel name
2	Here we generate the channel name from the email property.
3	Here we route the document to the channel. The channel comes into existence the first time a document is added to it.

Access Control

We can enforce access control to channels using the access() API. The approach shown in Example 8 ensures that only users with access to a specific channel are able to retrieve documents in the channel.

Example 8. Controlling access to documents using channel() and access() API

// Give user access to document (1)
access(username, channelId);

1	Here we use the email property retrieved in Example 7 as the `username` and specify the channel the user is allowed to access

Starting Replication

Two-way Replication between the app and the Sync Gateway is enabled when the user logs into the app.

To see the code behind this, open the project’s DatabaseManager.swift file and locate the startPushAndPullReplicationForCurrentUser() function — as shown in: Example 9.

Example 9. The startPushAndPullReplicationForCurrentUser function

func startPushAndPullReplicationForCurrentUser() {

In the function you will see we create an instance of the ReplicatorConfig, which specifies the source and target database — see: Example 10. You could also use this to, optionally, override the default configuration settings.

Example 10. Creating the replicator configuration instance

  let dbUrl = remoteUrl.appendingPathComponent(kDBName)
  var config = ReplicatorConfiguration.init(database: db, target: URLEndpoint.init(url:dbUrl)) (1)

  config.replicatorType = .pushAndPull (2)
  config.continuous =  true (3)
  config.authenticator =  BasicAuthenticator(username: user, password: password) (4)
//  config.acceptOnlySelfSignedServerCertificate = true;


  // This should match what is specified in the sync gateway config
  // Only pull documents from this user's channel
  let userChannel = "channel.\(user)"
  config.channels = [userChannel] (5)

1	Initialize with `source` as the local Couchbase Lite database and the `remote` target as the Sync Gateway
2	Replication `type` of `pushAndPull` indicates that we require two-way sync. A value of `.pull` specifies that we only pull data from the Sync Gateway. A value of `.push` specifies that we only push data.
3	The `continuous` mode is specified to be true which means that changes are synced in real-time. A value of false which implies that data is only pulled from the Sync Gateway.
4	This is where you specify the authentication credentials of the user. In the Authorization section, we discussed that the Sync Gateway can enforce authorization check using the `requireUser` API.
5	The `channels` are used to specify the channels to pull from. Only documents belonging to the specified channels are synced. This is subject to Access Control rights enforced at the Sync Gateway. This means that if a client does not have access to documents in a channel, the documents will not be synched even if the client specifies it in the replicator configuration.

Now we initialize the Replicator with the ReplicatorConfiguration — as shown in Example 11

Example 11. Initialize a replicator with our configuration details

_pushPullRepl = Replicator.init(config: config)

In order to follow the replicator’s progress, we can attach a callback listener to it.

Attaching a callback listener to the Replicator means we will be asynchronously notified of state changes. This could be useful for instance, to inform the user of the progress of the replication. It is an optional step and is shown here in Example 12.

Example 12. Attaching a callback listener

_pushPullReplListener = _pushPullRepl?.addChangeListener({ (change) in
    let s = change.status
    switch s.activity {
    case .busy:
        print("Busy transferring data")
    case .connecting:
        print("Connecting to Sync Gateway")
    case .idle:
        print("Replicator in Idle state")
    case .offline:
        print("Replicator in offline state")
    case .stopped:
        print("Completed syncing documents")
    }
})

Now, with all that done, we can start the replicator — see: Example 13.

Example 13. Starting a configured replicator

_pushPullRepl?.start()

Stopping Replication

When user logs out of the app, the replication is stopped before the database is closed.

Open the DatabaseManager.swift file and locate the stopAllReplicationForCurrentUser() function.
```
func stopAllReplicationForCurrentUser() {
```

Stop the replicator and remove any associated change listeners

if let pushPullReplListener = _pushPullReplListener{
    print(#function)
    _pushPullRepl?.removeChangeListener(withToken:  pushPullReplListener)
    _pushPullRepl = nil
    _pushPullReplListener = nil
}
_pushPullRepl?.stop()

When you close a database, any active replicators, listeners and-or live queries are also be closed.

Query Events / Live Queries

Couchbase Lite applications can set up live queries in order to be asynchronously notified of changes to the database that affect the results of the query. This can be very useful, for instance, in keeping a UI View up-to-date with the results of a query.

In our app, the user profile view is kept up-to-date using a live query that fetches the user profile data used to populate the view. This means that, if the replicator pulls down changes to the user profile, they are automatically reflected in the view.

To see this:

Open the UserPresenter.swift file and locate the fetchRecordForCurrentUserWithLiveModeEnabled() function. Calling this function with a value of true implies that the caller wishes to be notified of any changes to query results.
```
func fetchRecordForCurrentUserWithLiveModeEnabled(__ enabled:Bool = false) {
```
Build the Query — see: Example 14 — using QueryBuilder API. If you are unfamiliar with this API, please check out this Query tutorial.
Example 14. Defining a query using QueryBuilder
guard let db = dbMgr.db else { fatalError("db is not initialized at this point!") } userQuery = QueryBuilder .select(SelectResult.all()) .from(DataSource.database(db)) .where(Meta.id.equalTo(Expression.string(self.userProfileDocId))) (1)
1 We query for documents based on document Id. In our app, there should be exactly one user profile document corresponding to this Id.

Attach a listener callback to the query to make it live

Example 15. Attaching a listener callback to a query

userQueryToken = userQuery?.addChangeListener { [weak self] (change) in (1)
    guard let `self` = self else {return}
    switch change.error {
    case nil:
        var userRecord = UserRecord.init() (2)
        userRecord.email = self.dbMgr.currentUserCredentials?.user
        
        for (_, row) in (change.results?.enumerated())! {
            // There should be only one user profile document for a user
            print(row.toDictionary())
            if let userVal = row.dictionary(forKey: "userprofile") { (3)
                userRecord.email  =  userVal.string(forKey: UserRecordDocumentKeys.email.rawValue)
                userRecord.address = userVal.string(forKey:UserRecordDocumentKeys.address.rawValue)
                userRecord.name =  userVal.string(forKey: UserRecordDocumentKeys.name.rawValue)
                userRecord.university = userVal.string(forKey: UserRecordDocumentKeys.university.rawValue)
                userRecord.imageData = userVal.blob(forKey:UserRecordDocumentKeys.image.rawValue)?.content (4)
            }
        }

1	Attach a listener callback to the query. Attaching a listener automatically makes it live so any time there is a change in the user profile data in the underlying database, the callback would be invoked
2	Create an instance of a UserRecord — see: The User Record. This will be populated with the query results.
3	The `SelectResult.all()` method is used to query all the properties of a document. In this case, the document in the result is embedded in a dictionary where the key is the database name, which is "userprofile". So we retrieve the `DictionaryObject` at key "userprofile".
4	We use appropriate type getters to retrieve values and populate the UserRecord instance

Exercises

Exercise 1

In this exercise, we will observe how changes made on one app are synced across to the other app

Run the app side by side in two simulators
Log into both the simulators with same userId and password.
Use the values "demo@example.com" and "password" for user Id and password fields respectively
On one simulator, enter values in the user’s name and address fields.
Confirm that changes show up in the app on the other simulator.
Similarly, make changes to the app in the other simulator and confirm that the changes are synced over to the first simulator.

Exercise 2

In this exercise, we will observe changes made via Sync Gateway are synced over to the apps

Make sure you complete Exercise 1. This is to ensure that you have the appropriate user profile document (with document Id of "user::demo@example.com") created through the app and synced over to the Sync Gateway.
Open the command terminal and issue the following command to get the user profile document using the GET Document REST API. We will be using curl to issue the request. If you haven’t done so, please install curl as indicated in the Prerequisites section
```
curl -X GET http://localhost:4984/userprofile/user::demo@example.com --user demo@example.com (1)
```
1 This GET retrieves the userprofile document with the id user::demo@example.com`

Your response should look something like the response below. The exact contents depends on the user profile information that you provided via your mobile app.

{
    "_attachments": { (1)
        "blob_1": {
            "content_type": "image/jpeg",
            "digest": "sha1-S8asPSgzA+F+fp8/2DdIy4K+0U8=",
            "length": 14989,
            "revpos": 2,
            "stub": true
        }
    },
    "_id": "user::demo@example.com",
    "_rev": "3-033fcbaf269d65a9247067be76d664f1111d033b", (2)
    "address": "",
    "email": "demo@example.com",
    "image": {
        "@type": "blob",
        "content_type": "image/jpeg",
        "digest": "sha1-S8asPSgzA+F+fp8/2DdIy4K+0U8=",
        "length": 14989
    },
    "name": "",
    "type": "user",
    "university": "British Institute in Paris, University of London"
}

If you had updated an image via the mobile app, you should see an "_attachments" property. This entry holds an array of attachments corresponding to each image blob entry added by the mobile app. This property is added by the Sync Gateway when it processes the document.
You can learn more about how image Blob types are mapped to attachments here in the Couchbase Lite documentation: Working with Blobs.

Record the revision Id of the document. You will need this when you update the document

In the command terminal, issue the following command to update the user profile document via PUT Document REST API

We chose to update the address field via the REST API. You can choose to update any other profile information if you like. You will be prompted to enter the users password when you submit the curl command.

curl --location --request PUT 'http://localhost:4984/userprofile/user::demo@example.com?rev=3-033fcbaf269d65a9247067be76d664f1111d033b' --user 'demo@example.com' \
--header 'Content-Type: application/json' \
--data-raw '{
    "address": "101 Main Street",
    "email": "demo@example.com",
    "image": {
        "@type": "blob",
        "content_type": "image/jpeg",
        "digest": "sha1-S8asPSgzA+F+fp8/2DdIy4K+0U8=",
        "length": 14989
    },
    "name": "",
    "type": "user",
    "university": "British Institute in Paris, University of London"
}'

Here, in the PUT, we specify the:

user id (user::demo@example.com)
revision Id (from the previous step 3-033fcbaf269d65a9247067be76d664f1111d033b) to select the item we want to update

Confirm that you get a HTTP "201 Created" status code
As soon as you update the document via the Sync Gateway REST API, confirm that the changes show up in the mobile app on the simulator.

Handling Conflicts during Data Synchronization

Data conflicts are inevitable in an environment where you can potentially have multiple writes updating the same data concurrently. Couchbase Mobile supports Automated Conflict Resolution.

You can learn more about automated conflict resolution in this blog Document Conflicts & Resolution .

Learn More

Congratulations on completing this tutorial!

This tutorial walked you through an example of how to use a Sync Gateway to synchronize data between Couchbase Lite enabled clients. We discussed how to configure your Sync Gateway to enforce relevant access control, authorization and data routing between Couchbase Lite enabled clients.

Check out the following links for further details