Getting Started with Peer-to-Peer Sync on Xamarin (UWP, iOS, and Android)
This tutorial uses a simple inventory tracker app to demonstrate Couchbase Lite’s peer-to-peer database sync functionality.
Introduction
Couchbase Lite [1] provides out-of-the-box support for secure Peer-to-Peer Sync, over websockets. The sync, between Couchbase Lite enabled clients in IP-based networks, does not require a centralized control point. You do not need a Sync Gateway or Couchbase Server to get peer-to-peer database sync going.
You need to add the UWP app to the Exception list on the Windows Firewall. |
Throughout this tutorial, these terms are used interchangeably:
-
"passive peer", "server" and "listener" all refer to the peer on which the websocket listener is started
-
"active peer" and "client" both refer to the peer on which the replicator is initialized.
You can learn more about Couchbase Lite here
Prerequisites
This tutorial assumes familiarity with building Xamarin apps with Visual Studio and with Couchbase Lite.
-
Visual Studio 2019 (Download it from the Microsoft Website) with:
-
Universal Windows Platform component installed
-
Xamarin component installed (for Android and iOS development).
-
-
If you are unfamiliar with the basics of Couchbase Lite, it is recommended that you follow the Getting Started guides
-
Wi-Fi network that the peers can communicate over
You could run your peers in multiple simulators. But if you were running the app on real devices, you will need to ensure that the devices are on the same Wi-Fi network.
App Overview
The app uses a local database that is pre-populated with data. There is no Sync Gateway or Couchbase Server installed.
When used as a passive peer:
-
Users log in and start websockets listener for the couchbase lite database.
The Listener IP endpoint is advertised over UDP type socket when users enter theListenerPage
. -
View the status of connected clients
-
Directly sync data with connected clients.
When used as an active peer, users can:
-
Log in and enter the
ListenersBrowserPage
to start browsing for peers -
Connect to a listener
-
Directly sync data with connected clients.
Exploring the App Project
-
The Xamarin .NET project comes pre-bundled with some resource files that we will examine here.
-
userdb.cblite2.zip
:
A zip file containing a prebuilt Couchbase Lite database. It includes the data for a single document. See Data Model -
userallowlist.json
:
List of valid client users (and passwords) in the system. This list is looked up when the server tries to authenticate credentials associated with incoming connection request. -
listener-cert-pkey.p12
:
This is PKCS12 file archive that includes a public key cert corresponding to the listener and associated private key. The cert is a sample cert that was generated using OpenSSL tool. -
listener-pinned-cert.cer
:
This is the public key listener cert (the same cert that is embedded in thelistener-cert-pkey.p12
file) in DER encoded format. This cert is pinned on the client replicator and is used for validating server cert during connection setup.
Data Model
Couchbase Lite is a JSON Document Store.
A Document is a logical collection of named fields and values.
The values are any valid JSON types.
In addition to the standard JSON types, Couchbase Lite supports some special types like Date
and Blob
.
While it is not required or enforced, it is a recommended practice to include a "type" property that can serve as a namespace for related.
The "List" Document
The app deals with a single Document with a "type" property of "list".
An example of a document would be
{
"type":"list",
"list":[
{
"image":{"length":16608,"digest":"sha1-LEFKeUfywGIjASSBa0l/cg5rlm8=","content_type":"image/jpeg","@type":"blob"},
"value":10,
"key":"Apples"
},
{
"image":{"length":16608,"digest":"sha1-LEFKeUsswGIjASssSBa0l/cg5rlm8=","content_type":"image/jpeg","@type":"blob"},
"value":110,
"key":"oranges"
}
]
}
Initializing Local Database
The app extracts a prebuilt database zip file named userdb.cblite2.zip
into DBPath
the first time the database is created.
This is done regardless of whether the app is launched in passive or active mode.
-
Open the CoreApp.cs file and locate the
LoadAndInitDB
method.
This method extracts the Couchbase Lite database intoDBPath
for the user (if one does not already exist).
if (!Database.Exists(DbName, DBPath)) {
using (var dbZip = new ZipArchive(ResourceLoader.GetEmbeddedResourceStream(typeof(CoreApp).GetTypeInfo().Assembly, $"{DbName}.cblite2.zip"))) {
dbZip.ExtractToDirectory(DBPath);
}
}
DB = new Database(DbName, new DatabaseConfiguration() { Directory = DBPath });
-
Open the SeasonalItemsViewModel.cs file and locate the
SeasonalItemsViewModel
constructor.
It creates a LiveQuery to pick up document changes in the inventory list array when the ViewModel loads the first time. Each array item contains a dictionary with three key value pairs. Their keys arekey
,value
, andimage
. Their values are mapped to the propertiesName
,Quantity
, andImage
in .NET ObjectSeasonalItem
. Theimage
property holds a blob entry to an image.
var q = QueryBuilder.Select(SelectResult.All())
.From(DataSource.Database(_db))
.Where(Meta.ID.EqualTo(Expression.String(CoreApp.DocId)))
.AddChangeListener((sender, args) =>
{
var allResult = args.Results.AllResults();
var result = allResult[0];
var dict = result[CoreApp.DB.Name].Dictionary;
var arr = dict.GetArray(CoreApp.ArrKey);
if (arr.Count < Items.Count)
Items = new ObservableConcurrentDictionary<int, SeasonalItem>();
Parallel.For(0, arr.Count, i =>
{
var item = arr[i].Dictionary;
var name = item.GetString("key");
var cnt = item.GetInt("value");
var image = item.GetBlob("image");
if (_items.ContainsKey(i)) {
_items[i].Name = name;
_items[i].Quantity = cnt;
_items[i].ImageByteArray = image?.Content;
} else {
var seasonalItem = new SeasonalItem {
Index = i,
Name = name,
Quantity = cnt,
ImageByteArray = image?.Content
};
_items.Add(i, seasonalItem);
}
});
});
Passive Peer or Server
First, we will walk through the steps of using the app in passive peer mode.
Initializing Websocket Listener
-
Open the ListenerViewModel.cs file and locate the
CreateListener
function.
This is where the websocket listener for peer-to-peer sync is initialized.
var listenerConfig = new URLEndpointListenerConfiguration(_db); (1)
listenerConfig.NetworkInterface = GetLocalIPv4(NetworkInterfaceType.Wireless80211) ?? GetLocalIPv4(NetworkInterfaceType.Ethernet);
//listenerConfig.Port = 0; // Dynamic port
listenerConfig.Port = 35262; // Fixed port
switch (CoreApp.ListenerTLSMode) { (2)
case LISTENER_TLS_MODE.DISABLED:
listenerConfig.DisableTLS = true;
listenerConfig.TlsIdentity = null;
break;
case LISTENER_TLS_MODE.WITH_ANONYMOUS_AUTH:
listenerConfig.DisableTLS = false; // Use with anonymous self signed cert if TlsIdentity is null
listenerConfig.TlsIdentity = null;
break;
case LISTENER_TLS_MODE.WITH_BUNDLED_CERT:
listenerConfig.DisableTLS = false;
listenerConfig.TlsIdentity = ImportTLSIdentityFromPkc12(ListenerCertLabel);
break;
case LISTENER_TLS_MODE.WITH_GENERATED_SELF_SIGNED_CERT:
listenerConfig.DisableTLS = false;
listenerConfig.TlsIdentity = CreateIdentityWithCertLabel(ListenerCertLabel);
break;
}
listenerConfig.EnableDeltaSync = true; (3)
if (CoreApp.RequiresUserAuth) { (4)
listenerConfig.Authenticator = new ListenerPasswordAuthenticator((sender, username, password) =>
{
// ** This is only a sample app to use an existing users credential shared cross platforms.
// Developers should use SecureString password properly.
var found = CoreApp.AllowedUsers.Where(u => username == u.Username && new NetworkCredential(string.Empty, password).Password == u.Password).SingleOrDefault();
return found != null;
});
}
_urlEndpointListener = new URLEndpointListener(listenerConfig);
1 | Initialize the URLEndpointListenerConfiguration for the specified database.
There is a listener for a given database.
You can specify a port to be associated with the listener, or let Couchbase Lite choose the port.
We have hard-coded 35262 (in SeasonalItemsViewModel.cs ). |
2 | This is where we configure the TLS mode.
In the app, we have a flag named ListenerTLSMode that allows the app to switch between the various modes.
You can change the mode by changing the value of the variable.
See Testing Different TLS Modes |
3 | Enable delta sync. It is disabled by default |
4 | Configure the password authenticator callback function.
This function authenticates the username/password received from the client during replication setup.
The list of valid users are configured in userallowlist.json file bundled with the app |
Testing Different TLS Modes
The app can be configured to test different TLS modes as follows by setting the ListenerTLSMode
property in the CoreApp.cs
file
public static LISTENER_CERT_VALIDATION_MODE ListenerCertValidationMode = LISTENER_CERT_VALIDATION_MODE.SKIP_VALIDATION;
ListenerTLSMode Value | Behavior |
---|---|
DISABLED |
There is no TLS. All communication is plaintext (insecure mode and not recommended in production) |
WITH_ANONYMOUS_AUTH |
The app uses self-signed cert that is auto-generated by Couchbase Lite as |
WITH_BUNDLED_CERT |
The app generates |
WITH_GENERATED_SELF_SIGNED_CERT |
The app uses Couchbase Lite |
Start Websocket Listener
-
Open the ListenerViewModel.cs file and locate the
ExecuteStartListenerCommand
method.
_urlEndpointListener.Start();
Advertising Listener Service
In the app, we broadcast listener’s IP endpoint over UDP type socket.
-
Open the ListenerViewModel.cs file and look for
Broadcast
method.+ Here, we create a Socket with Udp ProtocolType and broadcast listener’s IP endpoint to the peers are listening in local network.
Please note, this App requires peers to start peer discovery before listener start broadcasting. Otherwise, you will have to manually broadcast the listener IP. Please see Try it out for detail.
public void Broadcast()
{
if (!IsListening)
return;
using (var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, System.Net.Sockets.ProtocolType.Udp)) {
socket.EnableBroadcast = true;
var group = new IPEndPoint(IPAddress.Broadcast, CoreApp.UdpPort);
var hi = Encoding.ASCII.GetBytes($"{CoreApp.Guid}:{_urlEndpointListener.Urls[0].Host}:{_urlEndpointListener.Port}");
socket.SendTo(hi, group);
socket.Close();
}
}
Active Peer or Client
We will walk through the steps of using the app in active peer mode
Discovering Listeners
In the app, we use UDP Type Socket to listen on port 15000 for listener.
Please note, port 15000 is used by UDP Type Socket, not used by the websocket listener.+
Couchbase Lite chooses the port when a websocket listener is created.
-
Open the ListenersBrowserViewModel.cs file and look for
ListenersBrowserViewModel
constructor.
Here, we create aUdpListener
with the port it listens on and pick up any raised Udp packet received event (Listener’s broadcasting IP endpoint).
{
Title = "Browser";
Items = new ObservableCollection<ReplicatorItem>();
_discovery = new UdpListener(CoreApp.UdpPort);
_discovery.UdpPacketReceived += DiscoveryOnUdpPacketReceived;
_discovery.Start();
}
#region discover event
private void DiscoveryOnUdpPacketReceived(object sender, UdpPacketReceivedEventArgs args)
{
var msg = Encoding.ASCII.GetString(args.Data);
var msgArr = msg.Split(':');
var remoteId = Guid.Parse(msgArr[0]);
if (remoteId == CoreApp.Guid) return;
var remoteIP = IPAddress.Parse(msgArr[1]);
var remotePort = Int32.Parse(msgArr[2]);
var remoteEndpoint = new IPEndPoint(remoteIP, remotePort);
AddReplicator(remoteEndpoint);
}
Explore the content in the UdpListener.cs
.
It includes implementation of creating Socket with Udp ProtocolType, start and stop the listener, and UdpPacketReceived
EventHandler.
Initializing and Starting Replication
Initializing a replicator for peer-to-peer sync is fundamentally the same as the case if the Couchbase Lite client were to sync with a remote Sync Gateway.
-
Open the ReplicatorItem.cs file and locate the
ExecuteStartReplicatorCommand
method.
If you have been using Couchbase Lite to sync data with Sync Gateway, this code should seem very familiar. In this function, we initialize a bi-directional replication to the listener peer in continuous mode. We also register a Replication Listener to be notified of status to the replication status.
public ReplicatorItem(IPEndPoint listenerEndpoint)
{
_listenerEndpoint = listenerEndpoint;
StartReplicatorCommand = new Command(() => ExecuteStartReplicatorCommand());
CreateReplicator(ListenerEndpointString);
}
~ReplicatorItem()
{
Dispose(disposing: false);
}
#endregion
public void CreateReplicator(string PeerEndpointString)
{
if(_repl != null) {
return;
}
Uri host = new Uri(PeerEndpointString);
var dbUrl = new Uri(host, _db.Name);
var replicatorConfig = new ReplicatorConfiguration(_db, new URLEndpoint(dbUrl)); (1)
replicatorConfig.ReplicatorType = ReplicatorType.PushAndPull;
replicatorConfig.Continuous = true;
if (CoreApp.ListenerTLSMode > 0) {
// Explicitly allows self signed certificates. By default, only
// CA signed cert is allowed
switch (CoreApp.ListenerCertValidationMode) { (2)
case LISTENER_CERT_VALIDATION_MODE.SKIP_VALIDATION:
// Use acceptOnlySelfSignedServerCertificate set to true to only accept self signed certs.
// There is no cert validation
replicatorConfig.AcceptOnlySelfSignedServerCertificate = true;
break;
case LISTENER_CERT_VALIDATION_MODE.ENABLE_VALIDATION_WITH_CERT_PINNING:
// Use acceptOnlySelfSignedServerCertificate set to false to only accept CA signed certs
// Self signed certs will fail validation
replicatorConfig.AcceptOnlySelfSignedServerCertificate = false;
// Enable cert pinning to only allow certs that match pinned cert
try {
var pinnedCert = LoadSelfSignedCertForListenerFromBundle();
replicatorConfig.PinnedServerCertificate = pinnedCert;
} catch (Exception ex) {
Debug.WriteLine($"Failed to load server cert to pin. Will proceed without pinning. {ex}");
}
break;
case LISTENER_CERT_VALIDATION_MODE.ENABLE_VALIDATION:
// Use acceptOnlySelfSignedServerCertificate set to false to only accept CA signed certs
// Self signed certs will fail validation. There is no cert pinning
replicatorConfig.AcceptOnlySelfSignedServerCertificate = false;
break;
}
}
if (CoreApp.RequiresUserAuth) {
var user = CoreApp.CurrentUser;
replicatorConfig.Authenticator = new BasicAuthenticator(user.Username, user.Password); (3)
}
_repl = new Replicator(replicatorConfig); (4)
_listenerToken = _repl.AddChangeListener(ReplicationStatusUpdate);
}
public void ExecuteStartReplicatorCommand()
{
if (!IsStarted) {
_repl.Start(); (5)
1 | Initialize a Repicator Configuration for the specified local database and remote listener URL endpoint |
2 | This is where we configure the TLS server cert validation mode - whether we enable cert validation or skip validation.
This would only apply if you had enabled TLS support on listener as discussed in TLS Modes on Listener. If you skip server cert validation, you still get encrypted communication, but you are communicating with an un-trusted listener. In the app, we have a flag named ListenerCertValidationMode that allows you to try the various modes. You can change the mode by changing the value of the variable. See Testing Different Server Authentication Modes |
3 | The app uses basic client authentication to authenticate with the server |
4 | Initialize the Replicator |
5 | Start replication. The app uses the events on the Replicator Listener to listen to monitor the replication. |
Testing Different Server Authentication Modes
In Initializing Websocket Listener section, we discussed the various ways the listener TLSIdentity can be configured.
Here, we describe the corresponding changes on the replicator side to authenticate the server identity.
The app can be configured to test the different TLS modes (Table 2) by setting the ListenerCertValidationMode
property in the CoreApp.cs
file.
Naturally, if you have initialized the listener with TLSDisabled
mode, then skip this section as there is no TLS.
public static LISTENER_CERT_VALIDATION_MODE ListenerCertValidationMode = LISTENER_CERT_VALIDATION_MODE.SKIP_VALIDATION;
ListenerCertValidationMode Value | Behavior |
---|---|
SKIP_VALIDATION |
There is no authentication of server cert. The server cert is a self-signed cert. This is typically used in dev or test environments. Skipping server cert authentication is discouraged in production environments. Communication is encrypted. |
ENABLE_VALIDATION |
If the listener cert is from a well known CA then you will use this mode.
Of course, in our sample app, the listener cert as specified in |
ENABLE_VALIDATION_WITH_CERT_PINNING |
In this mode, the app uses the pinned cert, |
Stopping Replication
-
Open the ReplicatorItem.cs file and locate the
StopReplicator
method.+ If you have been using Couchbase Lite to sync data with Sync Gateway, this code should seem very familiar. In this function, we remove any listeners attached to the replicator and stop it. You can restart the replicator withExecuteStartReplicatorCommand
method
_repl?.Stop();
What Next
As an exercise, switch between the various TLS modes and server cert validation modes and see how the app behaves. You can also try with different topologies to connect the peers.
Learn More
Congratulations on completing this tutorial!
This tutorial walked you through an example of how to directly synchronize data between Couchbase Lite enabled clients. While the tutorial is for iOS, the concepts apply equally to other Couchbase Lite platforms.
Troubleshoot
Having issue running the Xamarin iOS app?
-
Xamarin iOS p2p sample app should build and run with Visual Studio 2019 with latest updates and XCode 12.0.1
-
If you have Xcode 11 and try to run the iOS app on simulator and the simulator is not loading, try to launch the simulator via Xcode and select that simulator when launching it from VS.
Connecting to an Android emulator
Sync will not work between two emulators. At least one app must be running on a device. |
When starting a listener on Android emulator and trying to connect it from a device or iOS simulator on localhost, the following steps must be followed:
-
Hard-code a port so you know what endpoint to use
-
Start the App on your device or iOS simulator
-
Start an Android emulator from Visual Studio’s device manager
-
Use ADB bridge to set port forwarding using hard-coded port number
For instance, if the listener is listening on port 35262, the command to run on the terminal of the host machine would be:
adb forward tcp:35262 tcp:35262
-
On your device or iOS simulator,
-
Within the App, select Browser
-
Enter your required endpoint including the hard-coded port number
You cannot connect to an emulator directly over localhost.
Regardless of the IP address in the displayed URL, ignore it and use127.0.0.1
as the host address.
For example, if the listener is listening on10.2.0.15:35262
,
you must connect to URL127.0.0.1:35262
.
-
-
Within the Android emulater app, Start listener
You can make an inventory change and see the change sync to the other app.
Cannot connect Android app active peer to passive peer when you are using Xamarin.Android SDK 9.x or other older version?
Go to Advanced Android Options
(Android Project Properties → Android Options → Advanced button) and change SSL/TLS implementation
configuration to Managed TLS 1.0
from Native TLS 1.2+
.