Handling Data Conflicts
Description — Couchbase Lite Database Sync — Handling conflict between data changes
Causes of Conflicts
Document conflicts can occur if multiple changes are made to the same version of a document by multiple peers in a distributed system. For Couchbase Mobile, this can be a Couchbase Lite or Sync Gateway database instance.
Such conflicts can occur after either of the following events:
-
A replication saves a document change — in which case the change with the most-revisions wins (unless one change is a delete). See the example Case 1: Conflicts when a replication is in progress
-
An application saves a document change directly to a database instance — in which case, last write wins, unless one change is a delete — see Case 2: Conflicts when saving a document
| Deletes always win. So, in either of the above cases, if one of the changes was a Delete then that change wins. |
The following sections discuss each scenario in more detail.
|
Dive deeper …
Read more about Document Conflicts and Automatic Conflict Resolution in Couchbase Mobile.
|
Conflicts when Replicating
There’s no practical way to prevent a conflict when incompatible changes to a document are be made in multiple instances of an app. The conflict is realized only when replication propagates the incompatible changes to each other.
-
Molly uses her device to create DocumentA.
-
Replication syncs DocumentA to Naomi’s device.
-
Molly uses her device to apply ChangeX to DocumentA.
-
Naomi uses her device to make a different change, ChangeY, to DocumentA.
-
Replication syncs ChangeY to Molly’s device.
This device already has ChangeX putting the local document in conflict.
-
Replication syncs ChangeX to Naomi’s device.
This device already has ChangeY and now Naomi’s local document is in conflict.
Automatic Conflict Resolution
| The rules only apply to conflicts caused by replication. Conflict resolution takes place exclusively during pull replication, while push replication remains unaffected. |
Couchbase Lite uses the following rules to handle conflicts such as those described in A typical replication conflict scenario:
-
If one of the changes is a deletion:
A deleted document (that is, a tombstone) always wins over a document update.
-
If both changes are document changes:
The change with the most revisions will win.
Since each change creates a revision with an ID prefixed by an incremented version number, the winner is the change with the highest version number.
The result is saved internally by the Couchbase Lite replicator. Those rules describe the internal behavior of the replicator. For additional control over the handling of conflicts, including when a replication is in progress, see Custom Conflict Resolution.
Custom Conflict Resolution
Starting in Couchbase Lite 2.6, application developers who want more control over how document conflicts are handled can use custom logic to select the winner between conflicting revisions of a document.
If a custom conflict resolver is not provided, the system will automatically resolve conflicts as discussed in Automatic Conflict Resolution, and as a consequence there will be no conflicting revisions in the database.
| While this is true of any user defined functions, app developers must be strongly cautioned against writing sub-optimal custom conflict handlers that are time consuming and could slow down the client’s save operations. |
To implement custom conflict resolution during replication, you must implement the following steps.
Conflict Resolver
Apps have the following strategies for resolving conflicts:
-
Local Wins: The current revision in the database wins.
-
Remote Wins: The revision pulled from the remote endpoint through replication wins.
-
Merge: Merge the content bodies of the conflicting revisions.
-
Local Wins
-
Remote Wins
-
Merge
static const CBLDocument* local_win_conflict_resolver(void* context,
FLString documentID,
const CBLDocument* localDocument,
const CBLDocument* remoteDocument)
{
return localDocument;
}static const CBLDocument* remote_win_conflict_resolver(void* context,
FLString documentID,
const CBLDocument* localDocument,
const CBLDocument* remoteDocument)
{
return remoteDocument;
}static const CBLDocument* merge_conflict_resolver(void* context,
FLString documentID,
const CBLDocument* localDocument,
const CBLDocument* remoteDocument)
{
FLDict localProps = CBLDocument_Properties(localDocument);
FLDict remoteProps = CBLDocument_Properties(remoteDocument);
FLMutableDict mergeProps = FLDict_MutableCopy(localProps, kFLDefaultCopy);
FLDictIterator d;
FLDictIterator_Begin(localProps, &d);
FLValue value;
while((value = FLDictIterator_GetValue(&d))) {
FLString key = FLDictIterator_GetKeyString(&d);
if(FLDict_Get(mergeProps, key)) {
continue;
}
FLMutableDict_SetValue(mergeProps, key, value);
FLDictIterator_Next(&d);
}
CBLDocument* mergeDocument = CBLDocument_CreateWithID(documentID);
CBLDocument_SetProperties(mergeDocument, mergeProps);
FLMutableDict_Release(mergeProps);
return mergeDocument;
}When a null document is returned by the resolver, the conflict will be resolved as a document deletion.
Important Guidelines and Best Practices
-
If you have multiple replicators, it is recommended that instead of distinct resolvers, you should use a unified conflict resolver across all replicators. Failure to do so could potentially lead to data loss under exception cases or if the app is terminated (by the user or an app crash) while there are pending conflicts.
-
If the document ID of the document returned by the resolver does not correspond to the document that is in conflict then the replicator will log a warning message.
Developers are encouraged to review the warnings and fix the resolver to return a valid document ID. -
If a document from a different database is returned, the replicator will treat it as an error. A document replication event will be posted with an error and an error message will be logged.
Apps are encouraged to observe such errors and take appropriate measures to fix the resolver function. -
When the replicator is stopped, the system will attempt to resolve outstanding and pending conflicts before stopping. Hence apps should expect to see some delay when attempting to stop the replicator depending on the number of outstanding documents in the replication queue and the complexity of the resolver function.
-
If there is an exception thrown in the
resolve()method, the exception will be caught and handled:-
The conflict to resolve will be skipped. The pending conflicted documents will be resolved when the replicator is restarted.
-
The exception will be reported in the warning logs.
-
The exception will be reported in the document replication event.
While the system will handle exceptions in the manner specified above, it is strongly encouraged for the resolver function to catch exceptions and handle them in a way appropriate to their needs.
-
Configure the Replicator
The implemented custom conflict resolver can be registered on the replicator configuration object.
The default value of the conflictResolver is null.
When the value is null, the default conflict resolution will be applied.
// NOTE: No error handling, for brevity (see getting started)
CBLError err;
CBLEndpoint* target = CBLEndpoint_CreateWithURL(FLSTR("ws://localhost:4984/mydatabase"), &err);
CBLReplicationCollection collectionConfig;
memset(&collection, 0, sizeof(collectionConfig));
collectionConfig.collection = collection;
collectionConfig.conflictResolver = local_win_conflict_resolver;
CBLReplicatorConfiguration replConfig;
memset(&replConfig, 0, sizeof(replConfig));
replConfig.collectionCount = 1;
replConfig.collections = &collectionConfig;
replConfig.endpoint = target;
CBLReplicator* replicator = CBLReplicator_Create(&replConfig, &err);
CBLEndpoint_Free(target);
CBLReplicator_Start(replicator, false);Conflicts when Updating
When updating a document, you need to consider the possibility of update conflicts. Update conflicts can occur when you try to update a document that’s been updated since you read it.
Here’s a typical sequence of events that would create an update conflict:
-
Your code reads the document’s current properties, and constructs a modified copy to save.
-
Another thread (perhaps the replicator) updates the document, creating a new revision with different properties.
-
Your code updates the document with its modified properties, for example using CBLDatabase_SaveDocument().
Automatic Conflict Resolution
In Couchbase Lite, by default, the conflict is automatically resolved and only one document update is stored in the database. The Last-Write-Win (LWW) algorithm is used to pick the winning update. So in effect, the changes from step 2 would be overwritten and lost.
If the probability of update conflicts is high in your app and you wish to avoid the possibility of overwritten data, the save and delete APIs provide additional method signatures with concurrency control:
- Save operations
-
CBLDatabase_SaveDocumentWithConcurrencyControl() — attempts to save the document with a concurrency control.
The concurrency control parameter has two possible values:
-
lastWriteWins(default): The last operation wins if there is a conflict. -
failOnConflict: The operation will fail if there is a conflict.In this case, the app can detect the error that is being thrown, and handle it by re-reading the document, making the necessary conflict resolution, then trying again.
-
- Delete operations
-
As with save operations, delete operation also have two method signatures, which specify how to handle a possible conflict:
-
CBLDatabase_DeleteDocument(): The last write will win if there is a conflict.
-
CBLDatabase_DeleteDocumentWithConcurrencyControl(): attempts to delete the document with a concurrency control.
The concurrency control parameter has two possible values:
-
lastWriteWins(default): The last operation wins if there is a conflict. -
failOnConflict: The operation will fail if there is a conflict. In this case, the app can detect the error that is being thrown, and handle it by re-reading the document, making the necessary conflict resolution, then trying again.
-
Custom Conflict Handlers
Developers can hook a conflict handler when saving a document so they can easily handle the conflict in a single save method call.
To implement custom conflict resolution when saving a document, apps must call the save method with a conflict handler block ( CBLDatabase_SaveDocumentWithConflictHandler()).
The following code snippet shows an example of merging properties from the existing document (current) into the one being saved (new).
In the event of conflicting keys, it will pick the key value from new.
CBLDatabase* database = kDatabase;
CBLCollection* collection = CBLDatabase_DefaultCollection(database, NULL);
CBLError err;
CBLDocument* mutableDoc = CBLCollection_GetMutableDocument(collection, FLSTR("xyz"), &err);
FLMutableDict properties = CBLDocument_MutableProperties(mutableDoc);
FLMutableDict_SetString(properties, FLSTR("name"), FLSTR("apples"));
/*
static bool custom_conflict_handler(void* context, CBLDocument* documentBeingSaved,
const CBLDocument* conflictingDocument) {
FLDict currentProps = CBLDocument_Properties(conflictingDocument);
FLDict updatedProps = CBLDocument_Properties(documentBeingSaved);
FLMutableDict newProps = FLDict_MutableCopy(updatedProps, kFLDefaultCopy);
FLDictIterator d;
FLDictIterator_Begin(currentProps, &d);
FLSlice currentKey = FLDictIterator_GetKeyString(&d);
for(; currentKey.buf; currentKey = FLDictIterator_GetKeyString(&d)) {
if(FLDict_Get(newProps, currentKey)) {
continue;
}
FLValue currentValue = FLDictIterator_GetValue(&d);
FLMutableDict_SetValue(newProps, currentKey, currentValue);
}
return true;
}
*/
CBLCollection_SaveDocumentWithConflictHandler(collection, mutableDoc, custom_conflict_handler, NULL, &err);