Enhanced Conflict Resolution
About conflict resolution in inter-Sync Gateway replication
Introduces inter-Sync Gateway replication conflict resolution policies and behaviors
Other Topics: Legacy Pre-3.0 Configuration | Admin REST API
Context Clarification
This content relates only to inter-Sync Gateway replication in Sync Gateway 2.8+. For documentation on pre-2.8 inter-Sync Gateway replication (also known as SG Replicate) — see the documentation for the appropriate release. |
Automatic Conflict Resolution
Inter-Sync Gateway pull replications support automatic conflict resolution by default (no conflict mode).
In Pull replications the active Sync Gateway detects and resolves conflicts based on its configured conflict_resolution_type and conflict resolver policy. This policy will determine the winner or return an error if it cannot.
Conflicts are not resolved in push replications though.
The passive end of the push simply detects and rejects any conflicting revisions (409 Conflict
response).
Both approaches reflect the way conflicts are handled by Couchbase Lite clients. Not surprising since in both instances Couchbase Lite is acting like the active node in an inter-sync gateway exchange.
Conflicts are only resolved during a pull replication.
If conflicts are likely, you should configure a Alternatively: Run the replicator from the other side; flipping the direction (to See also — our blog post: Document Conflicts & Resolution in Couchbase Mobile |
For ENTERPRISE EDITION, a custom conflict resolver policy is available, providing additional flexibility by allowing users to provide their own conflict resolution logic — see: Inter Sync Gateway Sync - Custom Conflict Resolution
Configure Conflict Resolution
Invoke automatic conflict resolution by specifying the required conflict resolver policy in the replication definition. The specified policy is applied whenever a conflict is detected.
-
default
-
localWins
-
remoteWins
"databases:"
// other config as necessary
"this_db:"
// other config as necessary
"sgreplicate_enabled": "true",
"replications": [
{
"replication_id": "replication1",
"direction": "push_and_pull",
"continuous": true,
"filter": "sync_gateway/bychannel",
"query_params": [
"channel1",
"channel2"
],
"conflict_resolution_type": "default",
// other config as necessary
}
]
// other config as necessary
"databases:"
// other config as necessary
"this_db:"
// other config as necessary
"sgreplicate_enabled": "true",
"replications": [
{
"replication_id": "replication1",
"direction": "push_and_pull",
"continuous": true,
"filter": "sync_gateway/bychannel",
"query_params": [
"channel1",
"channel2"
],
"conflict_resolution_type": "localWins",
// other config as necessary
}
]
// other config as necessary
"databases:"
// other config as necessary
"this_db:"
// other config as necessary
"sgreplicate_enabled": "true",
"replications": [
{
"replication_id": "replication1",
"direction": "push_and_pull",
"continuous": true,
"filter": "sync_gateway/bychannel",
"query_params": [
"channel1",
"channel2"
],
"conflict_resolution_type": "remoteWins",
// other config as necessary
}
]
// other config as necessary
Build a Conflict Resolution Policy [EE]
This content relates only to ENTERPRISE EDITION |
Custom conflict resolution is handled by the active Sync Gateway using a user-provided custom conflict resolver. This Javascript function is embedded in the replication configuration.
The predefined conflict resolver policies are also available as Javascript functions that you can call from within that custom_conflict_resolver function This is useful when you want to apply greater selectivity to the automatic conflict resolution process. For example, you want to apply a 'remote wins' policy only for a specific type of document - see the 'Use Policies' tab in Example 5.
Conflict Resolution Approaches
There are two ways to handle conflicts in your custom_conflict_resolver, you can either:
-
Choose a winning revision from among the conflicting revisions (see Example 5), or
-
Merge conflicting revision to create a new winning revision; losing revisions are tomb-stoned.
When creating a new revision, do not provide a _rev
property. A new revision ID will be generated. Usedelete mergedDoc._rev
to invalidate the property.However, users should avoid overly-complex resolver logic that may impact performance.
Approaches to Error Handling
Your custom conflict resolver function should not terminate the replication when it encounters exceptions or errors. Instead, you should log sufficient information to aid troubleshooting and recovery.
For example, your custom conflict resolver function should:
-
Skip the document causing the issue
-
Log a suitable warning level message. Include at least the skipped document’s Id and the sequence Id of the revision in error.
Refer to log files when troubleshooting conflict resolution errors, to identify the document id and revision sequence in error.
- Unexpected data in the remote document
-
You should update the remote document to fix the issue. Doing so will cause replication of the update.
- Unexpected data in the local document
-
You should update the local document to fix the issue. This will not trigger a pull-replication. Do a no-op-update [1] of the remote document, which will trigger replication and conflict resolution.
- Fault in conflict resolution javascript function
-
Fix the Javascript logic and then either:
-
Do a no op update [1] of the remote document. This triggers a pull replication and subsequent conflict resolution.
-
Reset the replication (using
_replicationstatus/reset
endpoint). Not recommended as it introduces significant duplicate processing in resyncing previously synced documents.
-
Conflict Resolver Structure
Sync Gateway supports the use of Javascript functions to customize the sync process. These functions are referenced from within the Sync Gateway Configuration and may be provided either as:
-
An inline Javascript function
-
An external Javascript file
-
An external HTTP/HTTPS endpoint serving a JS function [2].
You can provide your conflict resolver as either an inline or external Javascript function. You can learn more about the ($db.custom_conflict_resolver) property in the Configuration Schema Reference — see: custom_conflict_resolver.
Sync gateway 3.x configuration of Javascript functions is done using the Admin REST API; specifically the Access Control and /{db}/_config/import_filter endpoints. |
Prior to this, configuration was done within the database configuration file — see: Example 3
-
Inline Javascript functions provided within the database configuration must be enclosed by a backtick pair (``).
-
To use an external Javascript function for any of the eligible options, you need to specify the absolute path to the Javascript. The format and content of the external Javascript is the same as that provided inline.
You must register a CA certificate for the appropriate server if external Javascript functions are hosted on HTTPS endpoints. For testing purposes you may use the unsupported configuration option unsupported.remote_config_tls_skip_verify
. Setting thistrue
will side-step essential security checks. Do not use in Production deployments.
This example shows the different ways you might provide a Javascript Sync Function. Although the example uses the Sync Function, the same approach applies wherever a Javascript function is valid (including with Import Filter, Webhook and Custom Conflict Resolver).
curl -X PUT 'http://localhost:4985/db1/_config' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
"sync": "/opt/couchbase-sync-gateway/sync.js" (1)
},
}
}'
curl -X PUT 'http://localhost:4985/db2/_config' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
"sync": "https://localhost/sync/func2" (2)
}
}
}
curl -X PUT 'http://localhost:4985/db3/_config' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
"sync": `function(doc,oldDoc, meta){ if (doc.published) { channel("public");} }`
} (3)
}
}
1 | Here we specify an external file sync.js as containing the external function to be provisioned |
2 | Here we specify a HTTPS endpoint as resolving to a Javascript function to be provisioned |
3 | Here we specify inline Javascript (surrounded by a pair of backticks (``) as the function to be provisioned |
The following example shows the basic structure of the conflict resolver function as it would be defined in the configuration file.
"custom_conflict_resolver": "`function(conflict) { (1)
// . . .
// . . . application logic to determine winner
// . . .
return conflict.LocalDocument; (2)
}`" (3)
1 | The conflict structure comprises both conflicting documents.
|
2 | You should return one of:
|
3 | The conflict resolver function is enclosed by backticks (`` ) |
Sample Conflict Resolvers
-
Use Built-in Policies
-
Nominate a Winner
-
Merge a Winner
This example uses the built-in resolver functions to resolve the conflict based-on the document type.
So, documents of type a-doc-type-1
are always resolved in favor of the remote revision. All other document types are resolved in accordance with the default resolver policy.
"replications": [
{
"replication_id": "replication1",
// other config as required
"conflict_resolution_type": "custom",
"custom_conflict_resolver": `
function(conflict) {
if (conflict.LocalDocument.type == "a-doctype-1") &&
(conflict.RemoteDocument.type == "a-doctype-1")
{
// Invoke the built in default resolver logic
return defaultPolicy(conflict);
}
else {
// Otherwise resolve in favor of remote document
return conflict.RemoteDocument;
}
}
`
// other config as required
}
]
This example selects a winner based on relative priorities and builds a return response of its own rather than using either the localWins or remoteWins policy, although it does rely on the default resolver policy as a backstop.
"replications": [
{
// . . . preceding replication details as required
},
{
"replication_id": "replication2",
// . . . other config as required
"conflict_resolution_type": "custom",
"custom_conflict_resolver": `
function(conflict) {
// Custom conflict resolution policy based on priority
if (conflict.LocalDocument.body.priority > conflict.RemoteDocument.body.priority) {
// Choose a local winner
// Optionally apply application logic to manipulate
// the local object before returning it as the winner
return conflict.LocalDocument;
} else if (local.body.priority < remote.body.priority) {
// Choose a remote winner
// Optionally apply application logic to manipulate
// the remote object before returning it as the winner
return conflict.RemoteDocument;
}; //end if
} //end func()
// Apply the default policy as a catch all
return defaultPolicy(conflict);
}` // end resolver property
}, // end replication2
{
// . . . further replication details as required
}
]
// . . . other config as required
This example creates a winner by merging changes from the local and remote documents to create a new document object, which is returned as the winner.
If both document.types are non-null and the local document.type is usedefault
, the merge path is overridden and the default resolver policy is applied.
"custom_conflict_resolver":`
function(conflict) {
if ( (conflict.LocalDocument.type != null) &&
(conflict.RemoteDocument.type != null) &&
(conflict.LocalDocument.type == "usedefault"))
{
console.log("Will use default policy");
// Resolve using built-in policy
return defaultPolicy(conflict);
}
else
{
// Merge local and remote docs
var remoteDoc = conflict.RemoteDocument;
console.log("full remoteDoc doc: "+JSON.stringify(remoteDoc));
var localDoc = conflict.LocalDocument;
console.log("full localDoc doc: "+JSON.stringify(localDoc));
var mergedDoc = extend({}, localDoc, remoteDoc);
delete mergedDoc._rev (1)
console.log("full mergedDoc doc: "+JSON.stringify(mergedDoc));
// Resolve using this merged doc as the winner
return mergedDoc;
function extend(target) {
var sources = [].slice.call(arguments, 1);
sources.forEach(function (source) {
for (var prop in source) {
if (prop.indexOf('_') != 0) { (2)
target[prop] = source[prop];
}
}
});
return target;
} // end function extend()
} // end if
}` // end function()
1 | Invalidate the _rev property.
A new revision ID will be generated. |
||
2 | Transcribe properties from source to target.
|