Import Filter
March 16, 2025
Introducing import filters and how to use them to speed-up the initial import process.
This topic provides an example configuration of Import filters
Introduction
Sync Gateway’s initial import process can take a considerable time to complete for clusters with a large amount of data. The process can be made more efficient by using an Import filter. Without a filter (the default), Sync Gateway imports all documents.
Configuration
To get started with a clusters which contains a large number of pre-existing documents, we recommend you use an import_filter to filter out unwanted data and reduce the initial import processing time — see: Example 1.
Example 1. Using an Import Filter
//
{
// ... may be preceded by additional configuration data as required by the user ...
"databases": {
"getting-started-db": {
"server": "http://localhost:8091",
"bucket": "getting-started-bucket",
"username": "sync_gateway", (1)
"password": "password", (2)
"enable_shared_bucket_access": true, (3)
"import_docs": true,
"num_index_replicas": 0, (4)
"import_filter": `
function(doc) { (5)
if (doc.type != "mobile") {
return false
}
return true
}`,
"users": {
"GUEST": { "disabled": false, "admin_channels": ["*"] }
},
"sync": `function (doc, oldDoc) {
if (doc.sdk) {
channel(doc.sdk);
}
}`
}
}
// ... may be followed by additional configuration data as required by the user ...
}
//Configuration properties:
| 1 | The user’s username that you created on the Couchbase Server Admin Console. |
| 2 | The user’s password that you created on the Couchbase Server Admin Console. |
| 3 | The Sync with Couchbase Server feature allows Couchbase Server SDKs to also perform operations on this bucket. |
| 4 | num_index_replicas is the number of index replicas stored in Couchbase Server, introduced with GSI/N1QL indexing — see Indexing versus Views.
If you’re running a single Couchbase Server node for development purposes the num_index_replicas must be set to 0. |
| 5 | Only import documents which have a type property equal to mobile. |