Import Filter

    +

    Introducing import filters and how to use them to speed-up the initial import process.
    This topic provides an example configuration of Import filters

    Introduction

    The initial import process can take considerable time to complete for clusters with a large amount of data.

    The process can be made more efficient by using an Import filter.

    Configuration

    To get started with a clusters which contains a large number of pre-existing documents, we recommend you use an import_filter to filter out unwanted data and reduce the initial import processing time — see: Example 1.

    Example 1. Using an Import Filter
    //
    {
      //  ... may be preceded by additional configuration data as required by the user ...
      "databases": {
        "getting-started-db": {
          "server": "http://localhost:8091",
          "bucket": "getting-started-bucket",
          "username": "sync_gateway", (1)
          "password": "password", (2)
          "enable_shared_bucket_access": true, (3)
          "import_docs": true,
          "num_index_replicas": 0, (4)
          "import_filter": `
            function(doc) { (5)
              if (doc.type != "mobile") {
                return false
              }
              return true
            }`,
          "users": {
            "GUEST": { "disabled": false, "admin_channels": ["*"] }
          },
          "sync": `function (doc, oldDoc) {
            if (doc.sdk) {
              channel(doc.sdk);
            }
          }`
        }
      }
      //  ... may be followed by additional configuration data as required by the user ...
    }
    //

    Configuration properties:

    1 The user’s username that you created on the Couchbase Server Admin Console.
    2 The user’s password that you created on the Couchbase Server Admin Console.
    3 The Sync with Couchbase Server feature allows Couchbase Server SDKs to also perform operations on this bucket.
    4 num_index_replicas is the number of index replicas stored in Couchbase Server, introduced with GSI/N1QL indexing — see Indexing versus Views. If you’re running a single Couchbase Server node for development purposes the num_index_replicas must be set to 0.
    5 Only import documents which have a type property equal to mobile.