Create a Search Vector Index in Quick Mode

Capella Operational

how-to

Use Quick Mode to create a Search Vector Index in Couchbase Capella.

Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your analyzer.

For more information about how to create a Search index in Advanced Mode, see Create a Search Index with the Capella UI.

Search Vector Indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the Search documentation.

You must create a Search Vector Index before you can run a search that supports vector comparisons.

Prerequisites

You have the Search Service enabled on a node in your operational cluster. For more information about how to change Services on your operational cluster, see Modify a Paid Cluster.
You have a bucket with scopes and collections in your operational cluster. For more information, see Manage Buckets.
You have documents in a keyspace inside your bucket that contain vector embeddings. Embeddings can be an array of floats or a base64 encoded string.

You can import a sample dataset to use with the procedure or examples on this page.

Go to Data Tools Import from your cluster and import the color-vector-sample sample data.
You have logged in to the Couchbase Capella UI.

Procedure

To create a Search Vector Index in Capella:

On the Operational Clusters page, select the operational cluster where you want to create a Search index.
Do 1 of the following:
1. Use the Vector Index creation flow.
2. Use the Search creation flow.

If you use the Vector Index creation flow, you cannot add fields other than vector fields to your index during your initial index creation.

Create a Search Vector Index from Vector Indexes

To use the Vector Indexes creation flow to create a new Search Vector Index:

Go to Data Tools Vector Indexes.
On the Vector Indexes page, click + Create Vector Index.
Click Search Vector Index.

In the Index Name field, enter a name for the Search Vector Index.

Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_).

For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope.

In the Bucket and Scope lists, choose the bucket and scope where you want to create your Search Vector index.
In the Collections list, select the collection or collections that contain documents with vector embeddings.
Under Type Mappings, in your document schema, expand a collection that contains these documents.
In your document schema, select the child field that contains your vector embeddings.
Configure the options for the child field as follows:
1. In the Type list, select 1 of the following:
  1. If your child field contains vector embeddings as an array, click vector.
    
    Vector embeddings formatted as arrays appear as {field-name} [ number ] in the editor.
  2. (Couchbase Server version 7.6.2 or later) If your child field contains vector embeddings formatted as a base64 encoded string, click vector_base64.
    
    Vector embeddings formatted as base64 strings appear as {field-name} [ string ] in the Capella Quick Mode editor.
2. In the Dimension field, check that the value matches the total number of elements in your vector embeddings array.
  
  The Search Service supports arrays up to 4096 elements. Capella automatically fills in the dimension value for your selected child field when you choose the vector or vector_base64 type.
3. In the Similarity metric list, choose the method to use to calculate the similarity between search term and Search index vectors.
  
  For more information, see Field Type Mapping Options.
4. In the Optimized for list, choose whether the Search Service should optimize Search queries for accuracy (recall) or speed (latency).
  
  For more information, see Field Type Mapping Options.
Click Add To Index.
Click Review Index.
Click Create Index.

After you create your Search Vector Index, click the index name to go back into the editor and add additional collections or child field type mappings to your index. For example, you could add the text field that you used to generate your vector embeddings. For more information, see Add Type Mappings and Mappings.

Create a Search Vector Index from Search

To use the Search creation flow to create a new Search Vector Index:

Go to Data Tools Search.
Click Create Search Index.

In the Index Name field, enter a name for the Search Vector Index.

Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_).

For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope.

In the Bucket and Scope lists, choose the bucket and scope where you want to create your Search index.
In the Collections list, select the collection or collections that contain documents with vector embeddings.
Under Type Mappings, in your document schema, expand a collection that contains these documents.
In your document schema, select the child field that contains your vector embeddings.
Configure the options for the child field as follows:
1. In the Type list, select 1 of the following:
  1. If your child field contains vector embeddings as an array, click vector.
    
    Vector embeddings formatted as arrays appear as {field-name} [ number ] in the Capella Quick Mode editor.
  2. (Couchbase Server version 7.6.2 or later) If your child field contains vector embeddings formatted as a base64 encoded string, click vector_base64.
    
    Vector embeddings formatted as base64 strings appear as {field-name} [ string ] in the Capella Quick Mode editor.
2. In the Dimension field, check that the value matches the total number of elements in your vector embeddings array.
  
  The Search Service supports arrays up to 4096 elements. Capella automatically fills in the dimension value for your selected child field when you choose the vector or vector_base64 type.
3. In the Similarity metric list, choose the method to use to calculate the similarity between search term and Search index vectors.
  
  For more information, see Field Type Mapping Options.
4. In the Optimized for list, choose whether the Search Service should optimize Search queries for accuracy (recall) or speed (latency).
  
  For more information, see Field Type Mapping Options.
5. Select Include in search results.
Click Add To Index.
(Optional) Add additional collections or child field type mappings to your index.

For example, you could add the text field that you used to generate your vector embeddings. For more information, see Add Type Mappings and Mappings.
Click Create Index.

Example: Creating a Search Vector Index for Vector Search Query Examples

If you want to use the sample dataset for the examples in Run a Vector Search with the Capella UI and Run a Vector Search with a Couchbase SDK, then you can import the following Search index definition into Capella UI:

{
    "name": "color-test",
    "type": "fulltext-index",
    "params": {
     "doc_config": {
      "docid_prefix_delim": "",
      "docid_regexp": "",
      "mode": "scope.collection.type_field",
      "type_field": "type"
     },
     "mapping": {
      "default_analyzer": "standard",
      "default_datetime_parser": "dateTimeOptional",
      "default_field": "_all",
      "default_mapping": {
       "dynamic": true,
       "enabled": false
      },
      "default_type": "_default",
      "docvalues_dynamic": false,
      "index_dynamic": true,
      "scoring_model": "bm25",
      "store_dynamic": false,
      "type_field": "_type",
      "types": {
       "color.rgb": {
        "dynamic": false,
        "enabled": true,
        "properties": {
         "description": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "name": "description",
            "type": "text",
            "store": true,
            "index": true,
            "include_term_vectors": true,
            "include_in_all": false,
            "docvalues": true
           }
          ]
         },
         "brightness": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "docvalues": true,
            "index": true,
            "name": "brightness",
            "store": true,
            "type": "number"
           }
          ]
         },
         "color": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "docvalues": true,
            "include_term_vectors": true,
            "index": true,
            "name": "color",
            "store": true,
            "type": "text"
           }
          ]
         },
         "colorvect_l2": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "dims": 3,
            "index": true,
            "name": "colorvect_l2",
            "similarity": "l2_norm",
            "type": "vector",
            "vector_index_optimized_for": "recall"
           }
          ]
         },
         "embedding_vector_dot": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "dims": 1536,
            "index": true,
            "name": "embedding_vector_dot",
            "similarity": "dot_product",
            "type": "vector",
            "vector_index_optimized_for": "recall"
           }
          ]
         }
        }
       }
      }
     },
     "store": {
      "indexType": "scorch",
      "segmentVersion": 16
     }
    },
    "sourceType": "gocbcore",
    "sourceName": "color-vector-sample",
    "sourceParams": {},
    "planParams": {
     "maxPartitionsPerPIndex": 1024,
     "indexPartitions": 1,
     "numReplicas": 0
    },
    "uuid": "42676f35cc30b84a"
   }

Make sure you imported the sample dataset with the recommended settings.

This Search Vector Index has a type mapping for a color.rgb collection and includes the following fields:

The brightness number field, which is included in search results and supports sorting and faceting.
The color string and description string fields, which are included in search results, support highlighting, phrase matching, and sorting and faceting.
The colorvect_l2 [ number ] field, which has a Dimension of 3 and uses the l2_norm Similarity Metric.
The embedding_vector_dot [ number ] field, which has a dimension of 1536 and uses the dot_product Similarity Metric.

Next Steps

This basic Search Vector Index includes the vector embeddings from the child field you specified in your type mapping. If you chose to add additional child fields and enabled Include in search results, the Search Service can also return data from those fields when you run a Vector Search query.

For example, if you used the Vector Search sample data, you might want to add another child field for the color string field to your Search Vector Index, to return color names with your Search query. For more information about how to add additional child fields to your index, see Add Type Mappings and Mappings.

You can customize your Search Vector Index like any other Search index to add additional data and improve search results. For more information about how to customize an index, see Search Index Features.

Some Search index features are only available in Advanced Mode. If you edit your Search index in Advanced Mode, you cannot make any additional edits in Quick Mode without losing all Advanced Mode settings.

For more information about how to run a search against a Search Vector Index, see Run a Vector Search with the Capella UI.