Create a Vector Search Index with the Server Web Console

how-to

You can create a Vector Search index with the Couchbase Server Web Console.

You must create a Vector Search index before you can run a search that supports vector comparisons.

Vector Search indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the Search documentation.

Prerequisites

You have the Search Service enabled on a node in your database. For more information about how to deploy a new node and Services on your database, see Manage Nodes and Clusters.
You have a bucket with scopes and collections in your database. For more information about how to create a bucket, see Create a Bucket.

You have documents in a keyspace inside your bucket that contain vector embeddings.

You can download a sample dataset to use with the procedure or examples on this page:

Download color_data_2vectors.zip

To get the best results with using the sample data with the examples in this documentation, import the sample files from the dataset into your database with the following settings:

Use a bucket called vector-sample.
Use a scope called color.
Use a collection called rgb for rgb.json.
To set your document keys, use the value of the id field from each JSON document.

Your user account has the Search Admin role for the bucket where you want to create the index.
You have logged in to the Couchbase Server Web Console.

Procedure

To create a Vector Search index with the Couchbase Server Web Console:

Go to Search.
Click Add Index.

In the Index Name field, enter a name for the Vector Search index.

Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_).

For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope.

In the Bucket list, select the bucket that contains the documents you want to include in your index.
Expand Customize Index.
Select Use non-default scope/collection(s).
1. In the Scope list, select the scope that contains the documents you want to include in your index.
Expand Mappings and create a new type mapping on a collection:
1. Click Add Type Mapping.
2. In the Collection list, select the collection that contains the documents you want to include in your index.
3. Select Only index specified fields.
4. Click OK.
Create a Child Field on the new collection type mapping with the following settings:
1. In the Field field, enter the name of the field in your documents that contains your vector embeddings.
  
  Vectors must be represented as an array of floating point numbers.
2. In the Type list, select vector.
3. In the Dimension field, enter the total number of elements in the array that holds the vector embeddings for your documents.
  
  The Search Service supports arrays up to 2048 elements. Arrays can be an array of arrays.
4. In the Similarity metric list, select the method to use to calculate the similarity between search term and Search index vectors.
  
  For more information, see Child Field Options.
5. In the Optimized for list, select whether the Search Service should optimize Search queries for accuracy (recall) or speed (latency).
  
  For more information, see Child Field Options.
6. Select Index.
7. Click OK.
(Optional) Create another child field on the new collection type mapping for any additional fields you want to return in your search results.

For example, you could add the text field that you used to generate your vector embeddings.
Click Create Index.

Example: Creating a Vector Search Index for Vector Search Query Examples

If you want to use the sample dataset for the examples in Run A Vector Search with the Server Web Console and Run a Vector Search with a Couchbase SDK, then you can import the following Search index definition into Server Web Console:

{
    "name": "color-test",
    "type": "fulltext-index",
    "params": {
     "doc_config": {
      "docid_prefix_delim": "",
      "docid_regexp": "",
      "mode": "scope.collection.type_field",
      "type_field": "type"
     },
     "mapping": {
      "default_analyzer": "standard",
      "default_datetime_parser": "dateTimeOptional",
      "default_field": "_all",
      "default_mapping": {
       "dynamic": true,
       "enabled": false
      },
      "default_type": "_default",
      "docvalues_dynamic": false,
      "index_dynamic": true,
      "store_dynamic": false,
      "type_field": "_type",
      "types": {
       "color.rgb": {
        "dynamic": false,
        "enabled": true,
        "properties": {
         "brightness": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "docvalues": true,
            "index": true,
            "name": "brightness",
            "store": true,
            "type": "number"
           }
          ]
         },
         "color": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "docvalues": true,
            "include_term_vectors": true,
            "index": true,
            "name": "color",
            "store": true,
            "type": "text"
           }
          ]
         },
         "colorvect_l2": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "dims": 3,
            "index": true,
            "name": "colorvect_l2",
            "similarity": "l2_norm",
            "type": "vector",
            "vector_index_optimized_for": "recall"
           }
          ]
         },
         "embedding_vector_dot": {
          "enabled": true,
          "dynamic": false,
          "fields": [
           {
            "dims": 1536,
            "index": true,
            "name": "embedding_vector_dot",
            "similarity": "dot_product",
            "type": "vector",
            "vector_index_optimized_for": "recall"
           }
          ]
         }
        }
       }
      }
     },
     "store": {
      "indexType": "scorch",
      "segmentVersion": 16
     }
    },
    "sourceType": "gocbcore",
    "sourceName": "vector-sample",
    "sourceParams": {},
    "planParams": {
     "maxPartitionsPerPIndex": 1024,
     "indexPartitions": 1,
     "numReplicas": 0
    },
    "uuid": "42676f35cc30b84a"
   }

Make sure you imported the sample dataset with the recommended settings.

This Vector Search index has a type mapping for a color.rgb collection and includes the following fields:

The brightness number field, which is included in search results and supports sorting and faceting.
The color string and description string fields, which are included in search results, support highlighting, phrase matching, and sorting and faceting.
The colorvect_l2 [ number ] field, which has a Dimension of 3 and uses the l2_norm Similarity Metric.
The embedding_vector_dot [ number ] field, which has a dimension of 1536 and uses the dot_product Similarity Metric.

Next Steps

A basic Vector Search index includes the vector embeddings from the child field you specified in your type mapping. If you choose to add additional child fields and enable Include in search results, the Search Service can also return data from those child fields when you run a Vector Search query.

For more information about how to add additional child fields to your index, see Create a Search Index with the Quick Editor or Create a Child Field.

You can customize your Vector Search index like any other Search index to add additional data and improve search results. For more information about how to customize an index, see Customize a Search Index with the Web Console.

For more information about how to run a search against a Vector Search index, see Run A Vector Search with the Server Web Console or Run a Vector Search with the REST API and curl/HTTP.