A newer version of this documentation is available.

View Latest

Full Text Search (FTS) Using the .Go SDK with Couchbase Server

    +
    You can use the Full Text Search service (FTS) to create queryable full-text indexes in Couchbase Server.

    Full Text Search or FTS allows you to create, manage, and query full text indexes on JSON documents stored in Couchbase buckets. It uses natural language processing for querying documents, provides relevance scoring on the results of your queries, and has fast indexes for querying a wide range of possible text searches. Some of the supported query types include simple queries like Match and Term queries; range queries like Date Range and Numeric Range; and compound queries for conjunctions, disjunctions, and/or boolean queries. The Go SDK exposes an API for performing FTS queries which abstracts some of the complexity of using the underlying REST API.

    When using a Couchbase version < 6.5 you must create a valid Bucket connection using cluster.Bucket(name) before you can use Search.

    Examples

    For the purposes of the below examples we will use the Travel Sample sample bucket with the below Full Text Search index.

    {
      "name": "travel-sample-index-hotel-description",
      "type": "fulltext-index",
      "params": {
        "doc_config": {
          "mode": "type_field",
          "type_field": "type"
        },
        "mapping": {
          "analysis": {
            "analyzers": {
              "myUnicodeAnalyzer": {
                "tokenizer": "unicode",
                "type": "custom"
              }
            }
          },
          "default_analyzer": "standard",
          "default_datetime_parser": "dateTimeOptional",
          "default_field": "_all",
          "default_mapping": {
            "dynamic": true,
            "enabled": false
          },
          "default_type": "_default",
          "index_dynamic": true,
          "store_dynamic": true,
          "type_field": "type",
          "types": {
            "hotel": {
              "dynamic": false,
              "enabled": true,
              "properties": {
                "description": {
                  "enabled": true,
                  "dynamic": false,
                  "fields": [
                    {
                      "include_in_all": true,
                      "include_term_vectors": true,
                      "index": true,
                      "name": "description",
                      "store": true,
                      "type": "text"
                    }
                  ]
                }
              }
            }
          }
        },
        "store": {
          "kvStoreName": "scorch"
        }
      },
      "sourceType": "couchbase",
      "sourceName": "travel-sample",
      "sourceUUID": "",
      "sourceParams": {},
      "planParams": {
        "maxPartitionsPerPIndex": 171,
        "numReplicas": 0
      },
        "uuid": ""
    }

    Search queries are executed at Cluster level (not bucket or collection). As of Couchbase Server 6.5+ they do also not require a bucket to be opened first. In older versions of Couchbase Server, even though executed at Cluster level, a bucket must be opened before performing queries. Search queries, facets, and sorting have a slightly different import to other components of gocb, you can import them using import "github.com/couchbase/gocb/v2/search". Here is a simple MatchQuery that looks for the text “swanky” using a defined index:

    	matchResult, err := cluster.SearchQuery(
    		"travel-sample-index-hotel-description",
    		search.NewMatchQuery("swanky"),
    		&gocb.SearchOptions{
    			Limit: 10,
    		},
    	)

    All simple query types are created in the same manner, although some have additional properties, which can be seen in common query type descriptions. Couchbase FTS’s range of query types enable powerful searching using multiple options, to ensure results are just within the range wanted. Here is a date range query that looks for dates between 1st January 2019 and 31st January, the second parameter is whether the date should be considered inclusive:

    	dateRangeResult, err := cluster.SearchQuery(
    		"travel-sample-index-hotel-description",
    		search.NewDateRangeQuery().Start("2019-01-01", true).End("2019-02-01", false),
    		&gocb.SearchOptions{
    			Limit: 10,
    		},
    	)

    Queries can also be combined together. A conjunction query contains multiple child queries; its result documents must satisfy all of the child queries:

    	conjunctionResult, err := cluster.SearchQuery(
    		"travel-sample-index-hotel-description",
    		search.NewConjunctionQuery(
    			search.NewMatchQuery("swanky"),
    			search.NewDateRangeQuery().Start("2019-01-01", true).End("2019-02-01", false),
    		),
    		&gocb.SearchOptions{
    			Limit: 10,
    		},
    	)

    Working with Results

    The result of a search query has three components: rows, facets, and metdata. Rows are the documents that match the query. Facets allow the aggregation of information collected on a particular result set. Metdata holds additional information not directly related to your query, such as success total hits and how long the query took to execute in the cluster.

    Iterating Rows

    Here we are iterating over the rows that were returned in the results. Note that Fields is a special case, where it’s a function. Fields will include any fields that were requested as part of the SearchQuery (Fields option within the options block).

    	for matchResult.Next() {
    		row := matchResult.Row()
    		docID := row.ID
    		score := row.Score
    
    		var fields interface{}
    		err := row.Fields(&fields)
    		if err != nil {
    			panic(err)
    		}
    
    		fmt.Printf("Document ID: %s, search score: %f, fields included in result: %v\n", docID, score, fields)
    	}
    
    	// always check for errors after iterating
    	err = matchResult.Err()
    	if err != nil {
    		panic(err)
    	}

    Take care to ensure you call Err after accessing rows and check for any errors returned.

    Iterating facets

    Facets can only be accessed once Close has been called on rows.

    	facets, err := matchResult.Facets()
    	if err != nil {
    		panic(err)
    	}
    	for _, facet := range facets {
    		field := facet.Field
    		total := facet.Total
    
    		fmt.Printf("Facet field: %s, total: %d\n", field, total)
    	}

    Consistency

    Like the Couchbase Query Service, FTS allows RequestPlus queries — Read-Your-Own_Writes (RYOW) consistency, ensuring results contain information from updated indexes:

    	hotel := struct {
    		Description string `json:"description"`
    	}{Description: "super swanky"}
    	myWriteResult, err := collection.Upsert("a-new-hotel", hotel, nil)
    	if err != nil {
    		panic(err)
    	}
    
    	consistentWithResult, err := cluster.SearchQuery(
    		"travel-sample-index-hotel-description",
    		search.NewMatchQuery("swanky"),
    		&gocb.SearchOptions{
    			Limit:          10,
    			ConsistentWith: gocb.NewMutationState(*myWriteResult.MutationToken()),
    		},
    	)