Using Full-Text Search

      +

      Description — Working with Couchbase Lite’s data model  — Querying the database using full text search
      Related Content — Predictive Queries | Indexing | QueryBuilder

      Overview

      To run a full-text search (FTS) query, you must create a full-text index on the expression being matched. Unlike regular queries, the index is not optional.

      You can choose to use SQL++ or QueryBuilder syntaxes to create and use FTS indexes.

      The following examples use the data model introduced in Indexing. They create and use an FTS index built from the hotel’s Overview text.

      SQL++

      Create Index

      SQL++ provides a configuration object to define Full Text Search indexes — FullTextIndexConfiguration.

      Example 1. Using SQL++'s FullTextIndexConfiguration
      // Insert documents
      NSArray *overviews = @[@"buy groceries", @"play chess", @"book travels", @"buy museum tickets"];
      for (NSString *overview in overviews) {
          CBLMutableDocument *doc = [[CBLMutableDocument alloc] init];
          [doc setString:@"task" forKey:@"type"];
          [doc setString:overview forKey:@"overview"];
          [collection saveDocument:doc error:&error];
      }
      
      // Create index
      CBLFullTextIndexConfiguration* config = [[CBLFullTextIndexConfiguration alloc]
                                               initWithExpression: @[@"overview"]
                                               ignoreAccents: NO
                                               language: nil];
      
      [collection createIndexWithName: @"overviewFTSIndex" config:config error: &error];

      Use Index

      FullTextSearch is enabled using the SQL++ match() function.

      With the index created, you can construct and run a Full-text search (FTS) query using the indexed properties.

      The index will omit a set of common words, to avoid words like "I", "the", "an" from overly influencing your queries. See full list of these stopwords.

      The following example finds all hotels mentioning Michigan in their Overview text.

      Example 2. Using SQL++ Full Text Search
      NSString *ftsQueryString =
      @"SELECT META().id FROM _ WHERE MATCH(overviewFTSIndex, 'Michigan') ORDER BY RANK(overviewFTSIndex)";
      CBLQuery *ftsQuery = [self.database createQuery:ftsQueryString error:&error];
      
      CBLQueryResultSet *resultSet = [ftsQuery execute:&error];
      NSArray* results  = [resultSet allResults];
      for (CBLQueryResult *result in results) {
          NSLog(@"document id %@", [result stringAtIndex:0]);
      }

      QueryBuilder

      Create Index

      The following example creates an FTS index on the overview property.

      Example 3. Using the IndexBuilder method
      // Insert documents
      NSArray *tasks = @[@"buy groceries", @"play chess", @"book travels", @"buy museum tickets"];
      for (NSString *task in tasks) {
          CBLMutableDocument *doc = [[CBLMutableDocument alloc] init];
          [doc setString:@"task" forKey:@"type"];
          [doc setString:task forKey:@"name"];
          [collection saveDocument:doc error:&error];
      }
      
      // Create index
      CBLFullTextIndex *index = [CBLIndexBuilder fullTextIndexWithItems:@[[CBLFullTextIndexItem property:@"name"]]];
      index.ignoreAccents = NO;
      [collection createIndex:index name:@"nameFTSIndex" error:&error];

      Use Index

      With the index created, you can construct and run a Full-text search (FTS) query using the indexed properties.

      The following example finds all hotels mentioning Michigan in their Overview text.

      Example 4. Using QueryBuilder Full Text Search
      id exp = [CBLQueryExpression fullTextIndex:@"nameFTSIndex"];
      CBLQueryExpression *where = [CBLQueryFullTextFunction matchWithIndex:exp query:@"'buy'"];
      
      CBLQuery *query =
        [CBLQueryBuilder
          select:@[[CBLQuerySelectResult expression:[CBLQueryMeta id]]]
          from:[CBLQueryDataSource collection:collection]
          where:where];
      
      NSEnumerator *rs = [query execute:&error];
      for (CBLQueryResult *result in rs) {
          NSLog(@"document id %@", [result stringAtIndex:0]);
      }

      Operation

      In the examples above, the pattern to match is a word, the full-text search query matches all documents that contain the word "michigan" in the value of the doc.overview property.

      Search is supported for all languages that use whitespace to separate words.

      Stemming, which is the process of fuzzy matching parts of speech, like "fast" and "faster", is supported in the following languages: Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish and Turkish.

      Pattern Matching Formats

      As well as providing specific words or strings to match against, you can provide the pattern to match in these formats.

      Prefix Queries

      The query expression used to search for a term prefix is the prefix itself with a "*" character appended to it.

      Example 5. Prefix query

      Query for all documents containing a term with the prefix "lin".

      "lin*"

      This will match

      • All documents that contain "linux"

      • And …​ those that contain terms "linear","linker", "linguistic" and so on.

      Overriding the Property Name

      Normally, a token or token prefix query is matched against the document property specified as the left-hand side of the match operator. This may be overridden by specifying a property name followed by a ":" character before a basic term query. There may be space between the ":" and the term to query for, but not between the property name and the ":" character.

      Example 6. Override indexed property name

      Query the database for documents for which the term "linux" appears in the document title, and the term "problems" appears in either the title or body of the document.

      'title:linux problems'

      Phrase Queries

      A phrase query is one that retrieves all documents containing a nominated set of terms or term prefixes in a specified order with no intervening tokens.

      Phrase queries are specified by enclosing a space separated sequence of terms or term prefixes in double quotes (").

      Example 7. Phrase query

      Query for all documents that contain the phrase "linux applications".

      "linux applications"

      NEAR Queries

      A NEAR query is a query that returns documents that contain a two or more nominated terms or phrases within a specified proximity of each other (by default with 10 or less intervening terms). A NEAR query is specified by putting the keyword "NEAR" between two phrase, token or token prefix queries. To specify a proximity other than the default, an operator of the form "NEAR/" may be used, where is the maximum number of intervening terms allowed.

      Example 8. Near query

      Search for a document that contains the phrase "replication" and the term "database" with not more than 2 terms separating the two.

      "database NEAR/2 replication"

      AND, OR & NOT Query Operators::

      The enhanced query syntax supports the AND, OR and NOT binary set operators. Each of the two operands to an operator may be a basic FTS query, or the result of another AND, OR or NOT set operation. Operators must be entered using capital letters. Otherwise, they are interpreted as basic term queries instead of set operators.

      Example 9. Using And, Or and Not

      Return the set of documents that contain the term "couchbase", and the term "database".

      "couchbase AND database"

      Operator Precedence

      When using the enhanced query syntax, parenthesis may be used to specify the precedence of the various operators.

      Example 10. Operator precedence

      Query for the set of documents that contains the term "linux", and at least one of the phrases "couchbase database" and "sqlite library".

      '("couchbase database" OR "sqlite library") AND "linux"'

      Ordering Results

      It’s very common to sort full-text results in descending order of relevance. This can be a very difficult heuristic to define, but Couchbase Lite comes with a ranking function you can use.

      In the OrderBy array, use a string of the form Rank(X), where X is the property or expression being searched, to represent the ranking of the result.