--- title: Default Token Filters description: Use a token filter to filter a tokenizer's results and get better search result matches. pubDate: 2026-08-17T09:53:44.266Z antora: editUrl: https://github.com/couchbaselabs/docs-devex/edit/release/7.2/modules/search/pages/default-token-filters-reference.adoc xref: xref:7.2@server:search:default-token-filters-reference.adoc[] --- [Consult the llms.txt file for a full list of contents](/llms.txt) [View original HTML](/server/7.2/search/default-token-filters-reference.html) # Default Token Filters > Use a token filter to filter a tokenizer's results and get better search result matches. The Search Service's [token filters](customize-index.md#token-filters) work with [tokenizers](customize-index.md#tokenizers) to filter search input tokens. Tokens can come from the content of your Search index or a Search query. For more information about token filters, see [Token Filters](customize-index.md#token-filters). The following token filters are available: | Token Filter Type | Description | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | apostrophe | Removes all characters after an apostrophe (') from tokenizer results. Also removes the apostrophe. For example, the token Couchbase's becomes Couchbase. | | camelCase | Splits text in camelCase inside a token into separate tokens. For example, the token filter splits the token camelCaseText into camel, Case, and Text. | | cjk\_bigram | Converts Chinese, Japanese, and Korean tokenizer results into bigrams, or groups of two consecutive words. | | cjk\_width | Converts Chinese, Japanese, and Korean tokenizer results from full width ASCII variants into Latin characters, and half-width katakana characters into their equivalent kana characters. | | elision\_ca | Removes all characters before an apostrophe from Catalan language tokenizer results. Also removes the apostrophe. | | elision\_fr | Removes all characters before an apostrophe from French language tokenizer results. Also removes the apostrophe. For example, the token filter converts the token l'avion to avion. | | elision\_ga | Removes all characters before an apostrophe from Gaelic language tokenizer results. Also removes the apostrophe. | | elision\_it | Removes all characters before an apostrophe from Italian language tokenizer results. Also removes the apostrophe. | | hr\_suffix\_transformation\_filter | Replaces suffixes in Croatian tokenizer results with normalized suffixes. | | lemmatizer\_he | Lemmatizes similar forms of Hebrew words. Corrects spelling mistakes. | | mark\_he | Marks the Hebrew, non-Hebrew, and numeric tokens from tokenizer results. | | niqqud\_he | Forces niqqud-less spelling for Hebrew text in tokenizer results. | | normalize\_ar | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize Arabic characters in tokens. | | normalize\_ckb | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize Kurdish characters in tokens. | | normalize\_de | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize German characters in tokens. | | normalize\_fa | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize Persian characters in tokens. | | normalize\_hi | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize Hindi characters in tokens. | | normalize\_in | Uses [Unicode Normalization](http://unicode.org/reports/tr15/) to normalize Indonesian characters in tokens. | | possessive\_en | Checks the second-last character in English-language tokenizer results for an apostrophe. If it finds an apostrophe, the token filter removes the last two characters from the token. | | reverse | Reverses the tokens in tokenizer results. For example, the token filter converts the token acrobat to taborca. | | stemmer\_ar | Checks Arabic tokenizer results for suffixes and prefixes. If it finds a suffix or any prefixes, the token filter removes them to leave the root word. | | stemmer\_ckb | Checks Kurdish tokenizer results for prefixes. If it finds a prefix, the token filter removes it to leave the root word. | | stemmer\_da\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Danish language tokenizer results into word stems. | | stemmer\_de\_light | Uses light stemming to convert German language tokenizer results into word stems. Regular stemming can affect the semantic meaning of words, as several words with different meanings might have the same root stem. Light stemming only removes frequently used prefixes and suffixes, and doesn't produce the root of a word to preserve semantics. | | stemmer\_de\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert German language tokenizer results into word stems. | | stemmer\_en\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert English language tokenizer results into word stems. | | stemmer\_es\_light | Uses light stemming to convert Spanish language tokenizer results into word stems. Regular stemming can affect the semantic meaning of words, as several words with different meanings might have the same root stem. Light stemming only removes frequently used prefixes and suffixes, and doesn't produce the root of a word to preserve semantics. | | stemmer\_es\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Castilian Spanish language tokenizer results into word stems. | | stemmer\_fi\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Finnish language tokenizer results into word stems. | | stemmer\_fr\_light | Uses light stemming to convert French language tokenizer results into word stems. Regular stemming can affect the semantic meaning of words, as several words with different meanings might have the same root stem. Light stemming only removes frequently used prefixes and suffixes, and doesn't produce the root of a word to preserve semantics. | | stemmer\_fr\_min | Uses minimal stemming to convert French language tokenizer results. Minimal stemming only removes the last character of a word or replaces some suffixes. For example, the stemmer\_fr\_min removes x, s, r, e, and é characters from the end of words and replaces the aux suffix with al. | | stemmer\_fr\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert French language tokenizer results into word stems. | | stemmer\_hi | Uses a [lightweight stemmer for Hindi](https://www.semanticscholar.org/paper/A-Lightweight-Stemmer-for-Hindi-Ramanathan-Rao/041a475a2b30b3f4397405e45098b40177e39de1) to remove suffixes from tokenizer results. | | stemmer\_hr | Uses an open source stemming rule set to find the root word in Croatian language tokenizer results. | | stemmer\_hu\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Hungarian language tokenizer results into word stems. | | stemmer\_it\_light | Uses light stemming to convert Italian language tokenizer results into word stems. Regular stemming can affect the semantic meaning of words, as several words with different meanings might have the same root stem. Light stemming only removes frequently used prefixes and suffixes, and doesn't produce the root of a word to preserve semantics. | | stemmer\_it\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Italian language tokenizer results into word stems. | | stemmer\_nl\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Dutch language tokenizer results into word stems. | | stemmer\_no\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Norwegian language tokenizer results into word stems. | | stemmer\_porter | Transforms tokenizer results with the porter stemming algorithm. For more information, see the [official Porter Stemming Algorithm documentation](https://tartarus.org/martin/PorterStemmer/). | | stemmer\_pt\_light | Uses light stemming to convert Portuguese language tokenizer results into word stems. Regular stemming can affect the semantic meaning of words, as several words with different meanings might have the same root stem. Light stemming only removes frequently used prefixes and suffixes, and doesn't produce the root of a word to preserve semantics. | | stemmer\_ro\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Romanian language tokenizer results into word stems. | | stemmer\_ru\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Russian language tokenizer results into word stems. | | stemmer\_sv\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Swedish language tokenizer results into word stems. | | stemmer\_tr\_snowball | Uses the [Snowball string processing language](https://snowballstem.org/) to convert Turkish language tokenizer results into word stems. | | stop\_ar | Removes tokens from tokenizer results that are unnecessary for a search, based on an Arabic dictionary. | | stop\_bg | Removes tokens from tokenizer results that are unnecessary for a search, based on a Bulgarian dictionary. | | stop\_ca | Removes tokens from tokenizer results that are unnecessary for a search, based on a Catalan dictionary. | | stop\_ckb | Removes tokens from tokenizer results that are unnecessary for a search, based on a Kurdish dictionary. | | stop\_cs | Removes tokens from tokenizer results that are unnecessary for a search, based on a Czech dictionary. | | stop\_da | Removes tokens from tokenizer results that are unnecessary for a search, based on a Danish dictionary. | | stop\_de | Removes tokens from tokenizer results that are unnecessary for a search, based on a German dictionary. | | stop\_el | Removes tokens from tokenizer results that are unnecessary for a search, based on a Greek dictionary. | | stop\_en | Removes tokens from tokenizer results that are unnecessary for a search, based on an English dictionary. For example, the token filter removes and, is, and the from tokenizer results. | | stop\_es | Removes tokens from tokenizer results that are unnecessary for a search, based on a Castilian Spanish dictionary. | | stop\_eu | Removes tokens from tokenizer results that are unnecessary for a search, based on a Basque dictionary. | | stop\_fa | Removes tokens from tokenizer results that are unnecessary for a search, based on a Persian dictionary. | | stop\_fi | Removes tokens from tokenizer results that are unnecessary for a search, based on a Finnish dictionary. | | stop\_fr | Removes tokens from tokenizer results that are unnecessary for a search, based on a French dictionary. | | stop\_ga | Removes tokens from tokenizer results that are unnecessary for a search, based on a Gaelic dictionary. | | stop\_gl | Removes tokens from tokenizer results that are unnecessary for a search, based on a Galician Spanish dictionary. | | stop\_he | Removes tokens from tokenizer results that are unnecessary for a search, based on a Hebrew dictionary. | | stop\_hi | Removes tokens from tokenizer results that are unnecessary for a search, based on a Hindi dictionary. | | stop\_hr | Removes tokens from tokenizer results that are unnecessary for a search, based on a Croatian dictionary. | | stop\_hu | Removes tokens from tokenizer results that are unnecessary for a search, based on a Hungarian dictionary. | | stop\_hy | Removes tokens from tokenizer results that are unnecessary for a search, based on an Armenian dictionary. | | stop\_id | Removes tokens from tokenizer results that are unnecessary for a search, based on an Indonesian dictionary. | | stop\_it | Removes tokens from tokenizer results that are unnecessary for a search, based on an Italian dictionary. | | stop\_nl | Removes tokens from tokenizer results that are unnecessary for a search, based on a Dutch dictionary. | | stop\_no | Removes tokens from tokenizer results that are unnecessary for a search, based on a Norwegian dictionary. | | stop\_pt | Removes tokens from tokenizer results that are unnecessary for a search, based on a Portuguese dictionary. | | stop\_ro | Removes tokens from tokenizer results that are unnecessary for a search, based on a Romanian dictionary. | | stop\_ru | Removes tokens from tokenizer results that are unnecessary for a search, based on a Russian dictionary. | | stop\_sv | Removes tokens from tokenizer results that are unnecessary for a search, based on a Swedish dictionary. | | stop\_tr | Removes tokens from tokenizer results that are unnecessary for a search, based on a Turkish dictionary. | | to\_lower | Converts all characters in tokens to lowercase. | | unique | Removes any tokens that aren't unique. |