---
title: "Set the language of every search field · Agent Wranglers"
url: https://agentwranglers.se/playbook/data/search/search-language
updated: 2026-10-21
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Set the language of every search field

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Search](https://agentwranglers.se/playbook/data/search.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers). Updated 2026-10-21.

- ID: DAT-63
- Level: Recommended
- Chapter: [Search](https://agentwranglers.se/playbook/data/search)
- Rule: Choose the language rules for each searched field on purpose. Test word endings, accents and typos with a fixed list of word pairs.
- How: In PostgreSQL, name the configuration in every `to_tsvector` and `to_tsquery` call, such as `'english'` or `'swedish'`. An expression index on `to_tsvector` serves only queries that use the same configuration name. Store a language column per row when content mixes languages. The `unaccent` dictionary makes "Hôtel" match "hotel". For typos, build a word list with `ts_stat` and match misspelled terms against it with `pg_trgm` similarity. In MySQL, natural language mode drops stopwords, so very common words never match. In SQLite FTS5, the `porter` and `unicode61` tokenizers handle word endings and accents. Sorting text for people is in [Sort and compare text by locale](https://agentwranglers.se/playbook/website/languages-and-regions/locale-collation).
- Check: A test file holds at least 20 word pairs per supported language, such as "running" and "run", "Hôtel" and "hotel", or "recieve" and "receive". Searching each word of a pair returns the same record in the top 10.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### SQL example

```sql
-- PostgreSQL: accent-insensitive French search
CREATE EXTENSION IF NOT EXISTS unaccent;
CREATE TEXT SEARCH CONFIGURATION shop_fr (COPY = french);
ALTER TEXT SEARCH CONFIGURATION shop_fr
  ALTER MAPPING FOR hword, hword_part, word WITH unaccent, french_stem;

-- PostgreSQL: suggest a spelling from the words in the catalogue
CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE TABLE product_words AS
  SELECT word FROM ts_stat('SELECT to_tsvector(''simple'', description) FROM products');
CREATE INDEX product_words_trgm ON product_words USING GIN (word gin_trgm_ops);
SELECT word FROM product_words WHERE word % $1 ORDER BY similarity(word, $1) DESC LIMIT 3;
```

### SQL example 2

```sql
-- SQLite: word endings and accents in FTS5
CREATE VIRTUAL TABLE product_search USING fts5(
  name, description, tokenize = 'porter unicode61 remove_diacritics 2');
```

## Sources

- [PostgreSQL text search tables and indexes](https://www.postgresql.org/docs/current/textsearch-tables.html)
- [PostgreSQL unaccent module](https://www.postgresql.org/docs/current/unaccent.html)
- [PostgreSQL pg_trgm module](https://www.postgresql.org/docs/current/pgtrgm.html)

## Related rules

- [Sort and compare text by locale](https://agentwranglers.se/playbook/website/languages-and-regions/locale-collation) in Website

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/data/search/search-language">Agent Wranglers data playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Index only the fields people search](https://agentwranglers.se/playbook/data/search/search-index-fields)
- Next: [Cap the size of every search](https://agentwranglers.se/playbook/data/search/search-limits)
- [Search](https://agentwranglers.se/playbook/data/search)