Agent Wranglers

Languages and regions Recommended

Sort and compare text by locale

Sort and compare visible text with a collator for the reader's locale, and normalise Unicode before comparing or storing it.

How to do it

Sort lists with a locale collator, such as Intl.Collator in JavaScript, PyICU in Python or java.text.Collator. Byte order puts "Ö" in the wrong place, and the right place differs by language. Normalise input to NFC, so "é" typed two ways compares equal. Change case with the locale, because Turkish lower-cases "I" to "ı".

Example

const words = ['ä', 'a', 'z', 'o', 'ö'];
words.sort(new Intl.Collator('sv').compare); // ["a", "o", "z", "ä", "ö"]
words.sort(new Intl.Collator('de').compare); // ["a", "ä", "o", "ö", "z"]
'Cafe\u0301'.normalize('NFC') === 'Café'; // true
import unicodedata
from icu import Collator, Locale
swedish = Collator.createInstance(Locale("sv_SE"))
sorted(["ä", "a", "z", "o", "ö"], key=swedish.getSortKey)  # ['a', 'o', 'z', 'ä', 'ö']
unicodedata.normalize("NFC", "Cafe\u0301") == "Café"  # True
import java.text.Collator;
import java.text.Normalizer;
import java.util.*;
List<String> words = new ArrayList<>(List.of("ä", "a", "z", "o", "ö"));
words.sort(Collator.getInstance(Locale.forLanguageTag("sv")));  // [a, o, z, ä, ö]
Normalizer.normalize("Cafe\u0301", Normalizer.Form.NFC).equals("Café");  // true

How to check

A unit test sorts a fixed list per locale and gets the expected order. Swedish puts "ä" after "z", and German puts it after "a".

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers website playbook: https://agentwranglers.se/playbook/website/languages-and-regions/locale-collation.