Sort and compare text by locale

How to do it

Sort lists with a locale collator, such as Intl.Collator in JavaScript, PyICU in Python or java.text.Collator. Byte order puts "Ö" in the wrong place, and the right place differs by language. Normalise input to NFC, so "é" typed two ways compares equal. Change case with the locale, because Turkish lower-cases "I" to "ı".

Examples

JavaScript example

const words = ['ä', 'a', 'z', 'o', 'ö'];
words.sort(new Intl.Collator('sv').compare); // ["a", "o", "z", "ä", "ö"]
words.sort(new Intl.Collator('de').compare); // ["a", "ä", "o", "ö", "z"]
'Cafe\u0301'.normalize('NFC') === 'Café'; // true

Python example

import unicodedata
from icu import Collator, Locale
swedish = Collator.createInstance(Locale("sv_SE"))
sorted(["ä", "a", "z", "o", "ö"], key=swedish.getSortKey)  # ['a', 'o', 'z', 'ä', 'ö']
unicodedata.normalize("NFC", "Cafe\u0301") == "Café"  # True

Java example

import java.text.Collator;
import java.text.Normalizer;
import java.util.*;
List<String> words = new ArrayList<>(List.of("ä", "a", "z", "o", "ö"));
words.sort(Collator.getInstance(Locale.forLanguageTag("sv")));  // [a, o, z, ä, ö]
Normalizer.normalize("Cafe\u0301", Normalizer.Form.NFC).equals("Café");  // true

How to check

A unit test sorts a fixed list per locale and gets the expected order. Swedish puts "ä" after "z", and German puts it after "a".

Sources

• UTS #10 (Unicode Collation Algorithm)
• UAX #15 (Unicode Normalization Forms)

Related rules

• DAT-43 Rebuild text indexes when the collation library changes in Data and databases
• DAT-63 Set the language of every search field in Data and databases
• APP-35 Sort and filter on the server in Web app interface