---
title: "Keep text in UTF-8 at every layer · Agent Wranglers"
url: https://agentwranglers.se/playbook/localization/text-handling/utf8-everywhere
id: LOC-53
level: Required
risk: data
updated: 2026-10-22
build: a2a6bdfed830
tokens: 877
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Keep text in UTF-8 at every layer

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Text handling](https://agentwranglers.se/playbook/localization/text-handling.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-22.

- ID: LOC-53
- Level: Required
- Chapter: [Text handling](https://agentwranglers.se/playbook/localization/text-handling.md)
- Rule: Store, send and read all text as UTF-8 in every layer, from forms and APIs to the database, files, email and exports. In MySQL and MariaDB use `utf8mb4`, never `utf8`.
- Check: A round-trip test sends "Åsa Ødegård 😀 中文" through a form, the API, the database, an export and an email. Every copy reads back byte for byte the same. A schema test finds `utf8mb4` on every text column, and the CSV export starts with the bytes `EF BB BF`.
- Risk: Data leaked or lost
- How: One layer in another encoding turns "Åsa" into "Ã…sa" or loses an emoji, and the damage stays in the data. In MySQL and MariaDB, `utf8` is an alias for `utf8mb3`, which holds at most three bytes per character. Set the server, database, tables, columns and connection to `utf8mb4`. Create PostgreSQL databases with `ENCODING 'UTF8'`. Send `charset=utf-8` in every text `Content-Type`, and declare it first in HTML, as [Declare UTF-8 first](https://agentwranglers.se/playbook/website/html-basics/meta-charset.md) says. Read uploads and imported files as UTF-8, and reject invalid bytes with a clear error instead of guessing. Write CSV files meant for Excel with a UTF-8 byte order mark, because Excel otherwise opens them in an older code page. Never put a byte order mark in JSON, which RFC 8259 forbids.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### SQL example

```sql
-- MySQL or MariaDB
ALTER DATABASE shop CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;
ALTER TABLE customers CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;
```

### Python example

```python
import csv
# utf-8-sig writes the byte order mark Excel needs
with open("customers.csv", "w", encoding="utf-8-sig", newline="") as file:
    csv.writer(file).writerows(rows)
```

### JavaScript example

```js
// Node.js: Excel reads the file as UTF-8 only with the byte order mark
res.type('text/csv; charset=utf-8').send('\uFEFF' + toCsv(rows));
```

## Sources

- [WHATWG Encoding Standard](https://encoding.spec.whatwg.org/)
- [RFC 3629 (UTF-8)](https://www.rfc-editor.org/rfc/rfc3629)
- [MySQL – The utf8mb4 Character Set](https://dev.mysql.com/doc/refman/8.4/en/charset-unicode-utf8mb4.html)

## Related rules

- WEB-3 [Declare UTF-8 first](https://agentwranglers.se/playbook/website/html-basics/meta-charset.md) in Website

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/localization/text-handling/utf8-everywhere">Agent Wranglers localization playbook</a>`. In code you write, add a comment next to it with the same link.

- Next: [Sort and compare text by locale](https://agentwranglers.se/playbook/localization/text-handling/locale-collation.md)
- [Text handling](https://agentwranglers.se/playbook/localization/text-handling.md)