---
title: "Split documents along their structure · Agent Wranglers"
url: https://agentwranglers.se/playbook/search/engines-and-indexes/structured-chunks
id: SRH-5
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 841
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Split documents along their structure

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Engines and indexes](https://agentwranglers.se/playbook/search/engines-and-indexes.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: SRH-5
- Level: Recommended
- Chapter: [Engines and indexes](https://agentwranglers.se/playbook/search/engines-and-indexes.md)
- Rule: Split documents into chunks along headings and paragraphs, of 200 to 800 tokens each. Store each chunk with its document ID, title, heading path and address.
- Check: A test chunks the sample documents. Every chunk has 200 to 800 tokens or is a whole short section. Each ends at a sentence end and has a document ID, heading path and address.
- How: Split at headings first, then at paragraphs, and never in the middle of a sentence, table row or code block. Add a small overlap, such as one sentence, only where a section must be cut. Prefix each chunk's text with its document title and heading path before embedding, so a chunk about "Returns" still says which product it covers. Keep the address, so answers can link to the source. Measure each change to the chunking as [Measure retrieval on its own](https://agentwranglers.se/playbook/search/relevance/search-quality-tests.md) says.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### TypeScript example

```ts
type Chunk = { documentId: string; url: string; headingPath: string[]; text: string };

export function chunkByHeadings(doc: { id: string; url: string; title: string; sections: Section[] }): Chunk[] {
  return doc.sections.flatMap(section =>
    splitParagraphs(section.paragraphs, { maxTokens: 800 }).map(text => ({
      documentId: doc.id, url: `${doc.url}#${section.anchor}`, headingPath: [doc.title, ...section.headings],
      text: `${[doc.title, ...section.headings].join(' > ')}\n\n${text}`,
    })));
}
```

### Python example

```python
def chunk_by_headings(doc: Document) -> list[Chunk]:
    chunks = []
    for section in doc.sections:
        path = [doc.title, *section.headings]
        for text in split_paragraphs(section.paragraphs, max_tokens=800):
            chunks.append(Chunk(document_id=doc.id, url=f"{doc.url}#{section.anchor}",
                                heading_path=path, text=" > ".join(path) + "\n\n" + text))
    return chunks
```

## Related rules

- SRH-17 [Test search results with a fixed query set](https://agentwranglers.se/playbook/search/relevance/search-quality-tests.md) in Search

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/search/engines-and-indexes/structured-chunks">Agent Wranglers search playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Record the embedding model with every vector](https://agentwranglers.se/playbook/search/engines-and-indexes/embedding-model-version.md)
- [Engines and indexes](https://agentwranglers.se/playbook/search/engines-and-indexes.md)