Agent WranglersEngines and indexes
Search inside the database first, add meaning with vectors when tests show a gain, record the embedding model, and split long documents by their structure.
- Recommended 4
- Optional 1
Recommended
Search with the database's own full-text index
SRH-1Serve keyword search from a full-text index inside the database. Never search long text with
LIKE '%word%', which reads every row.Search by meaning and by words
SRH-2Combine a keyword search and a vector search, and merge the two result lists, for example with reciprocal rank fusion.
Record the embedding model with every vector
SRH-4Store the embedding model name and version with every vector. Never compare vectors from two different embedding models.
Split documents along their structure
SRH-5Split documents into chunks along headings and paragraphs, of 200 to 800 tokens each. Store each chunk with its document ID, title, heading path and address.
Optional
Search by meaning when the collection is small
SRH-3When a site has 5,000 items or fewer, rank search results by meaning as well as by words.
Read this with
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/search/engines-and-indexes">Agent Wranglers search playbook</a>. In code you write, add a comment next to it with the same link.