Agent Wranglers

Retrieval Required

Copy access rights into the index and check them again

Store each document's access rights with its chunks, filter every search by them, and re-check the source's rights before a chunk reaches the model.

How to do it

Copy the owner, team or group list from the source onto each chunk. Filter the search query by the user's groups, as scope what the model can look up requires. Rights change faster than indexes, so check the remaining hits against the source system at answer time. Update stored rights on every permission change event.

Example

const candidates = await index.search(question, { filter: { groups: { anyOf: user.groups } }, limit: 20 });
const allowed = await documents.filterReadable(candidates.map(c => c.documentId), user);
const chunks = candidates.filter(c => allowed.has(c.documentId));
candidates = index.search(question, filter={"groups": {"any_of": user.groups}}, limit=20)
allowed = documents.filter_readable({c.document_id for c in candidates}, user)
chunks = [c for c in candidates if c.document_id in allowed]

How to check

Give a test user access to a document, ask about it, then remove the access. The next question returns nothing from that document, even before the index updates.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/retrieval/retrieval-permissions.