Cite a source for every answer

How to do it

Give each retrieved chunk a short ID in the prompt. Ask for the answer and a list of cited IDs in the output schema. Drop any cited ID that was not retrieved, because the model invented it. If no valid citation remains, do not show the answer as fact. Show each source's title and address under the answer. The security angle is in ground answers in your own data.

Example

const Answer = z.object({ text: z.string(), citedIds: z.array(z.string()) });

export function keepValidCitations(answer: z.infer<typeof Answer>, retrieved: Chunk[]) {
  const byId = new Map(retrieved.map(chunk => [chunk.id, chunk]));
  const sources = answer.citedIds.filter(id => byId.has(id)).map(id => byId.get(id)!);
  return { ...answer, sources, grounded: sources.length > 0 };
}

class Answer(BaseModel):
    text: str
    cited_ids: list[str]

def keep_valid_citations(answer: Answer, retrieved: list[Chunk]) -> dict:
    by_id = {chunk.id: chunk for chunk in retrieved}
    sources = [by_id[i] for i in answer.cited_ids if i in by_id]
    return {"text": answer.text, "sources": sources, "grounded": bool(sources)}

How to check

A unit test feeds the citation step an answer citing one retrieved and one unknown ID. Only the retrieved one is shown. An eval run shows every answer that states a fact has at least one valid citation.