Say so when nothing relevant was found

How to do it

Pick a threshold from the retrieval evals, such as the lowest score that still found a right document in 95 of 100 cases. Below it, skip the model and send a fixed reply with a link to search or support. Log these questions, because they show which documents are missing.

Example

if (!hits.length || hits[0].score < MIN_RELEVANCE) {
  log.info({ event: 'retrieval.no_match', feature: 'help-search' });
  return { text: NO_MATCH_REPLY, sources: [] };
}

if not hits or hits[0].score < MIN_RELEVANCE:
    log.info("retrieval.no_match", feature="help-search")
    return {"text": NO_MATCH_REPLY, "sources": []}

How to check

Ask a question the documents do not cover, such as one about a product you do not sell. The reply says the documents do not cover it, and the log shows a no-match event.