Search by meaning and by words

How to do it

Vector search finds text with the same meaning in other words. Keyword search, such as BM25, finds exact product names, codes and error messages that vectors often miss. Run both, take about 20 results from each, and merge them by rank. Reciprocal rank fusion scores each result as the sum of 1 divided by 60 plus its rank in each list. Optionally rerank the top results with a reranking model. Keep the merge only if it scores better in the retrieval evals.

Examples

TypeScript example

const RRF_K = 60;

export function reciprocalRankFusion(...rankings: string[][]): string[] {
  const scores = new Map<string, number>();
  for (const ranking of rankings) ranking.forEach((id, rank) => scores.set(id, (scores.get(id) ?? 0) + 1 / (RRF_K + rank + 1)));
  return [...scores.entries()].sort((a, b) => b[1] - a[1]).map(([id]) => id);
}

Python example

RRF_K = 60

def reciprocal_rank_fusion(*rankings: list[str]) -> list[str]:
    scores: dict[str, float] = {}
    for ranking in rankings:
        for rank, chunk_id in enumerate(ranking, start=1):
            scores[chunk_id] = scores.get(chunk_id, 0) + 1 / (RRF_K + rank)
    return sorted(scores, key=scores.get, reverse=True)

How to check

The retrieval eval set includes questions with exact product codes. Hybrid search scores at least as well as vector search alone on the whole set.

Sources

• Azure AI Search, hybrid search overview

Related rules

• STK-88 Use pgvector in PostgreSQL before a separate vector database in Software choices
• SRH-17 Test search results with a fixed query set in Search
• SRH-3 Search by meaning when the collection is small in Search