Record the embedding model with every vector

How to do it

Vectors from different models, or different versions of one model, live in different spaces. A search across both returns nonsense without any error. Add a column for the model to the vector table, and filter every search on the current model. A changed output size, such as the dimensions setting of some embedding APIs, counts as a new model. An approximate index, such as HNSW in pgvector, can apply filters after the scan and return fewer rows than asked. Turn on iterative index scans or keep one index per tenant, and test that a filtered search fills its limit. To change models, embed everything again into a new index, run the retrieval evals on it, then switch. Delete the old vectors afterwards.

Example

SELECT embedding_model, count(*) FROM chunks GROUP BY embedding_model;
-- PASS when one row is returned and it matches the configured model

const hits = await db.query(
  'SELECT id, text, url FROM chunks WHERE embedding_model = $1 AND tenant_id = $2 ORDER BY embedding <=> $3 LIMIT 20',
  [config.embeddingModel, user.tenantId, toVector(await embed(question))],
);

hits = db.execute(
    "SELECT id, text, url FROM chunks WHERE embedding_model = %s AND tenant_id = %s ORDER BY embedding <=> %s LIMIT 20",
    (settings.embedding_model, user.tenant_id, to_vector(embed(question))),
).fetchall()

How to check

A query counts vectors per embedding model in the live index. There is exactly one model, and it matches the model in the settings. A filtered search for a small tenant with at least 20 chunks returns 20 rows.