Log which prompt made each answer

How to do it

Have the gateway add these fields to its log line and return them with the result. Save them in the same row as any AI output you store, such as a draft reply or a summary. When a user reports a bad answer, you can then rerun the exact prompt and model. The full list of log fields is in log every model call.

Example

SELECT count(*) FROM ai_outputs
WHERE created_at > now() - interval '1 day'
  AND (prompt_id IS NULL OR prompt_version IS NULL OR model_version IS NULL);
-- PASS when the count is 0

await db.aiOutputs.insert({ orderId, text: answer.text, promptId: prompt.id, promptVersion: prompt.version, modelVersion: answer.model });

db.ai_outputs.insert(order_id=order_id, text=answer.text, prompt_id=prompt.id,
                     prompt_version=prompt.version, model_version=answer.model)

How to check

Query the stored AI outputs from the last day. Every row has a prompt ID, a prompt version and a model version, and none is empty.