Keep every model in a registry with versions and aliases

How to do it

Use a registry, such as the MLflow Model Registry, or a bucket with one folder per version and a small alias file. Store the model file's hash, the threshold, the evaluation report and the model card with each version. Never overwrite a version. Only the release gate may move the production alias. Serving reads the alias at start and reports the version it loaded, so every log line names it. Load files only from the registry, in a safe format, as Use models and data from trusted sources says.

Examples

Python example

from mlflow import MlflowClient

client = MlflowClient()

def promote(model_name: str, version: str) -> None:
    client.set_registered_model_alias(model_name, "champion", version)

def roll_back(model_name: str, previous_version: str) -> None:
    client.set_registered_model_alias(model_name, "champion", previous_version)

TypeScript example

type Registry = { versionForAlias(model: string, alias: string): Promise<{ version: string; uri: string; sha256: string }> };

export async function loadChampion(registry: Registry, model: string, load: (uri: string, sha256: string) => Promise<unknown>) {
  const entry = await registry.versionForAlias(model, 'champion');
  const loaded = await load(entry.uri, entry.sha256);
  console.info(JSON.stringify({ event: 'model_loaded', model, version: entry.version }));
  return { version: entry.version, model: loaded };
}

How to check

Move the alias to the previous version in a staging environment. Serving reports the old version within the time your runbook states, and no code change or deploy was needed.

Sources

• MLflow Model Registry

Related rules

• MLO-3 Choose the threshold from the cost of errors in Machine learning
• MLO-19 Gate every release on an offline evaluation in Machine learning
• SEC-91 Use models and data from trusted sources in Adversarial testing
• MLO-24 Check every input against what the model was trained on in Machine learning
• MLO-27 Release models through a canary that compares predictions in Machine learning
• MLO-38 Keep a record of every decision about a person in Machine learning