Record how every model was trained

How to do it

Without these, nobody can rebuild the model or explain why the new one differs. Log them from the training code with a tracking tool, such as MLflow, or as a JSON file next to the model. Refuse to train from a working tree with uncommitted changes. Set seeds for every random source, such as Python, NumPy and the framework. PyTorch states that results can still differ across releases and platforms, so pin the container image too. The data side is Freeze each training set as a versioned snapshot. Fine-tuned language models follow Rebuild fine-tuned models from stored data and compare before switching.

Examples

Python example

import random, subprocess
import mlflow, numpy as np, torch

SEED = 20261022

def start_run(snapshot_id: str, params: dict) -> None:
    if subprocess.run(["git", "status", "--porcelain"], capture_output=True, text=True).stdout:
        raise RuntimeError("commit your changes before training")
    random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
    torch.use_deterministic_algorithms(True)
    commit = subprocess.run(["git", "rev-parse", "HEAD"], capture_output=True, text=True).stdout.strip()
    mlflow.start_run()
    mlflow.log_params({**params, "seed": SEED, "snapshot_id": snapshot_id, "git_commit": commit, "torch": torch.__version__})

TypeScript example

const REQUIRED = ['snapshot_id', 'git_commit', 'seed', 'image', 'params', 'libraries', 'hardware'] as const;

export const runRecordProblems = (record: Record<string, unknown>): string[] =>
  REQUIRED.filter(field => record[field] === undefined || record[field] === '').map(field => `run record lacks ${field}`);

How to check

Retrain the current production model from its run record alone in a clean job. Its test score is within the tolerance written in the record, and the record has no empty field.

Sources

• PyTorch reproducibility notes
• MLflow tracking

Related rules

• MLO-6 Freeze each training set as a versioned snapshot in Machine learning
• AIF-62 Rebuild fine-tuned models from stored data and compare before switching in AI features