Agent Wranglers

MLO-37Required

Explain each decision to the person it affects

When model output shapes a decision about a person, such as credit, a job or an account ban, give them an explanation on request. It names the data used, how it counted and what would have changed the result.

How to do it

GDPR Articles 13 to 15 give people "meaningful information about the logic involved" in automated decisions. In case C-203/22 of February 2025, the EU Court of Justice ruled that sending the algorithm is not enough. The controller must describe the procedure and principles actually applied, so the person can understand and contest the decision. The Court named the effect of changed data on the result as one useful way to explain. Generate reason codes from the model, such as the three features that pushed the score most, and turn them into plain sentences. AI Act Article 86 adds a right to an explanation from the deployer of a high-risk system, so run the high-risk check. A person must review significant decisions, as Let a person decide what matters says. US rules add notice and appeal in Give notice and an appeal before automated decisions.

Examples

Python example

REASON_TEXT = {
    "refund_count_90d": "You asked for {value} refunds in the last 90 days.",
    "account_age_days": "Your account is {value} days old.",
    "amount_eur": "The refund is for {value} EUR.",
}

def explanation(contributions: dict[str, float], features: dict, model_version: str) -> dict:
    top = sorted(contributions, key=contributions.get, reverse=True)[:3]
    return {"model_version": model_version, "inputs": features,
            "reasons": [REASON_TEXT[name].format(value=features[name]) for name in top]}

TypeScript example

const REASON_TEXT: Record<string, (value: unknown) => string> = {
  refund_count_90d: v => `You asked for ${v} refunds in the last 90 days.`,
  account_age_days: v => `Your account is ${v} days old.`,
  amount_eur: v => `The refund is for ${v} EUR.`,
};

export function explanation(contributions: Record<string, number>, features: Record<string, unknown>, modelVersion: string) {
  const top = Object.keys(contributions).sort((a, b) => contributions[b] - contributions[a]).slice(0, 3);
  return { modelVersion, inputs: features, reasons: top.map(name => REASON_TEXT[name](features[name])) };
}

How to check

A test requests the explanation for a stored decision. It names the model version, the input values used and at least one reason with a change that would flip the outcome. It uses plain words and no feature codes.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/machine-learning/people-affected-by-the-model/explain-decisions">Agent Wranglers machine learning playbook</a>. In code you write, add a comment next to it with the same link.