Agent Wranglers

MLO-44Required

Remove erased people from every model trained on their data

When a person's data must be erased, drop their rows from every snapshot used for future training. Retrain each model that learned from them within a set time, such as 90 days, unless its membership test shows it holds no personal data.

By Agent Wranglers. Updated .

How to do it

Deleting the source row does not remove what a model learned. A model that is not anonymous is personal data itself, so erasure reaches it too. Keep a list of erased person keys, and filter it out of every new snapshot, as freeze each training set as a versioned snapshot allows. Map each person key to the snapshots and models that used it. A model that passes test that the model does not give away its training records may wait for its next scheduled retrain. Others retrain within the set time. Methods that make a model forget without retraining are still research, so do not rely on them alone. Language model stores follow erase a person's data from every AI store.

Examples

Python example

def build_snapshot(rows: pd.DataFrame, erased_keys: set[str]) -> pd.DataFrame:
    return rows[~rows["person_key"].isin(erased_keys)]

def models_due_for_retrain(models: list[ModelVersion], today: date) -> list[ModelVersion]:
    return [m for m in models if m.has_erased_rows and not m.membership_test_passed
            and (today - m.first_erasure_date).days > ERASURE_RETRAIN_DAYS]

TypeScript example

export function buildSnapshot(rows: TrainingRow[], erasedKeys: Set<string>): TrainingRow[] {
  return rows.filter(row => !erasedKeys.has(row.personKey));
}

How to check

A test erases a person and builds a new snapshot, and none of their rows appear in it. A script lists every production model trained on a snapshot holding erased rows. It fails when one is older than the set time and has no passing membership test.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/machine-learning/people-affected-by-the-model/erasure-from-models">Agent Wranglers machine learning playbook</a>. In code you write, add a comment next to it with the same link.