Limit what any one source can put into training data

How to do it

Poisoning hides a few crafted rows in the data, so the model learns a backdoor or a wrong label for one case. It works best when an attacker controls many rows, such as through user feedback, scraped pages or an outside dataset. Record the source of every row, as record where every dataset came from asks. Count rows per source and per account. Score every row with the current model and flag sources whose labels disagree with it at more than twice the overall rate. A person reviews flagged sources before the run continues. Feedback that reaches training goes through approve user feedback before it trains or teaches the model. Outside models and datasets follow use models and data from trusted sources.

Examples

Python example

MAX_SOURCE_SHARE = 0.01

def sources_over_cap(snapshot: pd.DataFrame) -> list[str]:
    shares = snapshot["source_id"].value_counts(normalize=True)
    return shares[shares > MAX_SOURCE_SHARE].index.tolist()

TypeScript example

const MAX_SOURCE_SHARE = 0.01;

export function sourcesOverCap(rows: { sourceId: string }[]): string[] {
  const counts = Map.groupBy(rows, row => row.sourceId);
  return [...counts].filter(([, group]) => group.length / rows.length > MAX_SOURCE_SHARE).map(([sourceId]) => sourceId);
}

How to check

A test adds rows from one account above the cap to a snapshot, and the pipeline stops and names the account. A second test flips the labels of one source, and the pipeline flags that source.

Sources

• NIST AI 100-2 E2025, Adversarial Machine Learning
• OWASP LLM05:2026 Data and Model Poisoning

Related rules

• MLO-4 Record where every dataset came from and why you may use it in Machine learning
• AIF-69 Approve user feedback before it trains or teaches the model in AI features
• APS-77 Use models and data from trusted sources in Application security