Agent Wranglers

TNS-21Recommended

Mark automated decisions and review a sample of them

Record on every decision whether automated means found the item and whether they decided it. Each week, give a person a random sample of 1% of automated removals and 1% of items the filters passed, with the filter's verdict hidden. Take at least 20 of each, or all of them when there are fewer, and at most 100.

By Agent Wranglers. Updated .

How to do it

The DSA Transparency Database asks two fields of each statement of reasons. automated_detection is yes or no, and automated_decision is fully, partially or not automated. Store both on the decision row, so statements and reports come from the data. Appeals in Have a second person decide each appeal show only the errors that users contest. A blind sample also finds wrong removals nobody appealed and harmful items the filter missed. Compute the share of wrong removals and of missed items per filter and reason. Send a filter's decisions to a person while its wrong removals pass 5%, as Let a person decide what matters says. Put the rates in the yearly moderation report.

Examples

JavaScript example

const SAMPLE_SHARE = 0.01;
const MIN_SAMPLE = 20;
const MAX_SAMPLE = 100;

export const sampleSize = total => Math.min(total, MAX_SAMPLE, Math.max(MIN_SAMPLE, Math.ceil(total * SAMPLE_SHARE)));

export async function weeklyReviewSample() {
  const removed = decisions.automated({ sinceDays: 7 });
  const passed = items.passedByFilters({ sinceDays: 7 });
  const picked = [
    ...await removed.sample(sampleSize(await removed.count())),
    ...await passed.sample(sampleSize(await passed.count())),
  ];
  for (const item of picked) await reviewTasks.create({ itemId: item.itemId, showVerdict: false });
}

Python example

SAMPLE_SHARE = 0.01
MIN_SAMPLE = 20
MAX_SAMPLE = 100

def sample_size(total: int) -> int:
    return min(total, MAX_SAMPLE, max(MIN_SAMPLE, math.ceil(total * SAMPLE_SHARE)))

def weekly_review_sample():
    removed = decisions.automated(since_days=7)
    passed = items.passed_by_filters(since_days=7)
    picked = [*removed.sample(sample_size(removed.count())), *passed.sample(sample_size(passed.count()))]
    for item in picked:
        review_tasks.create(item_id=item.item_id, show_verdict=False)

How to check

Every decision row from the last 30 days holds both automation fields. A unit test of the weekly job gives it 500 removals and 50,000 passed items and gets 20 and 100 review tasks. With 8 removals it takes all 8. No task shows the filter's verdict. A dashboard shows the error rates per filter for the last four weeks.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/trust-and-safety/running-trust-and-safety/automated-decision-sampling">Agent Wranglers trust and safety playbook</a>. In code you write, add a comment next to it with the same link.