Agent Wranglers

TNS-5Recommended

Limit what moderators must see of harmful content

Blur images and mute videos in the moderation tool until the moderator chooses to reveal them. Cap each moderator's hours on severity 1 queues per day.

By Agent Wranglers. Updated .

How to do it

Repeated exposure to violent or abusive material harms the people who review it. Show the text of a report, the reason and any automated label first, so many cases are decided without opening the media. Reveal one item at a time, in greyscale by default. Match uploads against hash lists of known abuse material first, so known items never reach a person. Report child sexual abuse material and keep the evidence explains how. Rotate people between severe and mild queues, and give them a way to step away without a penalty. A small team can buy review of the worst queues from a specialist firm that has these protections.

Examples

JavaScript example

const SEVERE_QUEUE_CAP_MINUTES = 4 * 60;

export const canTakeSevereCase = moderator => moderator.severeMinutesToday < SEVERE_QUEUE_CAP_MINUTES;

Python example

SEVERE_QUEUE_CAP_MINUTES = 4 * 60

def can_take_severe_case(moderator) -> bool:
    return moderator.severe_minutes_today < SEVERE_QUEUE_CAP_MINUTES

How to check

A browser test opens a case with an image in the moderation tool, and the image is blurred until the reveal control is pressed. The queue assignment refuses a severity 1 case to a moderator who has passed the daily cap.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/trust-and-safety/reports-and-queues/moderator-exposure">Agent Wranglers trust and safety playbook</a>. In code you write, add a comment next to it with the same link.