Agent Wranglers

TNS-16Recommended

Watch abuse rates and alert on jumps

Track sign-ups, the share of sign-ups held, reports per 1,000 new items and messages per new account, each per hour. Alert when one passes three times its average for the same hour over the last four weeks.

By Agent Wranglers. Updated .

How to do it

Abuse waves start at night and on holidays, when nobody reads the queue. Compare each hour with the same hour of the same weekday, so normal peaks do not page anyone. A jump in sign-ups with a rising held share points at a bot. A jump in reports points at a raid or a bad post going viral. Send the alert to the person on call, who may flip one of the switches that stop an abuse wave. Usage of costly free features has its own alert in Protect free tiers from automated abuse.

Examples

JavaScript example

const ALERT_FACTOR = 3;

export const ratesToAlert = (current, baseline) =>
  Object.keys(current).filter(rate => baseline[rate] > 0 && current[rate] > ALERT_FACTOR * baseline[rate]);

Python example

ALERT_FACTOR = 3

def rates_to_alert(current: dict, baseline: dict) -> list[str]:
    return [rate for rate, value in current.items() if baseline.get(rate, 0) > 0 and value > ALERT_FACTOR * baseline[rate]]

How to check

Replay a test hour on staging with five times the usual sign-ups from one network. The alert fires within 15 minutes and names the rate. A normal Monday peak in the replayed data fires no alert.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/trust-and-safety/running-trust-and-safety/abuse-rate-alerts">Agent Wranglers trust and safety playbook</a>. In code you write, add a comment next to it with the same link.