---
title: "Have a second person decide each appeal · Agent Wranglers"
url: https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals/appeal-review
id: TNS-10
level: Recommended
updated: 2026-10-22
tokens: 944
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Have a second person decide each appeal

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Decisions and appeals](https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-22.

- ID: TNS-10
- Level: Recommended
- Chapter: [Decisions and appeals](https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals.md)
- Rule: Give every appeal to a moderator who did not make the first decision. Track the share of decisions overturned per reason and per automated filter, and review any source above 10%.
- Check: A unit test assigns an appeal and never picks the first decider. A weekly report lists the overturn rate per reason and per filter for the last 90 days. It flags any above 10% with at least 20 appeals.
- How: The person who removed an item rarely reverses it. Assign appeals away from the first decider in the queue code. A team of one asks a second person outside the team, such as a founder. An appeal against an automated decision always reaches a person, as [Let a person decide what matters](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/human-decides.md) says. An upheld appeal restores the item, removes the strike and tells the user. The overturn rate is the best measure of moderation quality you have. A filter whose decisions are often overturned needs a new threshold, and a reason that is often overturned needs clearer wording in the [policy table](https://agentwranglers.se/playbook/trust-and-safety/reports-and-queues/policy-table.md).

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### JavaScript example

```js
const OVERTURN_ALERT_RATE = 0.1;
const MIN_APPEALS = 20;

export const pickAppealReviewer = (moderators, firstDeciderId) =>
  moderators.find(m => m.id !== firstDeciderId && m.available);

export const sourcesToReview = stats =>
  stats.filter(s => s.appeals >= MIN_APPEALS && s.overturned / s.appeals > OVERTURN_ALERT_RATE);
```

### Python example

```python
OVERTURN_ALERT_RATE = 0.1
MIN_APPEALS = 20

def pick_appeal_reviewer(moderators, first_decider_id):
    return next((m for m in moderators if m.id != first_decider_id and m.available), None)

def sources_to_review(stats: list[dict]) -> list[dict]:
    return [s for s in stats if s["appeals"] >= MIN_APPEALS and s["overturned"] / s["appeals"] > OVERTURN_ALERT_RATE]
```

## Sources

- [Digital Services Act Article 20](https://eur-lex.europa.eu/eli/reg/2022/2065/oj)

## Related rules

- AIF-36 [Let a person decide what matters](https://agentwranglers.se/playbook/ai-features/guardrails-and-human-review/human-decides.md) in AI features
- TNS-2 [Write the moderation policy as one table of reasons and actions](https://agentwranglers.se/playbook/trust-and-safety/reports-and-queues/policy-table.md) in Trust and safety

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals/appeal-review">Agent Wranglers trust and safety playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Hide removed content and keep it until appeals close](https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals/removed-content-hold.md)
- [Decisions and appeals](https://agentwranglers.se/playbook/trust-and-safety/decisions-and-appeals.md)