Let a person decide what matters

How to do it

List the decisions each feature touches, such as credit, hiring, insurance claims, account bans, refunds over a limit or medical advice. For those, the model only drafts or ranks, and a person approves in a review screen that shows the inputs and sources. Record who approved and when. In the EU, GDPR Article 22 limits decisions based solely on automated processing, and many uses in the AI Act's high-risk list need human oversight. Actions an agent takes need a person's approval too.

Example

SELECT count(*) FROM claim_decisions WHERE ai_suggested AND (reviewed_by IS NULL OR reviewed_at IS NULL);
-- PASS when the count is 0

export async function decideClaim(claimId: string, reviewer: Staff, decision: 'approve' | 'reject') {
  if (!reviewer.roles.includes('claims-reviewer')) throw new Forbidden();
  await claims.update(claimId, { decision, reviewedBy: reviewer.id, reviewedAt: new Date() });
}

def decide_claim(claim_id: str, reviewer: Staff, decision: Literal["approve", "reject"]) -> None:
    if "claims-reviewer" not in reviewer.roles:
        raise Forbidden()
    claims.update(claim_id, decision=decision, reviewed_by=reviewer.id, reviewed_at=datetime.now(UTC))

How to check

For each decision on the list, a test tries to complete it with no reviewer. It fails. Every completed decision in the database has a reviewer ID and a time.