Agent Wranglers

Rules of engagement Required

Turn every fix into a check

When a hole is closed, add the attack as an automated test, so the hole can never come back unnoticed.

By Erik Bergstedt, Agent Wranglers. Updated .

How to do it

Write the proof from the report as a test that expects the safe answer. Run it with the other tests before every deploy. See check every rule in code.

Example

test('a user cannot read another user\'s order', async () => {
  const res = await fetch(`${base}/api/orders/1002`, { headers: { authorization: `Bearer ${tokenOfUser1001}` } });
  assert.equal(res.status, 404);
});
def test_a_user_cannot_read_another_users_order(client, token_of_user_1001):
    res = client.get("/api/orders/1002", headers={"Authorization": f"Bearer {token_of_user_1001}"})
    assert res.status_code == 404

How to check

Undo the fix on a branch. The new test fails.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers security testing playbook: https://agentwranglers.se/playbook/adversarial-testing/rules-of-engagement/fix-becomes-check.