Make sure tests an agent wrote do not just repeat the code

How to do it

An agent asked to add tests often writes ones that pass whatever the code does. It computes the expected value by calling the same function, or copies the current output into the assertion. It may also assert on a mock it set up two lines earlier. Such tests raise coverage and catch nothing. Write expected values as literals worked out by hand or from the requirement. For example, 2 items at 10.00 with 25 percent VAT total 25.00. Run mutation testing on the changed files, because a tautological test lets every mutant survive. Review the agent's diff for weakened tests, as Look for weakened checks and skipped work says.

Examples

Python example

# Bad: the expected value comes from the code under test, so it can never fail
def test_order_total():
    order = Order(items=[Item(price=10), Item(price=10)], vat_rate=0.25)
    assert order.total() == order.total()

# Bad: only checks that the fake returns what it was told
def test_charge(mocker):
    gateway = mocker.Mock(charge=mocker.Mock(return_value="ok"))
    assert gateway.charge(25) == "ok"

# Good: the expected value is worked out by hand from the requirement
def test_order_total_includes_vat():
    order = Order(items=[Item(price=10), Item(price=10)], vat_rate=0.25)
    assert order.total() == 25

TypeScript example

// Bad: calcTotal is both the code under test and the source of the answer
test('order total', () => {
  const order = { items: [{ price: 10 }, { price: 10 }], vatRate: 0.25 };
  expect(calcTotal(order)).toBe(calcTotal(order));
});

// Good
test('order total includes VAT', () => {
  expect(calcTotal({ items: [{ price: 10 }, { price: 10 }], vatRate: 0.25 })).toBe(25);
});

How to check

Mutation testing on the files a pull request changes meets the project's mutation score floor. Every new test kills at least one mutant. A lint or review script fails an assertion whose expected side calls the function under test.

Related rules

• TST-31 Prove the tests catch bugs in Testing
• AID-18 Look for weakened checks and skipped work in AI-assisted development