Agent Wranglers

Recommended

Check rules that hold for any input with property tests

When a rule must hold for every input, such as a round trip or a total, state it once as a property. Let a tool generate hundreds of inputs to test it.

By Agent Wranglers. Updated .

How to do it

Look for three kinds of rule. Encoding and then decoding gives back the input. A result stays inside its limits, such as a discount that never passes the price. Two ways to compute one value agree, such as a fast version and a simple one. Write each as a property test with a library such as Hypothesis for Python, fast-check for JavaScript or jqwik for Java. On a failure, the tool shrinks the input to a small case that still fails. Hypothesis runs 100 inputs per test by default and replays saved failures on the next run. Copy each failure into a plain example test, so every machine runs it. Splitting money is a common case, as Split amounts without losing a cent shows.

Examples

Using these examples: they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

Python example

from hypothesis import given, strategies as st

@given(price_cents=st.integers(min_value=0, max_value=1_000_000), percent=st.integers(min_value=0, max_value=100))
def test_a_discount_never_passes_the_price(price_cents, percent):
    assert 0 <= apply_discount(price_cents, percent) <= price_cents

@given(order=st.builds(Order, total=st.integers(min_value=0), currency=st.sampled_from(["EUR", "SEK"])))
def test_an_order_survives_a_round_trip(order):
    assert parse_order(serialise_order(order)) == order

JavaScript example

import fc from 'fast-check';
import { expect, test } from 'vitest';

test('a discount never passes the price', () => {
  fc.assert(fc.property(fc.integer({ min: 0, max: 1_000_000 }), fc.integer({ min: 0, max: 100 }), (priceCents, percent) => {
    const discounted = applyDiscount(priceCents, percent);
    return discounted >= 0 && discounted <= priceCents;
  }));
});

test('an order survives a round trip', () => {
  const anyOrder = fc.record({ id: fc.uuid(), total: fc.nat(), currency: fc.constantFrom('EUR', 'SEK') });
  fc.assert(fc.property(anyOrder, order => {
    expect(parseOrder(serialiseOrder(order))).toEqual(order);
  }));
});

How to check

Every function on the risk list that converts, encodes or calculates amounts has at least one property test. Break its rule on purpose, such as rounding one way only. The property test fails and prints a shrunk input.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/testing/unit-tests/property-tests">Agent Wranglers testing playbook</a>. In code you write, add a comment next to it with the same link.