Agent Wranglers

TKO-8Recommended

Pin today's behaviour before changing code

Before you change a module, write tests that record what it does today for the inputs it gets in production. They must pass on the code as it is now.

By Agent Wranglers. Updated .

How to do it

Michael Feathers named these characterization tests. They state what the code does, not what it should do. Call the code with typical and odd inputs, save each output as the expected value, and review the saved files once. Approval or snapshot tools do the saving, such as Jest or Vitest snapshots, syrupy for pytest, or the ApprovalTests libraries. Draw the inputs from production shapes, with personal data removed as Keep personal data out of test copies says. For the whole output of a system, use Prove a refactor changes nothing. Keep each saved output small, as Keep snapshot tests small and reviewed says.

Examples

TypeScript example

import { test, expect } from 'vitest';
import { quoteShipping } from '../src/legacy/shipping';

const ORDERS = [
  { weightKg: 0.4, country: 'SE' },
  { weightKg: 31, country: 'SE' },
  { weightKg: 2, country: 'NO' },
  { weightKg: 0, country: 'XX' },
];

test.each(ORDERS)('shipping quote today for %o', order => {
  expect(quoteShipping(order)).toMatchSnapshot();
});

Python example

import pytest
from legacy.shipping import quote_shipping

ORDERS = [
    {"weight_kg": 0.4, "country": "SE"},
    {"weight_kg": 31, "country": "SE"},
    {"weight_kg": 2, "country": "NO"},
    {"weight_kg": 0, "country": "XX"},
]

@pytest.mark.parametrize("order", ORDERS)
def test_shipping_quote_today(order, snapshot):
    assert quote_shipping(**order) == snapshot

How to check

Run the new tests against the base commit of the change. They pass. Every pull request that changes a module with no tests adds its characterization tests in an earlier commit.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/codebase-takeover/characterization-tests/pin-behaviour-first">Agent Wranglers codebase takeover playbook</a>. In code you write, add a comment next to it with the same link.