Test both states of every live flag

How to do it

A flag doubles the paths through the code. The path nobody runs in staging is the one that breaks on the day of the switch. Read flags through one provider that a test can set, never from global state, as Pass state in instead of sharing globals says. Take the flags from the flag register, and run each affected test once per state with the runner's table of cases. A flag with more than two values needs a case for each value. Once a flag is on everywhere, delete it with its off cases, as Remove flags that never change says.

Examples

JavaScript example

// Vitest: the same checks run with the flag on and off
describe.each(['on', 'off'])('new-shipping-rates %s', (state) => {
  test('a cart over 500 ships free', () => {
    const flags = fakeFlags({ 'new-shipping-rates': state === 'on' });
    expect(checkoutTotal({ subtotal: 520 }, flags).shipping).toBe(0);
  });
});

Python example

# pytest: one case per flag state, named in the report
@pytest.mark.parametrize("enabled", [True, False], ids=["on", "off"])
def test_a_cart_over_500_ships_free(enabled):
    flags = FakeFlags({"new-shipping-rates": enabled})
    assert checkout_total(Cart(subtotal=520), flags).shipping == 0

How to check

For every flag in the register, the test report shows passing tests named with on and with off. Breaking the off path of a flag makes a test fail.

Related rules

• ENG-45 Keep every flag in one register in Engineering
• ARC-38 Pass state in instead of sharing globals in Clean code and architecture
• TST-11 Test rules with several conditions as a table in Testing
• ARC-60 Remove flags that never change in Clean code and architecture