---
title: "Catch speed regressions on every pull · Agent Wranglers"
url: https://agentwranglers.se/playbook/testing/performance-tests/benchmark-regressions
id: TST-61
level: Recommended
updated: 2026-10-22
build: a2a6bdfed830
tokens: 893
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Catch speed regressions on every pull request

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Performance tests](https://agentwranglers.se/playbook/testing/performance-tests.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-22.

- ID: TST-61
- Level: Recommended
- Chapter: [Performance tests](https://agentwranglers.se/playbook/testing/performance-tests.md)
- Rule: Benchmark the hot code paths on every pull request against the main branch on the same machine. Fail when one gets slower by more than a set margin.
- Check: The pull request job runs the benchmarks on both branches and posts the comparison. Add a deliberate slowdown, such as a sleep of 20 percent of the run time, to a benchmarked function on a branch. The job fails and names the function.
- How: A load test before launch finds a slow release weeks after the change that caused it. Benchmark the few functions and queries that most requests pass through, such as price calculation, search ranking or page rendering. Run the main branch and the pull request on the same runner, one after the other, so machine noise cancels out. Repeat each benchmark at least 10 times and compare with a statistical tool, such as `benchstat` for Go or the compare options of pytest-benchmark. Set a margin above the normal noise, such as 10 percent. Count allocations or database queries as well where the tool can, because those numbers do not drift with machine load. Keep the full load test for releases, as [Test load against a budget](https://agentwranglers.se/playbook/testing/performance-tests/load-tests.md) says.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### Shell example

```sh
# Go: run both branches on the same runner, then compare
git switch main && go test -run='^$' -bench=. -count=10 ./pricing > old.txt
git switch - && go test -run='^$' -bench=. -count=10 ./pricing > new.txt
benchstat old.txt new.txt
```

### Shell example 2

```sh
# Python with pytest-benchmark: save the main branch, then fail the branch on a 10 percent slowdown
git switch main && pytest benchmarks/ --benchmark-autosave
git switch - && pytest benchmarks/ --benchmark-compare --benchmark-compare-fail=mean:10%
```

### Go example

```go
func BenchmarkCartTotal(b *testing.B) {
    cart := sampleCart(50)
    b.ReportAllocs()
    for b.Loop() {
        cart.Total()
    }
}
```

### Python example

```python
def test_cart_total_speed(benchmark):
    cart = sample_cart(items=50)
    benchmark(cart.total)
```

## Sources

- [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat)

## Related rules

- TST-36 [Test load against a budget](https://agentwranglers.se/playbook/testing/performance-tests/load-tests.md) in Testing

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/testing/performance-tests/benchmark-regressions">Agent Wranglers testing playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Hold normal load for hours to find leaks](https://agentwranglers.se/playbook/testing/performance-tests/soak-tests.md)
- [Performance tests](https://agentwranglers.se/playbook/testing/performance-tests.md)