Agent WranglersFlaky and slow tests Recommended
Keep the suite within a time budget
Set a time limit for the whole suite and for each kind of test, and treat going over it as a failure.
How to do it
Run tests in parallel, such as pytest-xdist -n auto, Playwright workers or sharding across machines. Track the slowest tests and fix them first. A suite that takes too long gets skipped, which is worse than a smaller suite.
Example
pytest -n auto --durations=20
npx playwright test --shard=1/4How to check
The pipeline records each stage's duration. The run fails when a stage passes its budget.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers testing playbook: https://agentwranglers.se/playbook/testing/flaky-and-slow-tests/suite-time-budget.