Agent WranglersSCL-22Recommended
Spread scheduled work across the hour
Give each scheduled job its own start minute instead of the full hour. Split jobs that touch every customer into batches with gaps between them.
How to do it
Jobs set to 0 * * * * all start together and make a load peak every hour. Many teams pick midnight, so outside services are busy then too. Pick an odd minute per job, such as 17 or 43. A job over all customers works in batches, as Change large tables in small batches shows. Run each once, as Run each scheduled job once at a time says, and report a heartbeat.
Examples
Plain text example
# crontab for a made-up shop: one minute per job
17 * * * * /srv/shop/bin/run-job sync-stock
43 * * * * /srv/shop/bin/run-job expire-carts
11 3 * * * /srv/shop/bin/run-job nightly-invoicesHow to check
No two jobs that run longer than 1 minute share a start minute in the schedule. Over the last 7 days, database CPU at minute 0 of each hour is under 1.5 times the hourly average.
Related rules
- Change large tables in small batches in Scaling
- Run each scheduled job once at a time in Scaling
- Alert when a scheduled job stops running in Operations
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/scaling/background-jobs-and-queues/spread-scheduled-jobs">Agent Wranglers scaling playbook</a>. In code you write, add a comment next to it with the same link.