Agent WranglersCaching
Cache only what is read often and costly to build. Key it fully, let one caller rebuild an entry, and keep the app up when the cache is gone.
- Required 2
- Recommended 4
Required
Cap the cache server's memory and choose its eviction
SCL-14Set a memory limit and an eviction policy on every cache server. Keep data that must not be evicted, such as job queues, in a separate instance that never evicts.
Keep the app working when the cache is gone
SCL-15Treat every cache call as optional. On a cache error or timeout, read from the source, so the service stays up when the cache is down or empty.
Recommended
Cache what is read often and costly to build
SCL-11Add a cache only for a read that statistics or traces show is frequent and slow. Measure each cache's hit rate, and remove caches that mostly miss.
Put a format version in every cache key
SCL-12Build each server cache key from a prefix, the data's format version and every input that changes the value. Add the tenant or user for private data.
Let one caller rebuild an expired entry
SCL-13When a hot entry expires, let one caller rebuild it while the others wait for that result or get the old value. Add random jitter to every expiry time.
Decide how stale a cache may be
SCL-35Give every cached value an expiry or an invalidation rule, chosen for how stale it may safely be.
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/scaling/caching">Agent Wranglers scaling playbook</a>. In code you write, add a comment next to it with the same link.