Scaling playbook

A scaling playbook for apps that start to get busy. It starts with measuring, to find the bottleneck and write down the load you plan for. Then it covers stateless app servers, caching, background jobs and queues, and growing the database. It ends with load balancing, autoscaling, fair shares per customer and cost per request. Most apps never need half of it, so measure first and scale the part that is slow.

Chapters

1. Measure before you scale

   Find the resource that runs out first from metrics and profiles. Write down the load you plan for before you add servers or caches.

   • Recommended 5

2. Stateless app servers

   Any copy of the app can serve any request, start fast and stop without losing work. Then you can add or remove copies at any time.

   • Required 1
   • Recommended 4
   • Avoid 1

3. Caching

   Cache only what is read often and costly to build. Key it fully, let one caller rebuild an entry, and keep the app up when the cache is gone.

   • Required 2
   • Recommended 4

4. Background jobs and queues

   Move slow work out of requests into queues. Make every job safe to run twice, bound the backlog, and keep urgent work ahead of bulk work.

   • Required 2
   • Recommended 7

5. Growing the database

   Budget connections across every copy, keep reports off the primary, partition and archive big tables, and change large tables in small batches.

   • Required 2
   • Recommended 4
   • Avoid 1

6. Load balancing and autoscaling

   Send traffic only to ready servers and spread it by load. Scale on the resource that runs out, cap each customer's share and watch cost per request.

   • Recommended 5

All 38 rules on one page

This is a lot to check. An Expert can check it for you.

Related playbooks

• Capacity and cost

  In the Operations playbook. Watch saturation and spend in production, and add capacity before a limit is reached.

• Non-functional tests

  In the Testing playbook. Prove a change scales with a load test before users find the limit.

• Queries and indexes

  In the Data playbook. Most slow apps are slow queries. Fix the query plan before adding servers.

• Performance

  In the Website playbook. Fast pages and HTTP caching take load off the servers before it reaches them.