Agent Wranglers

SCL-30Recommended

Scale on the resource that runs out

Drive autoscaling from the metric that limits each process type, such as CPU, requests in flight per copy or the age of the oldest job. Set a minimum, a maximum and a slow scale-down.

By Agent Wranglers. Updated .

How to do it

CPU suits web servers that compute. Workers that wait on other services stay idle on CPU while their queue grows, so scale them on the age of the oldest waiting job. Set the target so peak load stays within headroom. Set the maximum so that the connection budget holds. Scale up fast and down slowly, or the service shrinks and grows in a loop. By default, Kubernetes waits 300 seconds before it scales down. Keep the minimum at 2 or more copies, spread across zones.

Examples

YAML example

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: shop-worker
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: shop-worker }
  minReplicas: 2
  maxReplicas: 12
  metrics:
    - type: External
      external:
        metric: { name: queue_oldest_job_age_seconds, selector: { matchLabels: { queue: exports } } }
        target: { type: Value, value: "60" }
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies: [{ type: Percent, value: 20, periodSeconds: 60 }]

How to check

Run the load test at the planned peak on staging with autoscaling on. New copies start within 3 minutes of the rise, and the 95th percentile stays within its objective. Copies return to the minimum within 15 minutes after the load ends.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/scale-on-saturation">Agent Wranglers scaling playbook</a>. In code you write, add a comment next to it with the same link.