Agent Wranglers

SCL-10Recommended

Run web and worker processes apart

Run request handling and background work as separate process types from the same build, so each scales on its own.

By Agent Wranglers. Updated .

How to do it

The Twelve-Factor App calls these process types, such as web and worker. A slow export then cannot take the CPU that pages need. You can also add workers without adding web servers. Give each type its own count, its own limits on memory and processes and its own scale rule. Run scheduled jobs as a third type, once at a time as Run each scheduled job once at a time says.

Examples

YAML example

# compose.yaml: one image, two process types for a made-up shop
services:
  web:
    image: registry.example.com/shop:1.42.0
    command: ["node", "server.js"]
    deploy: { replicas: 3, resources: { limits: { memory: 512M } } }
  worker:
    image: registry.example.com/shop:1.42.0
    command: ["node", "worker.js"]
    deploy: { replicas: 2, resources: { limits: { memory: 1G } } }

How to check

The deploy settings list web and worker as separate types with separate counts. Change the worker count on staging, and no web process restarts.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/scaling/stateless-app-servers/web-and-worker-processes">Agent Wranglers scaling playbook</a>. In code you write, add a comment next to it with the same link.