Prove one request cannot stall the site

How to do it

One request that builds a large file and compresses it inside the only worker blocks every other visitor. The proxy then answers 504, but the app keeps working, so the whole site stays down until it finishes. Find the most expensive endpoints from Server-Timing headers, traces or response sizes. Fix any that stall the site by building the file ahead of time, as Build large exports ahead of time says. Or move the work out of the request, as Move slow work out of the request says. Run the test in CI against the built app on staging, so a new slow endpoint cannot ship.

Examples

JavaScript example

const SMALL_PAGE = 'https://staging.example.com/contact';
const HEAVY_REQUEST = 'https://staging.example.com/export/orders.csv';
const LIMIT_MS = 200;
const PROBE_INTERVAL_MS = 1000;

async function timedFetch(url) {
  const start = performance.now();
  await (await fetch(url)).arrayBuffer();
  return performance.now() - start;
}

let heavyDone = false;
const heavy = timedFetch(HEAVY_REQUEST).finally(() => { heavyDone = true; });
let slowest = 0;
while (!heavyDone) {
  slowest = Math.max(slowest, await timedFetch(SMALL_PAGE));
  await new Promise(resolve => setTimeout(resolve, PROBE_INTERVAL_MS));
}
await heavy;
if (slowest > LIMIT_MS) throw new Error(`The small page took ${Math.round(slowest)} ms during the heavy request`);

Python example

import threading
import time

import httpx

SMALL_PAGE = "https://staging.example.com/contact"
HEAVY_REQUEST = "https://staging.example.com/export/orders.csv"
LIMIT_SECONDS = 0.2
PROBE_INTERVAL_SECONDS = 1


def test_a_heavy_request_does_not_stall_small_pages():
    heavy = threading.Thread(target=httpx.get, args=(HEAVY_REQUEST,), kwargs={"timeout": 120})
    heavy.start()
    slowest = 0.0
    while heavy.is_alive():
        start = time.monotonic()
        httpx.get(SMALL_PAGE, timeout=10)
        slowest = max(slowest, time.monotonic() - start)
        time.sleep(PROBE_INTERVAL_SECONDS)
    assert slowest < LIMIT_SECONDS

How to check

During each heavy request, a small page keeps answering within 200 ms, or twice its normal time when that is slower. An outside check stays green. The script exits with an error otherwise.

Sources

• Don't block the Node.js event loop
• OWASP Denial of Service Cheat Sheet

Related rules

• TST-53 Run load tests from a written runbook in Testing
• SCL-39 Build large exports ahead of time in Scaling
• SCL-16 Move slow work out of the request in Scaling