Logs Required
Tie every line to its request
Give every request an ID, pass it to every service it touches, and write it on every log line and error page.
How to do it
At the edge, accept an incoming traceparent header only if it matches the W3C format, otherwise create a new random ID. Keep the ID in request context, add it to every log line, and return it in a response header. Pass it on every outgoing call. Show it on error pages, so support can find the exact lines.
Example
import { AsyncLocalStorage } from 'node:async_hooks';
import { randomUUID } from 'node:crypto';
const TRACEPARENT = /^00-([0-9a-f]{32})-[0-9a-f]{16}-[0-9a-f]{2}$/;
export const requestContext = new AsyncLocalStorage();
app.use((req, res, next) => {
const requestId = TRACEPARENT.exec(req.get('traceparent') ?? '')?.[1] ?? randomUUID().replaceAll('-', '');
res.set('X-Request-Id', requestId);
requestContext.run({ requestId, log: log.child({ request_id: requestId }) }, next);
});
import re
import uuid
import structlog
TRACEPARENT = re.compile(r"^00-([0-9a-f]{32})-[0-9a-f]{16}-[0-9a-f]{2}$")
@app.middleware("http")
async def tag_request(request, call_next):
match = TRACEPARENT.match(request.headers.get("traceparent", ""))
request_id = match.group(1) if match else uuid.uuid4().hex
structlog.contextvars.clear_contextvars()
structlog.contextvars.bind_contextvars(request_id=request_id)
response = await call_next(request)
response.headers["X-Request-Id"] = request_id
return response
How to check
Trigger an error on staging. The error page shows an ID, and searching the logs for it returns every line of that request, across services.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers operations playbook: https://agentwranglers.se/playbook/operations/logs/request-ids.