Agent WranglersLatency, cost and fallbacks Optional
Send work nobody waits for as a batch
Run large jobs that no person waits for, such as nightly tagging or re-summarising documents, through the provider's batch interface.
How to do it
Several providers offer a batch interface that returns results within a day at a lower price, often half the normal one. Examples are the OpenAI Batch API, the Anthropic Message Batches API and Gemini batch mode. Results can come back in any order. Give each request your own ID, such as custom_id, and match results by that ID, never by position. Check every result as you would a live one, and retry failed items in the next batch.
How to check
The job's log shows it was sent as a batch, and its cost per item is below the live price for the same model.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/batch-offline-work.