Run code the model writes in a sandbox

How to do it

Use a separate container, micro virtual machine or WebAssembly runtime for each run, and delete it afterwards. Mount only the files the task needs, read-only where you can. Turn networking off, or allow a short list of hosts. Never run model code with eval or in the app's own process. The sandbox rule for coding agents covers agents that write your source code.

Example

docker run --rm --network none --read-only --memory 256m --cpus 0.5 --pids-limit 64 \
  --user 65534 --cap-drop ALL --security-opt no-new-privileges \
  -v "$PWD/task:/task:ro" sandbox-image timeout 10 python /task/main.py

# Bad: model code runs inside the app, with its keys and network
exec(model_code)
# Good: a fresh sandbox per run, with a time limit
subprocess.run(["docker", "run", "--rm", "--network", "none", "sandbox-image",
                "timeout", "10", "python", "-c", model_code], timeout=15, check=False)

// Bad: model code runs inside the app
eval(modelCode);
// Good: a fresh sandbox per run, with a time limit
execFile('docker', ['run', '--rm', '--network', 'none', 'sandbox-image', 'timeout', '10', 'node', '-e', modelCode], { timeout: 15_000 });

How to check

Ask the feature to run code that prints the environment, reads /etc/hostname, calls https://test.example.org and loops forever. No secret appears, the request never arrives, and the loop is stopped at the time limit.