# How Do I Run Many AI Agent Instances in Production?

> Keep the agent's control loop wherever you already run services, and give each task that runs code its own short-lived VM. A small orchestrator takes tasks from a queue, creates one labeled sandbox with a TTL per task, runs it, records output and exit code, and deletes the sandbox. Your plan's concurrency and start limits bound how far it scales.

Source: https://sandboxer.com/guides/run-many-agent-instances-in-production. Updated 2026-11-06.

**Short answer:** Keep the agent's control loop wherever you already run services, and give each task that runs code its own short-lived VM. A small orchestrator takes tasks from a queue, creates one labeled sandbox with a TTL per task, runs it, records output and exit code, and deletes the sandbox. Your plan's concurrency and start limits bound how far it scales.

"Many agents in production" is usually two different workloads. One is the agent service itself: a process that takes requests, calls a model and calls tools. The other is the work those agents do when they run code: installing packages, running tests, executing a script the model wrote. The first scales like any web service. The second needs a clean, isolated machine per task, and something that creates, watches and removes those machines. This page builds that something, runs it on prod, and shows what it leaves behind when tasks fail.

If you want several coding agents working on one repo, start with [run coding agents in parallel](/guides/run-coding-agents-in-parallel). If you want N attempts at one task and the best result, see [best-of-N with parallel forks](/guides/best-of-n-agents-with-parallel-forks). This page is the generic queue runner underneath both.

## The pattern

1. Read a queue of tasks.
2. For each task, create a sandbox with a **TTL** (it stops itself even if your process dies) and **labels** (run id, task number), using an `Idempotency-Key` so a retried create never makes a second one.
3. Wait for `ready`, run the task with a timeout, and save stdout, stderr, the exit code and the elapsed time.
4. Delete the sandbox. Or, for failed tasks, keep it and mark it so you can inspect it and clean up by label later.
5. Run tasks in parallel, capped below your plan's concurrency limit.

## The orchestrator

The script below is bash with `curl`, `jq` and the [`sbx` CLI](/docs/cli). Labels are set through the REST API ([labels](/docs/api#labels)): `sbx new` has no label flag, and `sbx ls` filters only by state, so the create and the label query go through `curl`. The run itself uses `sbx exec --json`, which returns `stdout`, `stderr`, `exitCode` and `timedOut` in one object and exits with the command's own code. The script reads `SANDBOXER_API_KEY` from the environment and passes it to `curl` through a file descriptor, so it never appears in the process list. It uses GNU `date`.

```bash
#!/usr/bin/env bash
# One sandbox per task, run in parallel, results collected, sandboxes deleted.
#   ./orchestrate.sh tasks.txt        # one shell command per line
# Env: RUN_ID, NAME_PREFIX, MAX_PARALLEL, TTL (s), TASK_TIMEOUT (s, max 600), KEEP_FAILED=1
set -uo pipefail
API=${SANDBOXER_API_URL:-https://sandboxer.com/api/v1}
RUN=${RUN_ID:-run-$(date -u +%Y%m%d-%H%M%S)}
PREFIX=${NAME_PREFIX:-agent}
TTL=${TTL:-900}                  # backstop: each sandbox stops itself after 15 min
TASK_TIMEOUT=${TASK_TIMEOUT:-120}
MAX_PARALLEL=${MAX_PARALLEL:-10}  # keep this below your plan's "active at once"
OUT=results/$RUN; mkdir -p "$OUT"

api() {  # the key goes in through a file descriptor, not the process's arguments
  curl -sS -H @<(printf 'Authorization: Bearer %s\n' "$SANDBOXER_API_KEY") \
    -H 'Content-Type: application/json' "$@"
}

create() {  # $1 = task number. Prints the sandbox id. Exit 75 = no capacity, try later.
  local n=$1 body resp code wait
  body=$(jq -nc --arg name "$PREFIX-$n" --arg run "$RUN" --arg task "$n" --argjson ttl "$TTL" \
    '{type:"small", name:$name, ttlSeconds:$ttl,
      labels:{owner:"orchestrator-demo", run:$run, task:$task}}')
  for attempt in 1 2 3; do
    resp=$(api -X POST "$API/sandboxes" -H "Idempotency-Key: $RUN-$n" \
      -D "$OUT/$n.headers" -w '\n%{http_code}' -d "$body")
    code=${resp##*$'\n'}; resp=${resp%$'\n'*}
    case $code in
      20?) jq -r '.sandbox.id // .id' <<<"$resp"; return 0 ;;
      429) wait=$(awk 'tolower($1)=="retry-after:"{print $2+0}' "$OUT/$n.headers")
           sleep "${wait:-10}" ;;                 # start or concurrency limit: wait, retry
      503) jq -c .error <<<"$resp" >&2; return 75 ;;  # capacity_exceeded: don't loop
      5??) sleep $((attempt * 2)) ;;              # same Idempotency-Key, so no duplicate
      *)   jq -c .error <<<"$resp" >&2; return 1 ;;
    esac
  done
  return 1
}

wait_ready() {  # poll until ready, about once a second, for at most 3 minutes
  local state
  for _ in $(seq 180); do
    state=$(api "$API/sandboxes/$1" | jq -r '.sandbox.state // .state')
    case $state in ready) return 0 ;; error|archived|cancelled) return 1 ;; esac
    sleep 1
  done
  return 1
}

run_task() {  # $1 = task number, $2 = command. Writes $OUT/$1.json.
  local n=$1 cmd=$2 t0 id res code outcome rc
  t0=$(date +%s%N)
  id=$(create "$n" 2>"$OUT/$n.err"); rc=$?
  if [ $rc -ne 0 ]; then
    outcome=$([ $rc -eq 75 ] && echo no_capacity || echo create_failed)
    jq -nc --arg n "$n" --arg o "$outcome" --rawfile e "$OUT/$n.err" \
      '{task:$n, outcome:$o, error:$e}' >"$OUT/$n.json"; return
  fi
  if wait_ready "$id"; then
    res=$(sbx exec "$id" --json --timeout "$TASK_TIMEOUT" "$cmd" 2>"$OUT/$n.err"); code=$?
  else
    res='{}'; code=1
  fi
  [ -n "$res" ] || res='{}'
  outcome=$(jq -r --argjson c "$code" 'if .timedOut then "timeout"
    elif $c == 0 then "ok" else "failed" end' <<<"$res")
  jq -c --arg n "$n" --arg id "$id" --arg o "$outcome" --argjson c "$code" \
    --argjson ms "$(( ($(date +%s%N) - t0) / 1000000 ))" \
    '{task:$n, sandbox:$id, outcome:$o, exit:$c, seconds:($ms / 100 | round / 10),
      stdout:(.stdout // "" | rtrimstr("\n")), stderr:(.stderr // "" | rtrimstr("\n"))}' \
    <<<"$res" >"$OUT/$n.json"
  if [ "$outcome" != ok ] && [ "${KEEP_FAILED:-0}" = 1 ]; then
    # Keep it to debug, and mark it so a cleanup job can find it (PATCH replaces all labels).
    api -X PATCH "$API/sandboxes/$id" -o /dev/null -d "$(jq -nc --arg run "$RUN" --arg task "$n" \
      --arg o "$outcome" '{labels:{owner:"orchestrator-demo", run:$run, task:$task, outcome:$o}}')"
  else
    sbx delete "$id" --yes --json >/dev/null 2>&1 || echo "delete failed: $id" >&2
  fi
}

n=0
while IFS= read -r cmd; do
  [ -n "$cmd" ] || continue
  while [ "$(jobs -rp | wc -l)" -ge "$MAX_PARALLEL" ]; do wait -n; done
  n=$((n + 1)); run_task "$n" "$cmd" &
done <"${1:?usage: orchestrate.sh tasks.txt}"
wait
jq -s -c 'sort_by(.task|tonumber)[]' "$OUT"/*.json
```

In a real system, each line of `tasks.txt` would be an agent's tool call or a job from your queue. The `owner` label should be a value only your app uses, so cleanup never matches someone else's sandboxes.

## Running it on three tasks

We ran it on November 6, 2026 from a laptop outside Sandboxer, with three tasks chosen to cover the three outcomes: one succeeds, one exits non-zero, one runs past the timeout.

```text
python3 -c 'import platform, hashlib; print("python", platform.python_version()); print(hashlib.sha256(b"x" * 50_000_000).hexdigest()[:16])'
test -f input.csv || { echo "input.csv not found" >&2; exit 3; }
sleep 300 && echo "never printed"
```

```bash
time RUN_ID=run-20261106-a NAME_PREFIX=aeo-agentscale TASK_TIMEOUT=20 KEEP_FAILED=1 ./orchestrate.sh tasks.txt
```

```json
{"task":"1","sandbox":"sb_fx9rgxfsfv","outcome":"ok","exit":0,"seconds":1.5,"stdout":"python 3.12.3\n6e937662ccf4d140","stderr":""}
{"task":"2","sandbox":"sb_tfxdu07y4q","outcome":"failed","exit":3,"seconds":1.4,"stdout":"","stderr":"input.csv not found"}
{"task":"3","sandbox":"sb_mb6pho9gmw","outcome":"timeout","exit":124,"seconds":21.4,"stdout":"","stderr":""}
```

```text
real	0m21.524s
```

`seconds` is create-to-result: the create call, waiting for `ready`, and the command. It doesn't include the delete. On this run, a new `small` sandbox went from create to a finished short command in about 1.5 seconds. Task 3's 21.4 seconds is the same startup plus the 20-second timeout. The three ran at once, so the whole run took as long as the slowest task. Treat these as one measurement on one day, not a guarantee: start time depends on capacity at that moment.

Task 1's sandbox was deleted by the script, and `sbx info sb_fx9rgxfsfv` then exited `4` with `"code":"not_found","status":404`.

## What a failed task leaves behind

With `KEEP_FAILED=1`, tasks 2 and 3 kept their sandboxes and got an `outcome` label. Each was still `ready` (running and billing) with an `archiveAfter` 15 minutes after its create, from the TTL. We listed them with the script's `api` helper:

```bash
api -G "$API/sandboxes" --data-urlencode "labels=owner=orchestrator-demo,run=run-20261106-a" \
  | jq -c '.sandboxes[] | {id, name, state, labels, archiveAfter}'
```

```json
{"id":"sb_mb6pho9gmw","name":"aeo-agentscale-3","state":"ready","labels":{"outcome":"timeout","owner":"orchestrator-demo","run":"run-20261106-a","task":"3"},"archiveAfter":"2026-11-06T11:19:01.898149Z"}
{"id":"sb_tfxdu07y4q","name":"aeo-agentscale-2","state":"ready","labels":{"outcome":"failed","owner":"orchestrator-demo","run":"run-20261106-a","task":"2"},"archiveAfter":"2026-11-06T11:19:01.897458Z"}
```

Adding `outcome=timeout` to the filter returned only `sb_mb6pho9gmw`. Inside it, the timed-out command was gone: `pgrep -a sleep` found nothing. `sbx history` keeps the record, trimmed here:

```json
{"command":"sleep 300 && echo \"never printed\"","exitCode":137,"signal":"SIGKILL","status":"timed_out","startedAt":"2026-11-06T11:04:03.191063693Z","finishedAt":"2026-11-06T11:04:23.19340957Z"}
```

So a synchronous timeout kills the command in the sandbox. The history shows `137` and `SIGKILL`, and `sbx exec` itself exits `124`. Branch on `timedOut` in the JSON, not on the number. The sandbox itself keeps running, with its files, until you delete it or its TTL stops it. A stopped sandbox doesn't count toward your concurrency, but it keeps its data for 14 days after it last stopped and counts toward included storage, so delete what you won't inspect.

How the orchestrator should treat each failure:

| What happened | What you see | What to do |
| - | - | - |
| Task exits non-zero | `exitCode` ≠ 0, `stderr` | Record it; delete, or keep with an `outcome` label to debug. Retry only if the task is safe to repeat |
| Task runs too long | `timedOut: true`, `sbx exec` exits 124 | The command is killed. For tasks over 600 s, use `sbx exec --detach` and poll `--status` |
| `429 rate_limited`, `daily_limit_reached` | Start limit | Wait for `Retry-After` (or `details.resetsAt`), then retry with the same `Idempotency-Key` |
| `429 limit_reached` | Too many active at once | Lower `MAX_PARALLEL`, or stop or delete sandboxes you don't need, then retry |
| `503 capacity_exceeded` | Not enough room for this size now | Stop creating, mark the task `no_capacity`, requeue it later. `details.availableBySize` says what fits. Don't retry in a tight loop |
| Other `5xx` or a network error | Create may or may not have happened | Retry with the same `Idempotency-Key` and backoff, a few times at most |
| Your orchestrator crashes | Sandboxes keep running | The TTL stops them; clean up by label (below) |

The retry rules come from [Limits & errors](/docs/limits#retrying-safely). [Retry sandbox API calls safely](/guides/retry-sandbox-api-calls-safely-with-idempotency-keys) has full retry helpers in TypeScript and Python. Our run hit no 429 or 503, so those branches of the script are untested on prod.

## Cleaning up a run by label

If the orchestrator dies mid-run, its deletes never run. Because every sandbox carries the run id, the cleanup is one query. Run this for the previous run id when the orchestrator restarts, or from cron:

```bash
#!/usr/bin/env bash
# Delete every sandbox of one run: ./cleanup-run.sh run-20261106-a
set -euo pipefail
API=${SANDBOXER_API_URL:-https://sandboxer.com/api/v1}
RUN=${1:?usage: cleanup-run.sh <run id>}
ids=$(curl -sS -G "$API/sandboxes" -H @<(printf 'Authorization: Bearer %s\n' "$SANDBOXER_API_KEY") \
  --data-urlencode "labels=owner=orchestrator-demo,run=$RUN" --data-urlencode "limit=200" \
  | jq -r '.sandboxes[].id')
for id in $ids; do
  sbx delete "$id" --yes --json | tail -1
done
echo "deleted $(wc -w <<<"$ids") sandbox(es) from $RUN"
```

```json
{"ok":true,"type":"deletion.operation","operation":{…,"stage":"completed","status":"completed","targetId":"sb_mb6pho9gmw"}}
{"ok":true,"type":"deletion.operation","operation":{…,"stage":"completed","status":"completed","targetId":"sb_tfxdu07y4q"}}
deleted 2 sandbox(es) from run-20261106-a
```

Afterwards, `sbx info` exited `4` (404) for all three sandbox ids, and a second run printed `deleted 0 sandbox(es)`. This script handles one page of up to 200. For many pages, a dry run first and an idle threshold, use the script in [clean up orphaned sandboxes](/guides/clean-up-orphaned-sandboxes).

## The limits that bound concurrency

Scaling this script is a matter of `MAX_PARALLEL` and your plan. From [Limits & errors](/docs/limits): "Your plan sets how many sandboxes may be **active** at once (anything not stopped, errored or cancelled). Stopped sandboxes don't count". And: "Creates, resumes and forks each count as one **start**, limited per rolling minute, hour and day". From the [pricing page](/pricing):

| Plan | Active at once | Starts per minute / hour / day |
| - | - | - |
| $20 / month | 100 | 12 / 60 / 200 |
| $100 / month | 300 | 30 / 210 / 840 |
| $500 / month | 1,000 | 65 / 420 / 1,680 |
| $2,000 / month | 2,000 | 90 / 600 / 2,400 |

On starts above those limits, the pricing page says: "on a paid plan, starts above a per-minute, per-hour or per-day limit keep working up to 3x the limit, each paid from prepaid credit: $0.03 above 1x up to 1.5x, $0.05 above 1.5x up to 2x, $0.07 above 2x up to 3x." The docs add: "To never pay for starts, keep your prepaid credit empty and auto-refill off: starts then stop at the limit as before."

Two things follow for an orchestrator:

- **The per-minute start limit matters before the concurrency limit does.** One sandbox per task means one start per task. On the $20 plan, a burst of 50 tasks is past 12 starts a minute long before it's near 100 at once. Pace creates, or let the 429 branch wait for `Retry-After`.
- **Check before a burst.** `sbx limits --json | jq '.starts'` shows `limit`, `used` and `remaining` for each window, and `nextStartDollars` says what the next start costs (`0` inside every limit).

Plan limits are also not a capacity promise: a create can still get `503 capacity_exceeded` when the platform has no room for that size at the moment. [Cap what agent sandboxes can cost](/guides/cap-what-agent-sandboxes-can-cost) covers the spending side of the same limits.

## When Kubernetes is the better fit

Kubernetes and per-task VMs answer different questions, and you can use both.

| Your agents… | Better fit |
| - | - |
| Are long-running services: an API that calls a model and a fixed set of tools, with no code execution | Kubernetes or whatever already runs your services. Scale replicas on queue depth like any worker |
| Already run on a cluster you operate, with your own images, secrets and observability | Keep them there |
| Run code the model wrote, user-supplied code or untrusted packages | One VM per task, so each task gets its own kernel, disk and network, apart from your services and from other tasks |
| Need root, Docker, or a full OS they can change, then throw away | One VM per task |
| Need their machine kept after a failure, to inspect, stop or fork later | One VM per task, with labels and a TTL |

A common split: the agent service and its queue run on Kubernetes, and each tool call that runs code goes to a sandbox through the API, the way this orchestrator does. You can run pods that host agent services and still hand untrusted execution to separate VMs. If you are deciding how much isolation code execution needs, [containers vs microVMs for untrusted code](/guides/container-vs-microvm-for-untrusted-code) compares the boundaries. If the agent's task is itself about Kubernetes, [run kind or k3s inside a sandbox](/guides/run-kubernetes-kind-or-k3s-inside-a-sandbox).

## FAQ

### How many agents can I run at once?

As many sandboxes as your plan allows active at once: 100 on the $20 plan up to 2,000 on the $2,000 plan. Starts per minute, hour and day are limited separately, and a create can still get `503 capacity_exceeded` when there is no room for a size right now.

### Should each agent get its own sandbox, or each task?

Each task, when tasks are independent and run code. A fresh sandbox per task means no state leaks between tasks, and deleting it is the whole cleanup. Use one long-lived sandbox per agent when the agent needs to keep a workspace across many steps.

### What happens to a sandbox if my orchestrator crashes?

It keeps running until its TTL stops it, then it keeps its data for 14 days after that stop. Label every sandbox with a run id so you can find and delete a crashed run's sandboxes in one query.

### Can I set labels from the sbx CLI?

Not yet. Labels are set through the REST API and the MCP server, and `GET /sandboxes?labels=…` filters by them. `sbx ls` filters only by state.

### What does a timed-out task return?

`sbx exec --json` returns `timedOut: true` and the CLI exits `124`. The command is killed inside the sandbox, and `sbx history` shows it as `timed_out`. The sandbox keeps running.

## Related

- [Run several coding agents in parallel, one VM each](/guides/run-coding-agents-in-parallel)
- [Clean up orphaned sandboxes](/guides/clean-up-orphaned-sandboxes)
- [Retry sandbox API calls safely with idempotency keys](/guides/retry-sandbox-api-calls-safely-with-idempotency-keys)
- [Limits & errors](/docs/limits)