Does an AI Agent's Sandbox Need a GPU?
Updated November 18, 2026 .md
Short answer: Usually not. Running code, tests, builds, headless browsers and data analysis all run on CPU, and the model itself runs at your provider's API. Sandboxer has no GPUs. When one step needs a GPU, such as training or a large local model, the sandbox calls a GPU service over HTTPS and keeps the code and files.
Where the GPU work happens in an agent#
An agent loop has two halves. The model decides what to do next. The sandbox does it: it writes files, installs packages, runs the tests, opens a page, crunches a CSV. When you use a hosted model, the first half runs on the provider's GPUs, and your sandbox only sends and receives text over HTTPS. The second half is ordinary Linux work, and that is CPU work.
Sandboxer's own llms.txt says it plainly. Sandboxer runs "on dedicated servers Sandboxer operates, in the two regions above. There's no SLA during early access, and no GPUs." Under "When something else fits better" it lists: "You need GPUs. Sandboxer has none."
What we measured on CPU#
All numbers below are from one default sandbox (4 vCPU, 8 GB RAM; the guest saw an AMD EPYC 9254) on prod on November 18, 2026. They're what we measured on that day and that machine, not a promise. Run the same scripts on your own sandbox before you size a job.
| Agent task | What we measured |
|---|---|
| GPU present? | No GPU PCI device, no nvidia-smi, no GPU device nodes in /dev |
| pandas: write a 5,000,000-row CSV (74 MB) | 2.9 s |
pandas: read_csv of that file |
0.10 to 0.11 s |
pandas: two groupby aggregations over 5M rows |
0.18 s |
| pytest: more-itertools suite (772 tests, 21,203 subtests) | 23 to 24 s; 10.8 s with pytest -n 4 |
| Headless Chrome: load a docs page and save a screenshot | 0.63 to 0.73 s per run |
| Embeddings: all-MiniLM-L6-v2 on CPU, 5,000 short sentences | 630 to 637 sentences/s |
| One "GPU step" sent to an HTTPS API from inside the sandbox | 0.03 to 0.04 s round trip to a public echo endpoint |
None of these needed a GPU. The slowest item, the test suite, is bound by CPU cores, so the fix is -n 4 or a bigger size, not a GPU.
1. Check for a GPU yourself#
lspci isn't preinstalled, so install pciutils first:
sbx new --type default
sbx exec sb_4reakqgf3j --timeout 180 "sudo apt-get update -qq >/dev/null 2>&1; sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq pciutils >/dev/null 2>&1; lspci"00:00.0 Host bridge: Intel Corporation Device 0d57
00:01.0 Mass storage controller: Red Hat, Inc. Virtio 1.0 block device (rev 01)
00:02.0 Ethernet controller: Red Hat, Inc. Virtio 1.0 network device (rev 01)
00:03.0 Unassigned class [ffff]: Red Hat, Inc. Virtio 1.0 RNG (rev 01)
00:04.0 Unassigned class [ffff]: Red Hat, Inc. Virtio 1.0 socket (rev 01)
00:05.0 Unassigned class [ffff]: Red Hat, Inc. Virtio 1.0 memory balloon (rev 01)A host bridge, five virtio devices and no display or 3D controller. The other checks agree:
sbx exec sb_4reakqgf3j 'command -v nvidia-smi || echo nvidia-smi: not found; ls /dev | grep -iE "nvidia|dri|kfd|accel" || echo no GPU device nodes; ls /dev/kvm; nproc'nvidia-smi: not found
no GPU device nodes
/dev/kvm
4/dev/kvm is there because a sandbox is a full microVM that can run nested VMs (Docker and nested KVM). It's hardware virtualization, not a graphics card.
2. Install a CPU-only Python stack#
PyTorch publishes CPU-only wheels on its own index (PyTorch install selector). Using it skips the large CUDA downloads you can't use anyway. uv is preinstalled.
sbx exec sb_4reakqgf3j --timeout 600 'cd ~ && uv venv -q work && . work/bin/activate && uv pip install -q pandas pyarrow pytest && uv pip install -q torch --index-url https://download.pytorch.org/whl/cpu && uv pip install -q sentence-transformers && python -c "import torch,pandas,sentence_transformers as s;print(torch.__version__, torch.cuda.is_available(), pandas.__version__, s.__version__)"'2.15.0+cpu False 3.1.1 6.3.0torch.cuda.is_available() is False, which is what you want libraries to see. They then pick their CPU code paths without errors.
3. Data analysis: pandas on 5 million rows#
# pandas_bench.py
import time, numpy as np, pandas as pd
n = 5_000_000
rng = np.random.default_rng(0)
df = pd.DataFrame({
"user_id": rng.integers(0, 100_000, n),
"country": rng.choice(["US", "DE", "IN", "BR", "JP"], n),
"amount": rng.gamma(2.0, 30.0, n).round(2),
})
t = time.perf_counter(); df.to_csv("events.csv", index=False); print(f"write csv: {time.perf_counter()-t:.1f} s")
import os; print(f"file: {os.path.getsize('events.csv')/1e6:.0f} MB, {n:,} rows")
t = time.perf_counter(); df = pd.read_csv("events.csv"); print(f"read_csv: {time.perf_counter()-t:.2f} s")
t = time.perf_counter()
out = df.groupby("country").agg(orders=("amount", "size"), revenue=("amount", "sum")).sort_values("revenue", ascending=False)
top = df.groupby("user_id")["amount"].sum().nlargest(5)
print(f"groupby + top users: {time.perf_counter()-t:.2f} s")
print(out.round(0).to_string())sbx scp pandas_bench.py sb_4reakqgf3j:/home/user/pandas_bench.py
sbx exec sb_4reakqgf3j --timeout 300 "cd ~ && work/bin/python pandas_bench.py"write csv: 2.9 s
file: 74 MB, 5,000,000 rows
read_csv: 0.10 s
groupby + top users: 0.18 s
orders revenue
country
US 1000267 60053668.0
BR 1000985 60036966.0
…A second run gave 0.11 s and 0.18 s. For a whole agent that answers questions about a CSV, see build a data-analysis agent.
4. Tests and builds: a real pytest suite#
We cloned more-itertools (commit c0ede7f) and ran its tests twice, then in parallel with pytest-xdist:
sbx exec sb_4reakqgf3j --timeout 300 'cd ~ && git clone -q --depth 1 https://github.com/more-itertools/more-itertools && cd more-itertools && uv pip install -q --python ../work/bin/python -e . && ../work/bin/python -m pytest -q -p no:cacheprovider | tail -1'
sbx exec sb_4reakqgf3j --timeout 300 'cd ~/more-itertools && uv pip install -q --python ../work/bin/python pytest-xdist && ../work/bin/python -m pytest -q -p no:cacheprovider -n 4 | tail -1'772 passed, 21203 subtests passed in 22.95s
772 passed, 21203 subtests passed in 10.76sSpreading tests over the 4 vCPUs halved the time. If an agent's test loop is slow, more cores help. A GPU doesn't.
5. Browsers: headless Chrome#
Google Chrome is installed in every sandbox. This loads a page and saves a screenshot three times:
sbx exec sb_4reakqgf3j --timeout 120 'cd /tmp && U=https://docs.python.org/3/library/asyncio.html; google-chrome --headless=new --dump-dom $U 2>/dev/null | grep -o "<title>[^<]*"; for i in 1 2 3; do t=$(date +%s%N); google-chrome --headless=new --no-first-run --screenshot=shot.png --window-size=1280,800 $U >/dev/null 2>&1; echo run $i: $(( ($(date +%s%N)-t)/1000000 )) ms; done'<title>asyncio — Asynchronous I/O — Python 3.15.2 documentation
run 1: 633 ms
run 2: 673 ms
run 3: 729 msEach run is a full Chrome start, page load, render and PNG write. For an agent that drives the browser, see Browser Use or Playwright with real Chrome.
6. Small models on CPU: embeddings#
Small encoder models are fine on CPU. This embeds 5,000 short sentences with sentence-transformers/all-MiniLM-L6-v2, three times:
# embed_bench.py
import time, torch
t = time.perf_counter()
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cpu")
print(f"load (incl. download on first run): {time.perf_counter()-t:.1f} s, threads={torch.get_num_threads()}")
sentences = [f"Ticket {i}: the build fails after upgrading dependency number {i % 97} on Linux." for i in range(5000)]
model.encode(sentences[:64]) # warm up
for run in range(3):
t = time.perf_counter()
emb = model.encode(sentences, batch_size=64)
dt = time.perf_counter() - t
print(f"run {run+1}: {len(sentences)} sentences in {dt:.2f} s = {len(sentences)/dt:.0f} sentences/s, shape {emb.shape}")sbx scp embed_bench.py sb_4reakqgf3j:/home/user/embed_bench.py
sbx exec sb_4reakqgf3j --timeout 300 "cd ~ && work/bin/python embed_bench.py"load (incl. download on first run): 4.4 s, threads=4
run 1: 5000 sentences in 7.93 s = 630 sentences/s, shape (5000, 384)
run 2: 5000 sentences in 7.86 s = 636 sentences/s, shape (5000, 384)
run 3: 5000 sentences in 7.85 s = 637 sentences/s, shape (5000, 384)At about 630 sentences a second, embedding a 50,000-chunk codebase or document set takes under a minute and a half. Small chat models also run on CPU, at a few dozen tokens per second; we measured that in Ollama on CPU and llama.cpp's llama-server on CPU.
When a step does need a GPU#
| Step | Run it in the sandbox on CPU? |
|---|---|
| Calling a hosted model (the agent's own LLM) | Yes. It's an HTTPS request; the GPUs are the provider's |
| Code, tests, builds, Docker, browsers, scraping | Yes |
| pandas, SQL, charts, classic ML (scikit-learn, gradient boosting) | Yes |
| Embeddings with a small model, speech-to-text with a small model | Yes, at the speeds above or in whisper.cpp |
| Chat with a 0.5B to 1.5B local model | Yes, slowly (tens of tokens per second) |
| Fine-tuning or training a neural network | No. Send it to a GPU cloud |
| A large local model (tens of billions of parameters) | No. Use a hosted inference API |
| Image or video generation, heavy batch inference | No. Use a hosted inference API or a GPU cloud |
The pattern for the "no" rows: the sandbox stays the agent's workspace. It holds the repo, the data, the scripts and the results. Only the GPU step leaves, as an HTTPS request, and its output comes back into the sandbox's files.
7. Send the GPU step out over HTTPS#
Write the step as a small script that reads its endpoint and key from environment variables, never prints the key, retries on 429/502/503, and writes the result to disk:
# gpu_step.py
# The one step that needs a GPU: send it out, keep everything else in the sandbox.
import json, os, sys, time, urllib.request, urllib.error
URL = os.environ["INFERENCE_URL"] # set per sandbox or in your environment
KEY = os.environ["INFERENCE_API_KEY"] # never print it
def call(payload, attempts=4):
req = urllib.request.Request(URL, data=json.dumps(payload).encode(), method="POST",
headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})
for i in range(attempts):
try:
with urllib.request.urlopen(req, timeout=120) as r:
return r.status, json.load(r)
except urllib.error.HTTPError as e:
if e.code in (429, 502, 503) and i < attempts - 1:
time.sleep(2 ** i); continue
return e.code, json.load(e)
text = open(sys.argv[1]).read()
t = time.perf_counter()
status, out = call({"model": "big-model", "input": text})
print(f"HTTP {status} in {time.perf_counter()-t:.2f} s")
json.dump(out, open("result.json", "w"))
print(json.dumps(out))We tested it two ways, with a made-up key (demo-key-not-real), not a real provider key.
Against a mock GPU API inside the sandbox. This stand-in checks the bearer key, waits 0.5 s to pretend it's working, and returns JSON:
# mock_gpu_api.py
import json, os, time
from http.server import BaseHTTPRequestHandler, HTTPServer
KEY = os.environ["MOCK_EXPECTED_KEY"]
class H(BaseHTTPRequestHandler):
def do_POST(self):
if self.headers.get("Authorization") != f"Bearer {KEY}":
return self.reply(401, {"error": "missing or wrong API key"})
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
time.sleep(0.5) # pretend the GPU is working
self.reply(200, {"model": body["model"], "output": f"summary of {len(body['input'])} chars", "gpu_seconds": 0.5})
def reply(self, code, obj):
data = json.dumps(obj).encode()
self.send_response(code); self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data))); self.end_headers(); self.wfile.write(data)
def log_message(self, *a): pass
HTTPServer(("127.0.0.1", 8000), H).serve_forever()sbx exec sb_4reakqgf3j --timeout 60 'cd ~ && export MOCK_EXPECTED_KEY=demo-key-not-real; nohup python3 mock_gpu_api.py >/dev/null 2>&1 & sleep 1; export INFERENCE_URL=http://127.0.0.1:8000/v1/generate INFERENCE_API_KEY=demo-key-not-real; python3 gpu_step.py more-itertools/README.rst; INFERENCE_API_KEY=wrong python3 gpu_step.py more-itertools/README.rst'HTTP 200 in 0.51 s
{"model": "big-model", "output": "summary of 41435 chars", "gpu_seconds": 0.5}
HTTP 401 in 0.01 s
{"error": "missing or wrong API key"}Against a public HTTPS echo endpoint. This proves the request leaves the sandbox over TLS with the header attached. We printed only whether the header arrived, never its value:
sbx exec sb_4reakqgf3j --timeout 60 'cd ~ && export INFERENCE_URL=https://httpbin.org/anything INFERENCE_API_KEY=demo-key-not-real; for i in 1 2 3; do python3 gpu_step.py more-itertools/README.rst | sed -n 1p; done'
sbx exec sb_4reakqgf3j 'cd ~ && python3 -c "import json;d=json.load(open(\"result.json\"));print(\"server saw Authorization header:\", \"Authorization\" in d[\"headers\"], \"| url:\", d[\"url\"], \"| body chars:\", len(d[\"json\"][\"input\"]))"'HTTP 200 in 0.03 s
HTTP 200 in 0.03 s
HTTP 200 in 0.04 s
server saw Authorization header: True | url: https://httpbin.org/anything | body chars: 41435The echo endpoint returns the request it received, so the second command reads result.json and checks for the header. Point INFERENCE_URL at your GPU cloud's or hosted inference API's endpoint and the same script works, as long as that API takes JSON with a bearer key. Long jobs, like training runs, usually return a job ID instead. Then poll it from the sandbox with sbx exec --detach and download the artifacts into the sandbox when the job is done.
Getting the real key into the sandbox#
Don't paste the key into the agent's prompt or into a command line that ends up in logs. Use either a variable in your default environment (base), which every new sandbox gets, or a per-sandbox one with -e on sbx new. Both are in Environments and secrets, and environment variables inside a sandbox shows how they reach sbx exec, systemd services and agents. If the sandbox runs code you don't trust, create it with --no-env and pass only the one key it needs.
FAQ#
Can I attach a GPU to a Sandboxer sandbox?#
No. Sandboxer has no GPUs, at any size. Every size is the same CPU machine with more or fewer vCPUs and RAM (machines).
Does my coding agent need a GPU to run its model?#
Not if it uses a hosted model. The model runs on the provider's hardware, and the sandbox just sends requests over HTTPS. The sandbox needs a GPU only if you insist on running a large model inside it.
Will PyTorch or TensorFlow code fail without a GPU?#
Code written for CPU or with a device check ("cuda" if torch.cuda.is_available() else "cpu") runs fine; in our sandbox torch.cuda.is_available() returned False. Code that hard-codes .cuda() fails, so change it to pick the device or move that step to a GPU service.
What do I do if the agent's tests or builds are slow?#
Add cores, not a GPU. Our test suite went from 23 s to 10.8 s with pytest -n 4 on 4 vCPUs. You can also stop the sandbox and resume it as a larger size, which keeps files but restarts processes (fix exit code 137 or a full disk shows the resize).
Is calling an outside GPU API from a sandbox allowed?#
Yes. Outbound traffic from a sandbox to the internet is allowed (network security), so HTTPS calls to any API work. Keep the API key in an environment variable, not in code.
Related#
More guides
For running AI-written code we rank Sandboxer, Freestyle and E2B first: each gives code its own VM with an API or SDK and can pause with memory. Pick Daytona for Windows or GPUs, Modal or Northflank for GPUs at scale, E2B or Northflank to run in your own cloud, and Vercel Sandbox if your app lives on Vercel.
For coding agents we rank Sandboxer first: a full Ubuntu VM with Claude Code and Codex preinstalled, Docker, and stop-and-resume that keeps the agent's session in memory. Freestyle and Amp Orbs also keep memory across pauses. Pick E2B for prebuilt agent templates on open source, Docker Sandboxes for free local runs, and Codespaces if you live in GitHub.
The best Daytona alternatives depend on why you're switching. For a full Linux microVM on every sandbox with memory-saving stop and Docker, we rank Sandboxer first. E2B suits open source and self-hosting, Modal massive short-job fleets, Northflank bring-your-own-cloud with GPUs, and Vercel Sandbox apps on Vercel.
If you're leaving E2B for longer sessions or a fuller machine, we rank Sandboxer first, then Freestyle: both give each sandbox a full Linux VM that keeps memory across pauses, with Docker and no short session cap. Pick Daytona for Windows or GPUs, Northflank for bring-your-own-cloud, and stay on E2B if you need open source.