# Are Sandboxer Sandboxes a Fit for DevRel and Runnable Docs?

> Run every bash block of a docs page, in order, in a fresh noEnv sandbox and compare stdout with the page's expected output, so stale samples fail before readers find them. Host the finished demo on a private HTTPS URL, start workshop machines from a named snapshot, and delete everything after. Billing is per second.

Source: https://sandboxer.com/guides/sandboxes-for-devrel-runnable-docs. Updated 2026-10-27.

**Short answer:** Run every bash block of a docs page, in order, in a fresh noEnv sandbox and compare stdout with the page's expected output, so stale samples fail before readers find them. Host the finished demo on a private HTTPS URL, start workshop machines from a named snapshot, and delete everything after. Billing is per second.

## The jobs, and what to use for each

A developer relations or docs team asks five things of a sandbox. Each row gives the short answer, the feature that does it, and the guide with a full run on prod.

| Job | Short answer | Sandboxer feature | Worked guide |
| - | - | - | - |
| Prove the docs still run | Run each page's code blocks in a fresh VM on a schedule or on every docs PR, and fail on a non-zero exit or wrong output | `sbx new --no-env`, `sbx exec` (exits with the remote exit code), `sbx delete` | This page, below; [Run each GitHub Actions job in a fresh microVM](/guides/run-each-github-actions-job-in-a-fresh-microvm) for the CI wiring |
| Preview the docs site per PR | One small sandbox per PR serves the built site on an HTTPS URL you post on the PR | `sbx host`, labels, idle pause | [A preview URL for every pull request](/guides/get-a-preview-url-for-every-pull-request), [Preview a Hugo or Astro site](/guides/preview-a-hugo-or-astro-site-in-a-sandbox) |
| A live demo for a talk or blog post | Run the demo as a service, host its port, and let it pause when nobody is looking | `sbx host` (private by default), idle auto-pause and wake on use | [Give a port an HTTPS URL](/guides/give-a-sandbox-port-an-https-url), [A service that comes back after resume](/guides/run-a-service-that-comes-back-after-resume), [Idle auto-pause](/guides/idle-auto-pause-and-wake-on-use) |
| A workshop machine per attendee | Build the workshop image once, then start one sandbox per attendee from it the morning of the event | `sbx snapshot <id> <name>`, `sbx new --from <name>` | [Turn a sandbox into a reusable template](/guides/turn-a-sandbox-into-a-reusable-template), [Rate limits and concurrency caps](/guides/handle-rate-limits-and-concurrency-caps) |
| A "Run this" button in your docs | Your backend runs the reader's snippet in a sandbox that holds none of your secrets | `noEnv` sandboxes, `exec` over the API | [Give every user their own sandbox](/guides/give-every-user-their-own-sandbox), [A fresh sandbox per tool call](/guides/a-fresh-sandbox-per-tool-call) |

`--no-env` matters for all of these. Docs code, workshop code and readers' code shouldn't see your account. A `noEnv` sandbox gets none of your variables or credentials, and its `SANDBOXER_TOKEN` can only read that one sandbox and host its ports ([environments](/docs/environments)).

## Test a docs page end to end

This is what we ran on prod on 2026-10-27 with one `small` sandbox. The page is a four-step Flask quickstart, written the way most docs pages are: a fenced `bash` block per step, and where the reader should see output, a fenced `text` block right after it. That pairing is the whole convention. The runner needs nothing else from your docs.

The steps, as the reader sees them:

1. Install: `python3 -m venv .venv && .venv/bin/pip install -q flask==3.0.3`, then print the Flask version. Expected output `3.0.3`.
2. Write `app.py` with a `GET /hello` route, bound to `127.0.0.1:8000`.
3. Start it in the background and call `/hello`. Expected output `{"message":"hello"}`.
4. Call `/v2/hello`. Expected output `{"message":"hello","version":2}`.

The page's step 4 documents an endpoint the page's own app doesn't have. That is the kind of drift that ships when a page is edited by hand.

### The runner

```bash
#!/bin/bash
# doctest.sh <sandbox-id> <page.md>: run each ```bash block of a docs page in order, in one
# fresh directory of the sandbox. If a ```text block follows, its stdout must match it exactly.
id=$1; page=$2; work=$(mktemp -d); dir=doc-$(date +%s)
awk -v d="$work" '
  /^```bash$/ {n++; f=sprintf("%s/%02d.sh",d,n); inb=1; next}
  /^```text$/ && n && !(n in seen) {f=sprintf("%s/%02d.out",d,n); seen[n]=1; inb=1; next}
  /^```$/ {inb=0; close(f); next}
  inb {print > f}
' "$page"
tar czf $work.tgz -C $work .
sbx exec $id "mkdir -p $dir && echo $(base64 -w0 $work.tgz) | base64 -d | tar xzm -C $dir"
fail=0
for b in $work/*.sh; do
  n=$(basename $b .sh); t0=$(date +%s%N)
  out=$(sbx exec $id --cwd $dir --timeout 300 "bash -e $n.sh" 2>$work/$n.err); rc=$?
  ms=$(( ($(date +%s%N) - t0) / 1000000 ))
  status=ok
  if [ $rc -ne 0 ]; then status="FAIL exit=$rc"
  elif [ -f $work/$n.out ] && [ "$out" != "$(cat $work/$n.out)" ]; then status="FAIL output differs"; fi
  [ "$status" = ok ] || fail=1
  printf 'block %s  %-20s %6d ms\n' $n "$status" $ms
  if [ "$status" != ok ]; then echo "  expected: $(cat $work/$n.out 2>/dev/null)"; echo "  got:      $out"; fi
  [ -s $work/$n.err ] && sed 's/^/  stderr:   /' $work/$n.err | cut -c1-110
done
exit $fail
```

Three choices in it are deliberate:

- **One directory, many `exec` calls.** Each block runs as its own `sbx exec`, so you get an exit code and a time per block. Files carry over between blocks because they share a directory, the way they do for a reader. Shell variables don't; if your docs set a variable in one block and use it in the next, run the page as one script instead.
- **`--timeout 300`.** `sbx exec` stops a command after 30 seconds by default. A `pip install` or `npm install` block can take longer.
- **The background server is detached with `setsid nohup … < /dev/null &`.** Without that, `exec` waits for the server's output to close and the block never returns.

### Run it

```bash
sbx --json new --name docs-test-1 --type small --ttl 1800 --no-env | jq -r 'select(.event=="ready") | .id'
./doctest.sh sb_1afl0i7swc quickstart.md
```

The first version of the runner compared stdout and stderr together, and used plain `tar xz`. It failed for two reasons that had nothing to do with the page:

```text
tar: ./04.out: time stamp 2026-10-27 12:02:23 is 0.151195748 s in the future
…
block 01  FAIL output differs    4377 ms
  expected: 3.0.3
  got:      3.0.3
<string>:1: DeprecationWarning: The '__version__' attribute is deprecated and will be removed in Flask 3.1. …
```

The sandbox's clock was about 0.15 s behind ours, so `tar` warned about every file; `tar xzm` skips the timestamps. Flask printed a deprecation warning on stderr, so the "same" output differed. Compare stdout only and show stderr beside it: a warning is something to read, not a failure. The runner above has both fixes. Its run on the same sandbox:

```text
block 01  ok                     4123 ms
  stderr:   <string>:1: DeprecationWarning: The '__version__' attribute is deprecated and will be removed in F
block 02  ok                      205 ms
block 03  ok                      321 ms
block 04  FAIL exit=22            208 ms
  expected: {"message":"hello","version":2}
  got:      
```

That is the real bug: `curl -f` exits 22 on the 404, and the app's log shows `"GET /v2/hello HTTP/1.1" 404`. We added the `/v2/hello` route to step 2, switched step 1 to `importlib.metadata.version("flask")` to clear the warning, stopped the old server and ran the page again:

```text
block 01  ok                     4341 ms
block 02  ok                      204 ms
block 03  ok                      429 ms
block 04  ok                      209 ms
exit=0
```

The runner's exit code is the page's verdict, so a CI step that calls it fails the docs PR the same way a unit test would.

### Turn the tested page into a live demo

The app from the page is still running. Hosting it took one call, with `--localhost` because the app binds `127.0.0.1`:

```bash
sbx --json host sb_1afl0i7swc 8000 --localhost --title "Quickstart demo"
```

```json
{"ok":true,"type":"port.hosted","sandboxId":"sb_1afl0i7swc","port":8000,"url":"https://dreamy-jade-prairie-8000.box.sandboxer.com?_token=…","access":"private","isProtected":true,"localhost":true}
```

From our laptop, without the token, three requests got `401`. With `?_token=…`, the first request got a `302` that sets a cookie and drops the token from the address bar. Following it returned `{"message":"hello","version":2}` with `200` in 0.32 s, and five more requests with the cookie took 0.19 s each. That is the link you put on a slide or in a blog post's "try it" box. Use `sbx host … --public` only for a demo that holds nothing private.

Then `sbx delete sb_1afl0i7swc --yes` took 2.13 s, and `sbx --json info` on the id returned `404 not_found`.

### Timings and cost

Single measurements, timed from our client, so they include CLI and network time:

| Step | Time |
| - | - |
| Create (`small`, `--no-env`) to ready | 3.19 s |
| Run the four blocks, including `pip install flask` | 5.48 s |
| Host port 8000 | 0.30 s |
| Delete | 2.13 s |
| Sandbox's whole life, with the two failing runs and the demo | About 74 s |

`small` costs $0.018/hour, metered per second while the sandbox is up. Our whole session, 74 seconds, cost about 74 × $0.018 / 3600 = $0.00037. One clean test of this page is about 11 seconds of machine time: create, run, delete. Some sums for planning:

- **Nightly docs tests**, 40 pages, one sandbox per page at about 11 s each: 440 s a night, about $0.0022, so about $0.07 for 30 nights. Your pages will take longer than ours; scale by their install time.
- **A conference demo** left up for an 8-hour day on `small`: 8 × $0.018 = $0.144. With idle auto-pause, compute isn't billed while it's paused.
- **A workshop** of 30 attendees for 3 hours: 90 sandbox-hours. On `small` that is 90 × $0.018 = $1.62; on `default` ($0.036/hour), $3.24.

## Workshop math before the day

Creates, resumes and forks each count as a start. The plan limits, from [billing](/docs/billing#plans):

| Plan | Running at once | Starts per minute / hour / day | A 30-person workshop |
| - | - | - | - |
| Free trial | 2 (`small` and `default` only, TTL up to 8 hours) | | Too small: test the image on the trial, not the event |
| $20 | 100 | 12 / 60 / 200 | 30 creates spread over 3 minutes at 12 a minute; start them before people arrive |
| $100 | 300 | 30 / 210 / 840 | 30 creates in about a minute |

Start the machines from a named snapshot, so nobody waits on an install while you talk. Give each one a `--ttl` that ends after the session, so a forgotten machine stops on its own. Hand each attendee their own hosted URL or SSH access, not your API key. The full per-student run, with two isolation checks, is in [Sandboxes for classrooms and teaching](/guides/sandboxes-for-classrooms-and-teaching); for team events, see [Sandboxes for hackathons](/guides/sandboxes-for-hackathons).

## Limits to plan around

- **The network is live.** A docs block that installs the latest version passes today and breaks next month. That is often what you want a docs test to catch. Pin versions in the snapshot for workshops, where you want the opposite.
- **Each `exec` is a new shell.** Environment variables and `cd` don't carry between blocks. Files do.
- **Hosted URLs belong to one sandbox.** A new sandbox or a fork gets a new subdomain and no routes. Switching a port between public and private mints a new token, so old private links stop working. A brand-new route can answer `404` for a second or two; retry.
- **A stopped demo doesn't wake itself.** An idle-paused sandbox wakes on the next request with its token (the request is held up to 45 seconds). One you stopped, or one stopped by its TTL, doesn't wake: on 2026-10-27 its URL showed a `404` page, "No sandbox uses this address", until it was resumed. Set the TTL past the end of your talk.
- **Private URLs are secrets.** The `_token` is in the link. Redact it from screenshots and recordings, or use `--public` for a demo with nothing to protect.
- **CPU only.** `small` is 2 vCPU and 4 GB, `default` 4 and 8, `large` 8 and 16. A docs page that needs a GPU can't be tested here.
- **Moving to a smaller size isn't supported yet.** Create a new, smaller sandbox and copy what you need.

## FAQ

### Can I test docs that call our own API with a key?

Yes, but give the sandbox a test key of its own with `sbx new --no-env -e YOUR_API_KEY=…`, not your account's environment. Use a key scoped to a test account, since the page's code runs with it.

### Should each page get its own sandbox?

For a docs PR, yes: one sandbox per changed page keeps one page's leftovers out of another's result. For a nightly run of every page, check your plan's starts per day; on the $20 plan that is 200, so group small pages into one sandbox per section if you have more.

### What should go in the expected output block?

Only output that is stable: a version you pinned, a JSON body, a line count. Leave out timestamps, ids and timings, or the test fails on every run. If a step's output always changes, don't add a `text` block after it; the runner then checks only the exit code.

### Can readers run the samples themselves from the docs page?

You can build that: your backend creates a `noEnv` sandbox per reader, runs their snippet with `exec`, and returns the output. The [per-user sandbox guide](/guides/give-every-user-their-own-sandbox) covers mapping readers to sandboxes and keeping your secrets out.

## Related

- [Run each GitHub Actions job in a fresh microVM](/guides/run-each-github-actions-job-in-a-fresh-microvm)
- [How do I get a live preview URL for every pull request?](/guides/get-a-preview-url-for-every-pull-request)
- [Turn a sandbox into a reusable template](/guides/turn-a-sandbox-into-a-reusable-template)
- [Hosting](/docs/hosting)