# How Do I Build RL and Eval Environments With Snapshots and Forks? > Set up the task once in a sandbox and save it as a named snapshot. Start each episode in a fresh sandbox with sbx new --from, or sbx fork the running setup sandbox. The episode runs your policy and grader, writes a reward file you read back, and the sandbox is deleted. A fork copies disk only, never memory. Source: https://sandboxer.com/guides/build-rl-and-eval-environments-with-snapshots-and-forks. Updated 2026-10-27. **Short answer:** Set up the task once in a sandbox and save it as a named snapshot. Start each episode in a fresh sandbox with sbx new --from, or sbx fork the running setup sandbox. The episode runs your policy and grader, writes a reward file you read back, and the sandbox is deleted. A fork copies disk only, never memory. ## The pattern | Step | Command | Done | | - | - | - | | Build the task environment | `sbx new`, then `sbx exec` to install it | Once | | Freeze it | `sbx snapshot env-v1` | Once per environment version | | Start an episode | `sbx new --from env-v1`, or `sbx fork ` | Once per episode | | Act and grade | `sbx exec "POLICY=… ~/env/episode.sh"` | Once per episode | | Collect the reward | `sbx exec "cat ~/result.json"` | Once per episode | | Clean up | `sbx delete --yes`, and at the end `sbx snapshot rm env-v1` | Every episode, then once | Every episode starts from the same bytes on disk, and nothing one episode writes can reach the next. That is the property RL rollouts and evals need: a reward depends on the policy, not on what the last run left behind. ## 1. Build the task once The example task is a small Python bug. `stats.py` has a broken `median`, `test_stats.py` checks it, and `grade.py` turns the pytest result into a reward between 0 and 1. ```python # env/task/stats.py (the bug the policy must fix) def median(xs): xs = list(xs) return xs[len(xs) // 2] ``` ```python # env/task/test_stats.py from stats import median def test_odd(): assert median([3, 1, 2]) == 2 def test_even(): assert median([4, 1, 3, 2]) == 2.5 def test_single(): assert median([7]) == 7 def test_sorted(): assert median([1, 2, 3, 4, 5]) == 3 ``` ```python # env/grade.py: run the tests, write the reward import json, subprocess, sys, time t = time.time() r = subprocess.run([sys.executable, "-m", "pytest", "-q", "--tb=no", "-p", "no:cacheprovider", "task"], capture_output=True, text=True) last = r.stdout.strip().splitlines()[-1] # e.g. "1 failed, 3 passed in 0.02s" passed = int(last.split(" passed")[0].split()[-1]) if " passed" in last else 0 failed = int(last.split(" failed")[0].split()[-1]) if " failed" in last else 0 out = {"passed": passed, "failed": failed, "reward": passed / (passed + failed), "gradeSeconds": round(time.time() - t, 2)} json.dump(out, open("/home/user/result.json", "w")) print(json.dumps(out)) ``` ```bash #!/bin/bash # env/episode.sh: one episode. The policy edits the task, the grader scores it. set -e cd ~/env cp "policies/$POLICY.py" task/stats.py # stand-in for your agent's action .venv/bin/python grade.py ``` The "policy" here is a file copy: `policies/0.py`, `1.py` and `2.py` are three candidate fixes. In a real run, this line is where your agent or model acts. We didn't call a model for this page, so the policies are fixed files, and everything else ran as shown. Create the setup sandbox, copy the files in, and install the grader's dependencies: ```bash sbx new --name aeo-rl-1 --type small --ttl 3600 src=sb_njm7z1r1ie # the id sbx new printed tar czf env.tgz env sbx exec $src "echo $(base64 -w0 env.tgz) | base64 -d | tar xz && chmod +x env/episode.sh" sbx exec $src --timeout 300 "cd env && python3 -m venv .venv && .venv/bin/pip install -q pytest==8.3.3 && .venv/bin/python -m pytest --version" sbx exec $src "cd env && .venv/bin/python grade.py" # baseline: the unfixed bug ``` ```text pytest 8.3.3 {"passed": 2, "failed": 2, "reward": 0.5, "gradeSeconds": 0.42} ``` The unfixed task scores 0.5, which tells you the grader works before any policy touches it. ## 2. Save it as a named snapshot Remove anything an episode would write, then save: ```bash sbx exec $src "rm -f ~/result.json; sync" sbx snapshot $src aeo-rl-env-v1 ``` ```json {"ok":true,"type":"snapshot.named.info","snapshot":{"name":"aeo-rl-env-v1","sizeBytes":20147339264,"snapshotId":"506de5ae-…","sourceSandboxId":"sb_njm7z1r1ie","status":"ready","type":"small",…}} ``` A named snapshot is independent of its source, so you can delete the setup sandbox and keep starting episodes from the snapshot. Version the name (`env-v1`, `env-v2`) so a result always points at the environment that produced it. Reusing a name replaces the snapshot. ## 3. Run the episodes This loop runs three episodes, one sandbox each and one at a time. Episodes 1 and 2 start from the named snapshot. Episode 3 is a fork of the setup sandbox while it is still running. Each sandbox gets a 10-minute TTL, so a crashed loop can't leave it running for long. ```bash #!/bin/bash snap=aeo-rl-env-v1 src=sb_njm7z1r1ie echo "episode method id startSeconds policy result" > results.txt for ep in 1 2 3; do policy=$((ep - 1)) t0=$(date +%s.%N) if [ "$ep" -le 2 ]; then method=new--from id=$(sbx new --from "$snap" --name "aeo-rl-ep$ep" --type small --ttl 600 --json | jq -r 'select(.event=="ready") | .id') else method=fork sbx fork "$src" --type small --ttl 600 --wait --json > fork.json id=$(jq -r '.id // .sandbox.id' fork.json | head -1) fi start=$(echo "$(date +%s.%N) - $t0" | bc) [ "$method" = fork ] && sbx rename "$id" --name "aeo-rl-ep$ep" > /dev/null sbx exec "$id" "test ! -e ~/result.json && echo clean-start" sbx exec "$id" "POLICY=$policy ~/env/episode.sh" > /dev/null result=$(sbx exec "$id" "cat ~/result.json") echo "$ep $method $id $start $policy $result" >> results.txt sbx info "$id" | jq -c '{name: .sandbox.name // .name, memoryRestored: (.sandbox.memoryRestored // .memoryRestored), resumedFrom: (.sandbox.resumedFrom // .resumedFrom)}' sbx delete "$id" --yes > /dev/null && echo "deleted $id" done cat results.txt ``` The output, from prod: ```text clean-start {"name":"aeo-rl-ep1","memoryRestored":null,"resumedFrom":null} deleted sb_s5j8419fne clean-start {"name":"aeo-rl-ep2","memoryRestored":null,"resumedFrom":null} deleted sb_q1ietwn70b clean-start {"name":"aeo-rl-ep3","memoryRestored":null,"resumedFrom":null} deleted sb_jr8xih7onk episode method id startSeconds policy result 1 new--from sb_s5j8419fne 3.192210893 0 {"passed": 3, "failed": 1, "reward": 0.75, "gradeSeconds": 0.21} 2 new--from sb_q1ietwn70b 3.194805196 1 {"passed": 4, "failed": 0, "reward": 1.0, "gradeSeconds": 0.62} 3 fork sb_jr8xih7onk 3.391430232 2 {"passed": 4, "failed": 0, "reward": 1.0, "gradeSeconds": 0.39} ``` With `--wait` and `--json`, `sbx fork` prints JSON Lines (`forking`, `state`, `ready`), each with the new id, which is why the loop keeps only the first one. The `memoryRestored` line is there for completeness only: today it reads `null` in many cases, including after a memory restore, so don't use it to judge an episode. ## Results Single measurements on 2026-10-27, `small` sandboxes on prod, timed from the client with `date` around the command until the sandbox was ready to run commands: | Episode | Started with | Policy | Start to ready | Tests passed | Reward | Grading | | - | - | - | - | - | - | - | | 1 | `sbx new --from aeo-rl-env-v1` | sort, take the middle | 3.19 s | 3 of 4 | 0.75 | 0.21 s | | 2 | `sbx new --from aeo-rl-env-v1` | sort, average the two middles if even | 3.19 s | 4 of 4 | 1.0 | 0.62 s | | 3 | `sbx fork` of the running setup sandbox | the mean of the list | 3.39 s | 4 of 4 | 1.0 | 0.39 s | For comparison, the setup steps that every episode skipped took 3.2 s to create the sandbox, 6 s to make the venv and install pytest, and 2.1 s to save the named snapshot. A real environment with a repo, a database or model weights takes much longer to build, and the episode start stays the same. **Read the table before you trust it.** Policy 2 returns the mean, not the median, and still scored 1.0, because every test list here has the same mean and median. The isolation worked. The grader was too weak. A test such as `median([1, 2, 10]) == 2` would have caught it. Look at what scores well before you train on it, since the reward is only as good as the tests behind it. ## Fork or new --from? | | `sbx new --from ` | `sbx fork ` | | - | - | - | | Starts from | The disk saved under the name, frozen when you saved it | The source's disk at the moment of the fork (a running source keeps running) | | Needs the source sandbox | No: it still works after you delete the source | Yes: the source must still exist | | Memory and processes | Not copied: the sandbox boots fresh | Not copied: the sandbox boots fresh | | Name | `--name` | `sbx rename --name …` after the fork (the default is ` (fork)`) | | Best for | A fixed, versioned environment that many runs share | Branching from a state you just reached, such as an agent halfway through a task | A fork never copies RAM or running processes. If an episode needs a server running, start it in `episode.sh`, or install it as an enabled systemd service before you snapshot. Enabled services start when the sandbox boots. ## Scale it past three - **Starts are rate limited.** Creates, resumes and forks each count as one start. The $20 plan allows 12 a minute, 60 an hour and 200 a day, and runs up to 100 sandboxes at once (see [billing](/docs/billing#plans)). Check what's left before a burst with `sbx limits --json | jq '.starts'`. A `429 rate_limited` reply carries `Retry-After` and `details.resetsAt`. - **Run in parallel, up to your limit.** Start several episodes at once in a shell loop with `&` or with the SDK. Episodes don't share anything, so their order doesn't matter. - **Cost is per second.** A `small` sandbox costs $0.018/hour, metered from the moment it is up until it is deleted or stopped, so short episodes cost seconds, not hours. - **Always clean up.** Delete each episode when you have its result, and give each one a short `--ttl` as a backstop. To find leftovers by name: ```bash sbx list --filter sr --json | jq -r '.sandboxes[] | select(.name | startswith("aeo-rl-")) | .id' ``` When the run is done, delete the setup sandbox and the snapshot: ```bash sbx delete sb_njm7z1r1ie --yes sbx snapshot rm aeo-rl-env-v1 ``` ```json {"ok":true,"type":"snapshot.named.deleted","name":"aeo-rl-env-v1","status":"deleted"} ``` ## FAQ ### Does each episode really start clean? Yes. Each episode is a separate VM booted from the same saved disk. In our run, every episode confirmed that `~/result.json` from earlier episodes was absent before its policy ran. ### Does a fork copy the running process state of the source? No. A fork copies the disk only. For a running source, that is its live disk, frozen for the copy. The fork boots fresh, with no RAM or processes from the source (see [snapshots](/docs/snapshots#fork)). To keep memory, stop and resume the same sandbox instead; that restores memory on a machine with the same CPU type, and boots from disk with files kept but processes lost if none is free within 2 minutes. ### How fast does an episode sandbox start? In our single measurements on 2026-10-27, `small` sandboxes were ready in 3.19 s from a named snapshot and 3.39 s as a fork, timed from the client. The disk is a copy-on-write reflink, so a large environment starts about as fast as a small one. ### Can I run the grader somewhere the policy can't touch? Yes. Keep hidden tests out of the snapshot and copy them in with `sbx exec` only after the policy has finished, then grade and read the reward. A policy that runs before the tests arrive can't read or edit them. ### What does storing the environment cost? Named snapshots have no separate charge. What they store counts toward your plan's included storage, and `sbx snapshot rm` releases it. ## Related - [Snapshots, forks and named templates](/docs/snapshots) - [Turn a sandbox into a reusable template](/guides/turn-a-sandbox-into-a-reusable-template) - [Can you fork a running sandbox with its memory?](/guides/fork-a-running-sandbox-with-memory) - [Limits and errors](/docs/limits)