Skip to content
sandboxer
Start free trial
Guides · Agent cookbook

How Can a Coding Agent Check Its UI Change Before Opening a PR?

Updated October 27, 2026 .md

Short answer: Write a Playwright check for the change before the edit, and have the agent run it in the sandbox after every edit until it exits 0. It also fails on console errors. Then host the app and screenshots on private URLs for a human. Our scripted loop took 26 seconds.

"The tests pass" doesn't mean the UI works. A unit test won't see a button that is never rendered, or a click handler that throws. This guide gives the agent one command to run after each edit. It loads the page in real Chrome, clicks through the change, and returns a JSON verdict and screenshots. A second command publishes the result for a person. Everything below ran on prod on 2026-10-27, on small sandboxes (2 vCPU, 4 GB). No model ran: the agent's two edits were scripted, and the first one has a typo on purpose so you can see a failed check.

Other guides cover the pieces around this one. Turn a GitHub issue into a pull request covers tokens, cloning and pushing. Sandboxes for QA and test automation covers full suites, traces and flaky tests. A Next.js dev server with a preview URL covers hot reload over the hosted URL.

At a glance#

Step Command the agent runs Measured
Fresh sandbox to ready sbx new --type small (from outside) 3.2 s
Copy the repo, npm install Vite 7.3.7 and Playwright 1.64.1 sbx scp, sbx exec 10.6 s
Edit 1 plus check: fails with cuont is not defined agent/ui-check.sh 2.5 s (starts the dev server)
Edit 2 plus check: passes agent/ui-check.sh 1.6 s
Host the app and screenshots, copy them out agent/share.sh, sbx scp 8.3 s
Whole loop 26.2 s

Times are wall clock from our machine, so they include CLI and network time. Playwright itself took 0.8 to 1.0 s per check. The 3.2 s create is one run from earlier on October 27, 2026. New small and default sandboxes from the standard image now start from a warm pool, and Sandboxer measured a median of 0.69 s to ready from US-East (90% within 1.15 s) (changelog).

1. Write the check before the edit#

The task in our test: "Add a Reset button that sets the counter back to 0." The app is a small Vite page with a +1 button. Write the check from the task, before anyone touches the code, and run it once on the unchanged app. It should fail. If it passes, it doesn't test the change.

The script uses the Chrome that is already in the sandbox (channel: "chrome"), so there is no browser download. It treats any page error, console error or failed request as a failure. It prints one line of JSON for the agent and exits 1 on failure.

// agent/verify.mjs: check the "Reset button" change in real Chrome and save screenshots.
import { chromium } from "playwright";
import { mkdirSync } from "node:fs";

const url = process.env.APP_URL ?? "http://127.0.0.1:5173/";
const out = process.env.OUT ?? "artifacts";
mkdirSync(out, { recursive: true });
const t0 = Date.now();
const checks = [];
const errors = [];
const check = (name, ok, detail = "") => checks.push({ name, ok, ...(detail && { detail }) });

const browser = await chromium.launch({ channel: "chrome" });
const page = await browser.newPage({ viewport: { width: 800, height: 500 } });
page.on("pageerror", (e) => errors.push(e.message));
page.on("console", (m) => m.type() === "error" && !m.text().startsWith("Failed to load resource") && errors.push(m.text()));
page.on("response", (r) => r.status() >= 400 && !r.url().endsWith("/favicon.ico") && errors.push(`${r.status()} ${r.url()}`));

try {
  await page.goto(url);
  for (let i = 0; i < 3; i++) await page.click("#inc");
  check("count is 3 after three clicks", (await page.textContent("#count")) === "3");
  const reset = page.getByRole("button", { name: "Reset" });
  check("Reset button is visible", await reset.isVisible());
  await page.screenshot({ path: `${out}/before-reset.png` });
  if (await reset.isVisible()) {
    await reset.click();
    const v = await page.textContent("#count");
    check("count is 0 after Reset", v === "0", `count=${v}`);
  }
  await page.screenshot({ path: `${out}/after-reset.png` });
} catch (e) {
  check("page loaded and ran", false, e.message.split("\n")[0]);
}
check("no console or page errors", errors.length === 0, errors.join(" | "));
await browser.close();

const ok = checks.every((c) => c.ok);
console.log(JSON.stringify({ ok, ms: Date.now() - t0, checks: checks.filter((c) => !c.ok), screenshots: [`${out}/before-reset.png`, `${out}/after-reset.png`] }));
process.exit(ok ? 0 : 1);

The JSON lists only the failed checks. That keeps the agent's context short, and each failure names what to fix.

Our first version flagged Failed to load resource: … 404. That was Chrome asking for /favicon.ico. Chrome's console message doesn't include the URL, so the script ignores that message and checks responses directly. It skips only the favicon.

2. One command that starts the app and runs the check#

The agent shouldn't have to manage the dev server. This script starts it only if nothing answers yet, waits until it does, then runs the check:

#!/usr/bin/env bash
# agent/ui-check.sh: run after every edit. Exit code 0 means the change works.
set -u
cd "$(dirname "$0")/.."
if ! curl -fs -o /dev/null http://127.0.0.1:5173/; then
  setsid nohup npm run dev > dev.log 2>&1 < /dev/null &
  for i in $(seq 1 60); do
    curl -fs -o /dev/null http://127.0.0.1:5173/ && break
    sleep 0.25
  done
fi
node agent/verify.mjs

The dev script is vite --host 127.0.0.1 --port 5173 --strictPort. Pin the host. Without it, Vite 7.3.7 in the sandbox listened on [::1]:5173 only, so every request to 127.0.0.1 was refused. Our first run timed out that way. vite.config.js sets server: { allowedHosts: [".box.sandboxer.com"] } so the hosted URL in step 4 isn't rejected (hosting).

Run on the unchanged app, the check fails the way it should:

{"ok":false,"ms":858,"checks":[{"name":"Reset button is visible","ok":false}],"screenshots":["artifacts/before-reset.png","artifacts/after-reset.png"]}

3. The edit, check, fix loop#

Tell the agent the rule in its instructions: "After every edit, run agent/ui-check.sh. Don't open a pull request until it exits 0." Our first scripted edit added the button but misspelled a variable:

sbx exec $id "bash edit1.sh && proj/agent/ui-check.sh"
 index.html  | 1 +
 src/main.js | 4 ++++
 2 files changed, 5 insertions(+)
{"ok":false,"ms":857,"checks":[{"name":"count is 0 after Reset","ok":false,"detail":"count=3"},{"name":"no console or page errors","ok":false,"detail":"cuont is not defined"}],"screenshots":["artifacts/before-reset.png","artifacts/after-reset.png"]}

The button rendered, so a check that only looks for the button would have passed. The click is what failed, and the page error names the bug. The second edit fixed the typo:

{"ok":true,"ms":922,"checks":[],"screenshots":["artifacts/before-reset.png","artifacts/after-reset.png"]}

Each round trip took about 1.5 s, because the dev server was already running. Vite serves the edited file on the next page load, so nothing needs a restart.

4. Publish the result for a human#

Inside the sandbox, the host command publishes a port without an API key. It uses the sandbox's own SANDBOXER_TOKEN (inside the sandbox). host url <port> prints the URL, hosting the port first if needed. This script serves the screenshots on a second port and writes the text the agent posts:

#!/usr/bin/env bash
# agent/share.sh: after ui-check.sh passes, publish the app and the screenshots on private URLs.
set -eu
cd "$(dirname "$0")/.."
pgrep -f "http.server 8081" > /dev/null || \
  setsid nohup python3 -m http.server 8081 --bind 127.0.0.1 --directory artifacts > /dev/null 2>&1 < /dev/null &
app=$(host url 5173 --localhost)
shots=$(host url 8081 --localhost)
cat > artifacts/REVIEW.md <<MD
UI check passed in the agent's sandbox ($(date -u +%FT%TZ)).

- Live app (private link): $app
- Screenshots: ${shots%%\?*}/before-reset.png?${shots#*\?} and ${shots%%\?*}/after-reset.png?${shots#*\?}

$(git diff --stat)
MD

The REVIEW.md it wrote (tokens replaced with …):

UI check passed in the agent's sandbox (2026-10-27T15:57:21Z).

- Live app (private link): https://light-mild-papaya-5173.box.sandboxer.com?_token=…
- Screenshots: https://light-mild-papaya-8081.box.sandboxer.com/before-reset.png?_token=… and https://light-mild-papaya-8081.box.sandboxer.com/after-reset.png?_token=…

 index.html  | 1 +
 src/main.js | 4 ++++
 2 files changed, 5 insertions(+)

Both servers bind to 127.0.0.1 and are hosted with --localhost. A sandbox has a public IPv6 address, and loopback keeps them off it. From outside, each URL returned 401 without the token and 302 then 200 with it. The 302 sets a cookie and redirects to the clean path. Browsers keep the cookie. A curl -L without a cookie jar didn't, and got 401 on the redirect. If you script the screenshot download, use -c/-b, or copy the file out with sbx scp.

Where to post it: a private link opens for anyone who has the full URL, so treat it like a password (private and public). It's fine in a pull request on a private repository. On a public repository, attach the screenshot file you copied out instead. Don't post the link there.

5. The whole loop from outside#

This is the driver we timed. The two edit scripts stand in for the agent:

#!/usr/bin/env bash
# ui-loop.sh: fresh sandbox -> edit -> UI check (retry once) -> share.
set -u
t() { printf '%-28s %5.1fs\n' "$1" "$(echo "$(date +%s.%N) - $T0" | bc)"; }
T0=$(date +%s.%N)
id=$(sbx --json new --name ui-check --type small --ttl 3600 | jq -r 'select(.event=="ready") | .id')
t "sandbox ready ($id)"
sbx scp proj.tgz edit1.sh edit2.sh "$id":/home/user/ > /dev/null
sbx exec "$id" --timeout 300 "tar xzf proj.tgz && cd proj && npm install --no-audit --no-fund --silent && git init -q && git add -A && git -c user.name=agent -c user.email=agent@example.com commit -qm baseline"
t "repo copied, deps installed"
for attempt in 1 2; do
  sbx exec "$id" "bash edit$attempt.sh > /dev/null"          # the agent's edit
  if sbx exec "$id" --timeout 120 "proj/agent/ui-check.sh"; then t "check passed (attempt $attempt)"; break; fi
  t "check failed (attempt $attempt)"
done
sbx exec "$id" --timeout 60 "proj/agent/share.sh" > /dev/null
sbx scp "$id":/home/user/proj/artifacts/after-reset.png "$id":/home/user/proj/artifacts/REVIEW.md . > /dev/null
t "shared, artifacts copied out"
sandbox ready (sb_pwhuxh9cl6)   3.2s
repo copied, deps installed   13.8s
{"ok":false,"ms":1028,"checks":[{"name":"count is 0 after Reset","ok":false,"detail":"count=3"},{"name":"no console or page errors","ok":false,"detail":"cuont is not defined"}],…}
check failed (attempt 1)      16.3s
{"ok":true,"ms":798,"checks":[],…}
check passed (attempt 2)      17.9s
shared, artifacts copied out  26.2s

sbx exec returns the check's exit code, so the driver doesn't parse the JSON to decide. It parses it only to give the agent the reason. The copied after-reset.png is an 800 x 500 PNG showing "Count: 0" and both buttons. When the human is done, sbx delete $id --yes removes the sandbox, and both URLs with it. Afterwards, sbx info returned 404 not_found.

On small ($0.018/hour, half the default rate), a 30-second loop costs 30 / 3,600 × $0.018 = $0.00015. A sandbox kept running for an hour of human review costs $0.018.

Things that tripped us up#

  • pkill -f vite killed our own command. sbx exec runs your command in a shell, and that shell's command line contained "vite" too. The exec ended with exit 143. Write the pattern as pkill -f '[v]ite', which still matches the server but not the shell's own command line, or kill by PID.
  • sbx exec waits 30 s by default. A first npm install or a slow page can go past that. Pass --timeout, up to 600 s, before the command. For longer runs, start with --detach and poll (long-running jobs).
  • A check that only looks for the element passes a broken change. Our typo would have passed "Reset button is visible". Click it and assert the result.
  • Don't let the agent edit verify.mjs. If it can, it can make the check pass without fixing anything. Keep the check outside the agent's task, or check git diff --stat for changes under agent/.

FAQ#

Does the agent need a Sandboxer API key inside the sandbox?#

No. The in-sandbox host command uses the sandbox's own SANDBOXER_TOKEN, which is scoped to that one sandbox. We ran host url without setting any key. The sbx CLI isn't installed inside sandboxes, and the loop doesn't need it there.

Do I need to run npx playwright install?#

Not for Chrome. channel: "chrome" uses the Google Chrome preinstalled in every sandbox. In our first run, npm install of Playwright and Vite took 7.3 s, with no browser download.

Can a reviewer click around the live app, not just look at screenshots?#

Yes. The app link is the real dev server, hosted on a private URL. The reviewer can use the app until you delete the sandbox. For previews that last the life of a pull request, see a preview URL for every pull request.

What if my app needs a database or a backend?#

Start them in the same sandbox before the check, for example with Docker Compose, and keep their ports on 127.0.0.1. The check only needs the page to load at APP_URL. Set it if your app isn't on port 5173.

Did a real model make these edits?#

No. Two shell scripts made the edits, one with a typo and one with the fix, so the loop's timings don't include model time. Everything else, including Chrome, the hosted URLs and the screenshots, ran for real on prod.

More guides

Top AI sandbox platforms for code execution in 2026, ranked

For running AI-written code we rank Sandboxer, Freestyle and E2B first: each gives code its own VM with an API or SDK and can pause with memory. Pick Daytona for Windows or GPUs, Modal or Northflank for GPUs at scale, E2B or Northflank to run in your own cloud, and Vercel Sandbox if your app lives on Vercel.

Read the guide
Best sandboxes for running coding agents (Claude Code, Codex, Gemini CLI) in 2026, ranked

For coding agents we rank Sandboxer first: a full Ubuntu VM with Claude Code and Codex preinstalled, Docker, and stop-and-resume that keeps the agent's session in memory. Freestyle and Amp Orbs also keep memory across pauses. Pick E2B for prebuilt agent templates on open source, Docker Sandboxes for free local runs, and Codespaces if you live in GitHub.

Read the guide
Best Daytona sandbox alternatives in 2026, ranked

The best Daytona alternatives depend on why you're switching. For a full Linux microVM on every sandbox with memory-saving stop and Docker, we rank Sandboxer first. E2B suits open source and self-hosting, Modal massive short-job fleets, Northflank bring-your-own-cloud with GPUs, and Vercel Sandbox apps on Vercel.

Read the guide
Best E2B alternatives in 2026, ranked

If you're leaving E2B for longer sessions or a fuller machine, we rank Sandboxer first, then Freestyle: both give each sandbox a full Linux VM that keeps memory across pauses, with Docker and no short session cap. Pick Daytona for Windows or GPUs, Northflank for bring-your-own-cloud, and stay on E2B if you need open source.

Read the guide