# Run a Computer-Use Agent on a Linux Desktop in the Cloud

> Start a Sandboxer sandbox and open its desktop once with sbx desktop, which starts XFCE with Chrome on display :1. Give your agent two kinds of tools that run inside it, a screenshot (ffmpeg x11grab) and actions (xdotool click, type, key), and loop. The desktop URL lets you watch the screen live.

Source: https://sandboxer.com/guides/run-a-computer-use-agent-on-a-linux-desktop. Updated 2026-10-27.

**Short answer:** Start a Sandboxer sandbox and open its desktop once with sbx desktop, which starts XFCE with Chrome on display :1. Give your agent two kinds of tools that run inside it, a screenshot (ffmpeg x11grab) and actions (xdotool click, type, key), and loop. The desktop URL lets you watch the screen live.

## At a glance

| Topic | What we found on a fresh `default` sandbox |
| - | - |
| Desktop | XFCE on Xvfb, display `:1`, 1920×1080, Chrome open and maximized. It starts the first time you open it with `sbx desktop` |
| Already installed | `xdotool`, `ffmpeg`, `wmctrl`, `xclip`, Python 3 with Pillow 10.2 |
| Not installed | `scrot`, ImageMagick `import`, `xwd` (use `ffmpeg -f x11grab` instead) |
| `DISPLAY` in `sbx exec` | Not set: export `DISPLAY=:1` yourself |
| Watch it | `sbx desktop <id>` returns a noVNC URL that works for ten minutes |
| Cost | `default` is 4 vCPU, 8 GB RAM, $0.036/hour, billed per second |

A computer-use agent is a loop: take a screenshot, send it to the model, run the action the model picks, repeat. The model only sees pixels and only acts through mouse and keyboard. All you have to supply is a screen and those two kinds of tool. Everything below ran on production. We called the tool functions directly and did not run a model, because this page is about the sandbox side. Plug them into whichever model's computer-use API you use.

## 1. Start a sandbox and find the screen

```bash
sbx new --type default
id=sb_4943jbsept    # the id from sbx new
sbx desktop $id --no-open    # starts the desktop and prints a viewer URL
sbx exec $id "echo DISPLAY=\$DISPLAY; pgrep -a Xvfb"
```

```text
DISPLAY=
569 /usr/bin/Xvfb :1 -screen 0 1920x1080x24 -noreset -nolisten tcp +extension RANDR
```

The desktop starts the first time you open it, so run `sbx desktop` before any screenshot or `xdotool` call. Before that there is no X server, and a headed program fails with `Missing X server or $DISPLAY` (see [Playwright with real Chrome](/guides/run-browser-use-or-playwright-with-real-chrome)). Commands from `sbx exec` don't inherit `DISPLAY`, so point tools at `:1`:

```bash
sbx exec $id "DISPLAY=:1 xdotool getdisplaygeometry; DISPLAY=:1 xdotool getmouselocation"
```

```text
1920 1080
x:960 y:540 screen:0 window:20971523
```

With `DISPLAY=:0`, the same `xdotool` call fails with `Can't open display`.

## 2. Write the tool functions

Put these in `computer.py`. They run inside the sandbox, so the screenshot never crosses the network until your agent asks for it.

```python
# computer.py: the tool functions a computer-use agent calls, run inside the sandbox.
import os, subprocess, time
from PIL import Image, ImageChops

os.environ["DISPLAY"] = ":1"  # the sandbox desktop

def screenshot(path):
    subprocess.run(["ffmpeg", "-loglevel", "error", "-y", "-f", "x11grab",
                    "-video_size", "1920x1080", "-i", ":1", "-frames:v", "1", path], check=True)
    return path

def xdo(*args):
    return subprocess.run(["xdotool", *map(str, args)], check=True,
                          capture_output=True, text=True).stdout.strip()

def click(x, y):
    xdo("mousemove", x, y, "click", 1)

def type_text(text):
    xdo("type", "--delay", 20, text)

def key(combo):
    xdo("key", combo)

def active_window():
    try:
        return xdo("getactivewindow", "getwindowname")
    except subprocess.CalledProcessError:
        return None  # bare desktop: nothing focused

def changed_box(a, b):
    return ImageChops.difference(Image.open(a).convert("RGB"),
                                 Image.open(b).convert("RGB")).getbbox()
```

`changed_box` is optional. It returns the rectangle that differs between two screenshots, which is a cheap way to check that an action did something before you spend a model call on the next screenshot.

## 3. Run one look, act, look cycle

Add this to the bottom of `computer.py`. It stands in for the model: it opens a terminal, clicks into it, types a command, presses Enter and takes a second screenshot.

```python
if __name__ == "__main__":
    # 1. Look
    before = screenshot("/home/user/step0.png")
    print("step0", Image.open(before).size, "active:", active_window())

    # 2. Act: open a terminal, wait for its window
    subprocess.Popen(["xfce4-terminal", "--title", "agent-term"])
    for _ in range(50):
        try:
            wid = xdo("search", "--sync", "--onlyvisible", "--name", "agent-term")
            break
        except subprocess.CalledProcessError:
            time.sleep(0.2)
    geo = xdo("getwindowgeometry", wid.splitlines()[0])
    print(geo.replace("\n", " | "))

    # 3. Act: click inside the window, type a command, press Enter
    pos = geo.split("Position: ")[1].split(" ")[0]
    x, y = map(int, pos.split(","))
    click(x + 100, y + 100)
    type_text("echo hello from the agent > ~/agent.txt; date")
    key("Return")
    time.sleep(1)

    # 4. Look again and compare
    after = screenshot("/home/user/step1.png")
    print("step1", Image.open(after).size, "active:", active_window())
    print("changed region:", changed_box(before, after))
    print("file written:", open("/home/user/agent.txt").read().strip())
```

```bash
sbx scp ./computer.py $id:/home/user/computer.py
sbx exec $id "python3 ~/computer.py"
```

```text
step0 (1920, 1080) active: None
Window 52428803 |   Position: 10,85 (screen: 0) |   Geometry: 817x483
step1 (1920, 1080) active: agent-term
changed region: (0, 0, 1606, 564)
file written: hello from the agent
```

The first screenshot showed Chrome on its New Tab page. The second showed the terminal on top of it, with the command, its output and a fresh prompt. The file on disk confirms the keystrokes reached the right window. Each PNG was 1920×1080 and 55–77 KB.

## 4. Drive the browser the same way

Chrome is the window most agents need. This step focuses it, uses the address bar shortcut and loads a page:

```python
# browse.py
import time
from computer import xdo, key, type_text, screenshot, active_window, changed_box

wid = xdo("search", "--onlyvisible", "--class", "google-chrome").splitlines()[-1]
xdo("windowactivate", "--sync", wid)
key("ctrl+l")
type_text("example.com")
key("Return")
time.sleep(3)
print("active:", active_window())
print("changed region:", changed_box("/home/user/step1.png", screenshot("/home/user/step2.png")))
```

```bash
sbx scp ./browse.py $id:/home/user/browse.py
sbx exec $id "cd ~ && python3 browse.py"
```

```text
active: Example Domain - Google Chrome
changed region: (0, 0, 1920, 1080)
```

The window title is a useful second signal next to the pixels: it tells your loop the page loaded without a model call.

## 5. Get the screenshots out

When the model runs outside the sandbox, fetch each screenshot and send it with the next request:

```bash
sbx scp $id:/home/user/step1.png .
```

For a faster loop, run the whole agent inside the sandbox instead. Then only model API calls leave the VM.

## 6. Watch it live

```bash
sbx desktop $id --no-open
```

The response has `"mode": "vnc"` and a `desktopUrl` that opens noVNC in a browser. We checked it from outside: the page loaded (HTTP 200, title `noVNC`) and its WebSocket answered with the VNC handshake. The first request, made a second after the sandbox became ready, got a "Not found" page; a few seconds later the same call worked. That matches the docs: the first open takes a few seconds while the desktop starts. In a script, retry the desktop URL for a few seconds before your first screenshot.

The URL carries a secret token and works for ten minutes. Anyone with it can see and control the screen, so keep it out of logs and shared chats. You can watch the agent and take over with your own mouse at the same time.

## Built-in agents already have these tools

If you don't want to write the loop, Sandboxer's [integrated agents](/docs/agents#computer-use) get `computer` tools (screenshot, mouse, keyboard) that act on this same desktop. They use your own model key. We didn't run them for this page.

## FAQ

### Which display number does the desktop use?

`:1`. Commands run with `sbx exec` don't have `DISPLAY` set, so export `DISPLAY=:1` or pass `-i :1` to ffmpeg. `:0` does not exist.

### Can I change the screen size?

Yes. The screen starts at 1920×1080, and you can change it inside the sandbox with `xrandr`. A smaller screen means smaller screenshots and fewer image tokens per step.

### Does the desktop survive a stop?

Usually. Stopping a sandbox saves its memory, so open windows and a running agent are still there after `sbx resume` when it lands on a machine with the same CPU type. If none is free within 2 minutes, it boots from disk: files are kept, but windows and processes are not, so open the desktop again and restart your agent (see [snapshots](/docs/snapshots#resume)).

### Do I need an LLM key to try this?

No. The screenshot and action tools are plain Linux programs. You only need a model key when you connect a model to choose the actions.

## Related

- [Desktop](/docs/desktop)
- [Integrated agents](/docs/agents)
- [Best sandboxes for browser and computer-use agents](/guides/best-sandboxes-for-browser-and-computer-use-agents)
- [Move files in and out of a sandbox](/guides/move-files-in-and-out-of-a-sandbox)