# How Do I Run Ollama on CPU in a Cloud Sandbox? > Install Ollama with its Linux script in a default sandbox, pull a small model such as qwen2.5:0.5b, and bind it to 0.0.0.0 with a systemd override. Then sbx host the port for a private HTTPS URL. There is no GPU: expect about 30 to 55 tokens per second for 0.5B and 13 to 22 for 1B. Source: https://sandboxer.com/guides/run-ollama-on-cpu-in-a-cloud-sandbox. Updated 2026-10-27. **Short answer:** Install Ollama with its Linux script in a default sandbox, pull a small model such as qwen2.5:0.5b, and bind it to 0.0.0.0 with a systemd override. Then sbx host the port for a private HTTPS URL. There is no GPU: expect about 30 to 55 tokens per second for 0.5B and 13 to 22 for 1B. ## At a glance We ran everything below on October 27, 2026, on a `default` sandbox (4 vCPU, 8 GB RAM). For the last test we resumed it as `large` (8 vCPU, 16 GB RAM). Ollama `0.40.11`. **CPU only:** a sandbox has no GPU, and Ollama runs every layer on the CPU (`ollama ps` shows `100% CPU`). | Step | What we saw | | - | - | | Install script | 17 s, creates an enabled `ollama` systemd service on `127.0.0.1:11434` | | `ollama pull qwen2.5:0.5b` / `llama3.2:1b` | 397 MB in 7 s / 1.3 GB in 16 s | | Generation speed, `default` | `qwen2.5:0.5b` 28 to 57 tokens/s, `llama3.2:1b` 13 to 19 tokens/s | | Generation speed, `large` | `qwen2.5:0.5b` 50 to 59 tokens/s, `llama3.2:1b` 17 to 22 tokens/s | | `sbx host --localhost` with Ollama on 127.0.0.1 | `403` from Ollama | | `OLLAMA_HOST=0.0.0.0:11434`, then `sbx host` | `401` without the token, `200` with it | | Short request through the hosted URL | 0.3 to 0.55 s round trip | | `sbx stop --wait` + `sbx resume` | `memoryRestored: true`, same PID, model still loaded | | Cold boot (resume as `large`) | Service came back on its own, first request reloaded the model (1.4 s) | The speed ranges are real. The same prompt on the same sandbox varied by up to 2x between runs a minute apart. Plan for the low end. ## 1. Create a sandbox and install Ollama Use `default`. `small` has 4 GB of RAM, which is tight for anything above 1B. Install with the script from [Ollama's Linux docs](https://docs.ollama.com/linux): ```bash sbx new --name ollama --type default id=sb_o32n3ltk3i # the id that sbx new printed sbx exec $id "curl -fsSL https://ollama.com/install.sh | sh" ``` ```text WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies. >>> Installing ollama to /usr/local >>> Downloading ollama-linux-amd64.tar.zst … >>> Creating ollama systemd service... >>> Enabling and starting ollama service... Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service. ``` The GPU warning is expected. Ignore it. ## 2. Pull a small model and run a prompt Pick models from the [Ollama library](https://ollama.com/library) that fit in RAM with room to spare. We used [`qwen2.5:0.5b`](https://ollama.com/library/qwen2.5) and [`llama3.2:1b`](https://ollama.com/library/llama3.2): ```bash sbx exec $id "ollama pull qwen2.5:0.5b && ollama pull llama3.2:1b && ollama list" sbx exec $id "ollama run qwen2.5:0.5b 'Write a haiku about a sandbox.'" ``` ```text NAME ID SIZE MODIFIED llama3.2:1b baf6a787fdff 1.3 GB Less than a second ago qwen2.5:0.5b a8b0c5157701 397 MB 16 seconds ago In the sandbox, children play and laugh, playground laughter flows. ``` ## 3. Measure tokens per second Ollama's [generate API](https://docs.ollama.com/api/generate) returns `eval_count` (tokens generated) and `eval_duration` (nanoseconds). Tokens per second is `eval_count / eval_duration * 1e9`. Save this as `bench.sh` and run it in the sandbox with `sbx exec $id "$(cat bench.sh)"`: ```bash for m in qwen2.5:0.5b llama3.2:1b; do for i in 1 2 3; do curl -s http://127.0.0.1:11434/api/generate -d "{\"model\":\"$m\",\"prompt\":\"Explain what a microVM is in three sentences.\",\"stream\":false,\"options\":{\"num_predict\":128,\"temperature\":0}}" \ | python3 -c "import json,sys; r=json.load(sys.stdin); print('$m', 'run $i', 'load_s=%.2f'%(r['load_duration']/1e9), 'prompt_tok_s=%.1f'%(r['prompt_eval_count']/r['prompt_eval_duration']*1e9), 'eval_count=%d'%r['eval_count'], 'tok_s=%.1f'%(r['eval_count']/r['eval_duration']*1e9))" done done ``` On `default`: ```text qwen2.5:0.5b run 1 load_s=0.00 prompt_tok_s=575.4 eval_count=111 tok_s=56.3 qwen2.5:0.5b run 2 load_s=0.00 prompt_tok_s=1763.0 eval_count=111 tok_s=48.2 qwen2.5:0.5b run 3 load_s=0.00 prompt_tok_s=1458.7 eval_count=111 tok_s=57.4 llama3.2:1b run 1 load_s=0.00 prompt_tok_s=355.7 eval_count=107 tok_s=19.4 llama3.2:1b run 2 load_s=0.00 prompt_tok_s=648.7 eval_count=107 tok_s=16.3 llama3.2:1b run 3 load_s=0.00 prompt_tok_s=307.9 eval_count=107 tok_s=16.9 ``` A pass a minute earlier gave 28 to 30 tokens/s for `qwen2.5:0.5b` and 13 to 15 for `llama3.2:1b`. The first request for a model also paid `load_s=2.57` to load `llama3.2:1b` from disk. On `large`, with the same script: ```text qwen2.5:0.5b run 2 load_s=0.00 prompt_tok_s=1460.5 eval_count=111 tok_s=51.3 qwen2.5:0.5b run 3 load_s=0.00 prompt_tok_s=1352.5 eval_count=111 tok_s=52.3 llama3.2:1b run 2 load_s=0.00 prompt_tok_s=669.0 eval_count=107 tok_s=18.0 llama3.2:1b run 3 load_s=0.01 prompt_tok_s=527.1 eval_count=107 tok_s=22.4 … ``` Doubling the vCPUs didn't double the speed. With `"num_thread":8` in `options`, `llama3.2:1b` on `large` went from 17 to 19–22 tokens/s, against 17 with 4 threads. For one small model, `default` at $0.036 an hour is the better buy than `large` at twice that. Choose `large` when you need its 16 GB of RAM for a bigger model. `large` isn't available during the free trial. ## 4. Expose the API on a private URL Ollama listens on `127.0.0.1:11434` by default. We tried both ways to reach it from outside. **`sbx host --localhost` doesn't work for Ollama.** The relay reaches the port, but Ollama rejects requests whose `Host` header isn't local. The proxy passed the request with the token, and Ollama answered `403`. Inside the sandbox, `curl -H "Host: -11434.box.sandboxer.com" http://127.0.0.1:11434/api/tags` also got `403`. **Binding to `0.0.0.0` works.** [Ollama's FAQ](https://docs.ollama.com/faq) sets `OLLAMA_HOST` on the systemd service. Then host the port without `--localhost`: ```bash sbx exec $id "sudo mkdir -p /etc/systemd/system/ollama.service.d && \ printf '[Service]\nEnvironment=\"OLLAMA_HOST=0.0.0.0:11434\"\n' | sudo tee /etc/systemd/system/ollama.service.d/override.conf && \ sudo systemctl daemon-reload && sudo systemctl restart ollama && ss -ltn | grep 11434" sbx host $id 11434 --title Ollama ``` ```text LISTEN 0 4096 *:11434 *:* https://maple-glossy-dune-11434.box.sandboxer.com?_token= ``` Before the override, the same URL returned `502` ("Nothing is answering on port 11434"). Ollama has no authentication of its own. The hosted URL's token is the lock, so keep the port private. Binding to `0.0.0.0` doesn't open the port to the internet: the sandbox firewall allows only SSH, and the hosting proxy reaches only ports you host. See [what can reach a sandbox](/guides/what-can-reach-a-sandbox-and-what-it-can-reach). ## 5. Call it from outside with curl Put the token in the query string. A `GET` with the token answers `302` and sets a cookie, so use `-L` and a cookie jar. A `POST` with the token goes straight through. ```bash BASE=https://maple-glossy-dune-11434.box.sandboxer.com TOKEN= curl -s -o /dev/null -w "%{http_code}\n" $BASE/api/tags # 401 curl -sL -c jar -b jar "$BASE/api/tags?_token=$TOKEN" # 200, the model list curl -s "$BASE/api/generate?_token=$TOKEN" \ -d '{"model":"qwen2.5:0.5b","prompt":"Reply with one word: ready?","stream":false}' ``` ```text 401 {"models":[{"name":"llama3.2:1b","model":"llama3.2:1b","modified_at":"2026-10-27T09:23:12.624999959Z","size":1321098329,… {"model":"qwen2.5:0.5b","created_at":"2026-10-27T09:25:35.546740041Z","response":"ready","done":true,"done_reason":"stop",…,"load_duration":794796193,…} ``` Ollama's [OpenAI-compatible endpoint](https://docs.ollama.com/api/openai-compatibility) works through the same URL: ```bash curl -s "$BASE/v1/chat/completions?_token=$TOKEN" -H "Content-Type: application/json" \ -d '{"model":"qwen2.5:0.5b","messages":[{"role":"user","content":"Name one prime number. Reply with the number only."}]}' ``` ```text {"id":"chatcmpl-685","object":"chat.completion",…,"choices":[{"index":0,"message":{"role":"assistant","content":"1"},"finish_reason":"stop"}],"usage":{"prompt_tokens":40,…,"completion_tokens":2,"total_tokens":42}} ``` That answer is wrong: 1 isn't prime. A 0.5B model is fast on a CPU and fine for classification, extraction or test fixtures. Don't rely on it for facts or reasoning. Five short requests (`num_predict: 8`) from our workstation to the sandbox in eu-central took 0.38 to 0.55 s each. ## 6. Stop, resume and cold boot ```bash sbx stop $id --wait sbx resume $id sbx exec $id "ollama ps; pgrep -f 'ollama serve'" ``` The resume took 1.4 s and reported `memoryRestored: true`. The server kept PID 947, `qwen2.5:0.5b` was still loaded, and the hosted URL answered at once with `load_duration` of 1.2 ms. The URL and token didn't change. A resume that boots from disk loses running processes. That happens when the resume lands on a different CPU type, or when you change size. We stopped again and ran `sbx resume $id --type large` (`memoryRestored: false`, uptime 0 min). Ollama came back because the install script enabled its systemd service, and our `OLLAMA_HOST` override came back with it. The first request through the URL returned `502` for a moment, then `200` in 1.4 s while the model loaded from disk. Later requests took 0.3 s. Retry a `502` right after a cold boot. Don't start Ollama with `ollama serve &` in a sandbox you plan to stop: nothing would restart it after a cold boot. ## FAQ ### Does a Sandboxer sandbox have a GPU? No. Sandboxes are CPU-only microVMs, and Ollama runs the whole model on the CPU. Use small models. In our test, 0.5B models ran at 28 to 59 tokens per second and 1B models at 13 to 22. Bigger models are slower. ### Which model size fits which sandbox? The model has to fit in RAM next to everything else. `llama3.2:1b` used 1.5 GB loaded in our test, out of 8 GB on `default`. For models above about 3B, use `large` (16 GB) and expect lower tokens per second. ### Why does `sbx host --localhost` return 403 for Ollama? Ollama checks the `Host` header when it listens on loopback and rejects names it doesn't know. Set `OLLAMA_HOST=0.0.0.0:11434` on the service and host the port without `--localhost`. ### Is my Ollama API open to the internet? Only through the hosted URL, and that URL is private by default. Requests without the token get `401` before they reach Ollama. Don't make the port `--public`, because Ollama has no login of its own. ### Does the model stay loaded while the sandbox is stopped? Nothing runs while it is stopped, and nothing is billed. After a memory-restoring resume, the model was still in memory. After a cold boot, Ollama loads it again on the first request. ## Related - [Run Open WebUI with Ollama in a sandbox](/guides/run-open-webui-with-ollama-in-a-sandbox) - [Give a sandbox port an HTTPS URL](/guides/give-a-sandbox-port-an-https-url) - [Pause and resume a cloud sandbox with its memory](/guides/pause-and-resume-a-sandbox-with-memory) - [Hosting](/docs/hosting)