# How to Run a Chroma Vector Database in a Cloud Sandbox

> Run the chromadb/chroma Docker image in a Sandboxer microVM with a named volume and --restart unless-stopped, host port 8000 on a private HTTPS URL, and give the Python HttpClient the URL's auth cookie as a header. Our query took about 2 ms inside the VM and 129 ms from outside.

Source: https://sandboxer.com/guides/run-chroma-vector-database-in-a-sandbox. Updated 2026-10-27.

**Short answer:** Run the chromadb/chroma Docker image in a Sandboxer microVM with a named volume and --restart unless-stopped, host port 8000 on a private HTTPS URL, and give the Python HttpClient the URL's auth cookie as a header. Our query took about 2 ms inside the VM and 129 ms from outside.

## What we ran

Everything below ran on production on October 27, 2026, in one `default` sandbox (4 vCPU, 8 GB). The server was the `chromadb/chroma` image (`latest` at the time, which reported `chroma 1.4.4`). The Python client was `chromadb` 1.5.9, both in the sandbox and on our client machine. The two versions worked together for everything on this page.

The data is real text: the 27 Markdown pages of the Sandboxer docs, split into 430 paragraphs. Chroma embedded them with its default model (`all-MiniLM-L6-v2`, run on the CPU through ONNX). There is no GPU in this setup.

| Step | Result |
| - | - |
| Pull the image and start Chroma | 7.3 s |
| First `add` (downloads the 79 MB default model) | 6.1 s |
| Embed and add 429 paragraphs on the sandbox CPU | 12.6 s (34 paragraphs per second) |
| Embed one query on the sandbox CPU | 248 to 302 ms |
| Top-5 query inside the VM (vector given) | 1.8 to 2.1 ms median over 20 runs |
| Same query from outside, over the private hosted URL | 128 to 129 ms median over 20 runs |
| Private URL without the token | `401` |
| `sbx stop --wait` then `sbx resume` | Memory restored: Chroma kept running, same URL and token worked |
| Delete and re-create the container on the same volume | Ready in 0.2 s, all 430 records there |

If you want vectors next to relational data in SQL, see [Postgres with pgvector in a sandbox](/guides/run-postgres-with-pgvector-in-a-sandbox) instead. For a vector server where you send your own embeddings and set an API key, see [Qdrant in a sandbox](/guides/run-qdrant-vector-database-in-a-sandbox). This page is about Chroma's own HTTP server and its Python client.

## 1. Start Chroma in a sandbox

Chroma documents a one-line Docker install ([Chroma Docker guide](https://docs.trychroma.com/guides/deploy/docker)). Docker is preinstalled in every sandbox. We added a restart policy and used a named volume instead of a bind mount:

```bash
sbx new --name chroma --type default --no-env
id=sb_m8h798mxvr    # the id that sbx new printed

sbx exec $id --timeout 300 "docker run -d --name chroma --restart unless-stopped \
  -p 127.0.0.1:8000:8000 -v chroma-data:/data chromadb/chroma"
sbx exec $id "until curl -sf localhost:8000/api/v2/heartbeat; do sleep 0.5; done; echo; docker exec chroma chroma --version"
```

```text
Status: Downloaded newer image for chromadb/chroma:latest
{"nanosecond heartbeat":1793095888453411535}
chroma 1.4.4
```

Chroma stores everything under `/data` in the container, in a SQLite file (`chroma.sqlite3`) plus index files. After loading our 430 paragraphs, the `chroma-data` volume was 2.4 MB. For anything you keep, pin the version you tested (`chromadb/chroma:1.4.4` here) instead of `latest`.

## 2. Load documents, embedding them in the sandbox

Chroma's Python client computes embeddings on the client side. With no `embedding_function` set, it uses `all-MiniLM-L6-v2`, which it downloads to `~/.cache/chroma` on first use. Running the loader inside the sandbox means the model download and the CPU work happen there, not on your laptop.

Install the client with `uv` (preinstalled):

```bash
sbx exec $id --timeout 600 "bash -lc 'uv venv -q -p 3.12 ~/venv && VIRTUAL_ENV=~/venv uv pip install -q chromadb==1.5.9'"
```

This is `ingest.py`. It splits each Markdown file on blank lines, skips code blocks and short lines, and adds the rest in batches of 100. The `configuration` argument sets cosine distance for the collection's HNSW index:

```python
import glob, os, time
import chromadb

paras = []
for path in sorted(glob.glob("docs/*.md")):
    text = open(path).read().split("\n---\n", 1)[-1]  # drop frontmatter
    for i, p in enumerate(text.split("\n\n")):
        p = p.strip()
        if len(p) > 80 and not p.startswith("```"):
            paras.append((f"{os.path.basename(path)[:-3]}#{i}", p))

client = chromadb.HttpClient(host="localhost", port=8000)
col = client.get_or_create_collection("docs", configuration={"hnsw": {"space": "cosine"}})

t0 = time.perf_counter()
col.add(ids=[paras[0][0]], documents=[paras[0][1]])  # first call loads the default model
print(f"first add (model load): {time.perf_counter() - t0:.1f} s")

t0 = time.perf_counter()
for i in range(1, len(paras), 100):
    batch = paras[i:i + 100]
    col.add(ids=[b[0] for b in batch], documents=[b[1] for b in batch])
dt = time.perf_counter() - t0
print(f"added {len(paras) - 1} paragraphs in {dt:.1f} s ({(len(paras) - 1) / dt:.0f}/s)")
print("count:", col.count())
```

```bash
sbx scp ./docs.tgz $id:/home/user/      # the docs folder and ingest.py
sbx exec $id --timeout 600 "tar xzf docs.tgz && ~/venv/bin/python ingest.py"
```

```text
first add (model load): 6.1 s
added 429 paragraphs in 12.6 s (34/s)
count: 430
```

The 6.1 seconds include the 79 MB model download. The 34 paragraphs per second is the embedding model on 4 vCPU; the database writes are a small part of it. The download progress bar goes to stderr, so add `2>/dev/null` if it clutters your logs.

To see what the server itself costs, we timed a query with the vector already computed (`lat.py`, run in the sandbox):

```python
import statistics, time
import chromadb

col = chromadb.HttpClient(host="localhost", port=8000).get_collection("docs")
t0 = time.perf_counter()
q = col._embedding_function(["how do I keep a process running after the sandbox resumes"])
print(f"embed one query on the sandbox CPU: {(time.perf_counter() - t0) * 1000:.0f} ms")
times = []
for _ in range(20):
    t0 = time.perf_counter()
    col.query(query_embeddings=q, n_results=5)
    times.append((time.perf_counter() - t0) * 1000)
print(f"query inside the VM: median {statistics.median(times):.1f} ms, min {min(times):.1f} ms (20 runs)")
```

```text
embed one query on the sandbox CPU: 302 ms
query inside the VM: median 2.1 ms, min 1.7 ms (20 runs)
```

A second run gave 248 ms and 1.8 ms. At this size, embedding the question takes about 100 times longer than searching for it. `_embedding_function` is a private attribute; we used it only to split the timing. In your own code, pass `query_texts` and let Chroma embed.

## 3. Query it from your machine over a private URL

Chroma speaks HTTP, so unlike Postgres it can go on a [hosted URL](/docs/hosting) instead of an SSH tunnel. Hosted URLs are private by default:

```bash
sbx host $id 8000 --localhost --title Chroma
curl -s -o /dev/null -w "%{http_code}\n" https://fine-civic-aspen-8000.box.sandboxer.com/api/v2/heartbeat
```

```text
https://fine-civic-aspen-8000.box.sandboxer.com?_token=<token>
401
```

A browser handles the `?_token=` for you: the first request answers `302` and sets an `__sbx_auth` cookie. Chroma's `HttpClient` takes a host and port, not a URL with a query string, and it doesn't keep cookies. Without the cookie, every call gets the proxy's `401` page, which the client reports as a `ValueError`:

```text
ValueError <!doctype html><html><head><meta charset="utf-8">…<title>Access token required</title>
```

So trade the token for the cookie once, then pass the cookie as a header on every request. This is `query.py`; it reads the full private URL from an environment variable so the token stays out of your code:

```python
# CHROMA_URL="https://…-8000.box.sandboxer.com?_token=…" python query.py
import os, statistics, time
from urllib.parse import urlsplit
import requests, chromadb

url = urlsplit(os.environ["CHROMA_URL"])
r = requests.get(os.environ["CHROMA_URL"], allow_redirects=False)  # trade ?_token= for the cookie
cookie = r.headers["set-cookie"].split(";")[0]

client = chromadb.HttpClient(host=url.hostname, port=443, ssl=True, headers={"Cookie": cookie})
print("heartbeat ok:", client.heartbeat() > 0)
col = client.get_collection("docs")
print("count:", col.count())

res = col.query(query_texts=["how do I keep a process running after the sandbox resumes"], n_results=3)
for i, d in zip(res["ids"][0], res["distances"][0]):
    print(f"{d:.3f}  {i}")

q = col._embedding_function(["how do I keep a process running after the sandbox resumes"])
times = []
for _ in range(20):
    t0 = time.perf_counter()
    col.query(query_embeddings=q, n_results=5)
    times.append((time.perf_counter() - t0) * 1000)
print(f"query over hosted URL: median {statistics.median(times):.0f} ms, min {min(times):.0f} ms (20 runs)")
```

```text
heartbeat ok: True
count: 430
0.251  openai-agents#22
0.297  snapshots#22
0.320  faq#3
query over hosted URL: median 129 ms, min 128 ms (20 runs)
```

The three hits are on topic. The third one, from the FAQ, starts: "Stop freezes the filesystem, writes the VM's memory and device state to disk, and shuts it down. Resume restores that memory, so processes keep running…". Filters work over the same connection too: `col.get(where_document={"$contains": "memoryRestored"})` returned the three paragraphs of the snapshots page that mention that field.

Two things to know about the client side:

- **The client embeds the question.** `query_texts` runs `all-MiniLM-L6-v2` on your machine, so the first query there downloads the same 79 MB model. Whatever embeds your queries must be the same model that embedded your documents, or the distances mean nothing.
- **Most of the 129 ms is the network.** The search takes about 2 ms in the VM; the rest is the round trip from our client to the sandbox through the HTTPS proxy, and it depends on where you are. An agent that queries in a loop should run in the same sandbox and use `localhost:8000`.

The cookie is the route's access token, so treat it like the URL: keep it out of logs. To cut off everyone who has the link, run `sbx host $id 8000 --localhost --public` and then the same with `--private`; that mints a new token. Keep `--localhost` on every `sbx host` call for this port, since each call sets the relay mode again. `sbx host hide $id 8000` takes the URL down and leaves Chroma running.

Chroma also has its own server configuration file ([Chroma Docker guide](https://docs.trychroma.com/guides/deploy/docker), mounted at `/config.yaml`). We didn't configure Chroma's own auth here: the private URL was the only gate we tested.

### A note on the published port

The commands above publish Chroma as `-p 127.0.0.1:8000:8000` and host it with `--localhost`, so the hosted URL relays to the VM's loopback. The sandbox also has a public IPv6 address. In our check on October 27, 2026, its firewall (`ufw`, closed except SSH) blocked a port Docker published on all interfaces, but Docker can [bypass ufw rules](https://docs.docker.com/engine/network/packet-filtering-firewalls/#docker-and-ufw) on other setups. Binding to `127.0.0.1` keeps the private URL the only way in, whatever the firewall does. Our test run used `-p 8000:8000` and a plain `sbx host`; the timings above come from that run.

## 4. What survives a stop, a resume and an upgrade

```bash
sbx stop $id --wait
sbx resume $id
sbx exec $id "docker ps --format '{{.Names}} {{.Status}}'"
```

```text
chroma Up About a minute
```

The stop took 2.2 seconds and the resume 1.4 seconds, with `memoryRestored: true`. The container was never restarted. The hosted URL kept the same token, and `query.py` returned the same three hits and a 128 ms median.

When a resume boots from disk instead (no machine with the same CPU type is free, or you change the size), running processes are gone. `--restart unless-stopped` makes Docker start Chroma again, and the data is on the `chroma-data` volume. See [snapshots](/docs/snapshots#resume) for when each path happens.

The same volume is how you upgrade. We removed the container and started a new one with the same command:

```bash
sbx exec $id --timeout 120 "docker rm -f chroma >/dev/null; docker run -d --name chroma --restart unless-stopped \
  -p 127.0.0.1:8000:8000 -v chroma-data:/data chromadb/chroma >/dev/null; \
  until curl -sf localhost:8000/api/v2/heartbeat >/dev/null; do sleep 0.2; done; \
  ~/venv/bin/python -c 'import chromadb; print(chromadb.HttpClient().get_collection(\"docs\").count())'"
```

```text
430
```

The new container answered in 0.2 seconds with every record in place. To move to a new Chroma version, change the image tag in that command. Check Chroma's release notes first, and fork the sandbox before you try it ([fork copies the disk](/docs/snapshots)), so you can go back.

## 5. Clean up

```bash
sbx delete $id --yes
```

A `default` sandbox costs $0.036/hour while it runs and nothing while it is stopped.

## FAQ

### Why does chromadb.HttpClient fail with an "Access token required" ValueError?

The hosted URL is private and the client sent neither the token nor the cookie. `HttpClient` can't carry `?_token=` in its host, so fetch the URL once with redirects off, take the `__sbx_auth` cookie from the `Set-Cookie` header, and pass it with `headers={"Cookie": cookie}`.

### Where do the embeddings get computed?

In the client, not the Chroma server. With the default embedding function, the Python client runs `all-MiniLM-L6-v2` with ONNX wherever the client runs. On a `default` sandbox, that was 34 paragraphs per second for loading and about 250 to 300 ms per query.

### Will my collections survive if the sandbox auto-stops?

Yes. An auto-stop is a normal stop: disk and memory are saved and `sbx resume` brings them back. The data also lives on the `chroma-data` Docker volume, so it survives a resume that boots from disk and a re-created container.

### Should I use Chroma or pgvector?

Use pgvector if your data is already in Postgres or you want SQL joins and transactions over the same rows. Use Chroma if you want a small HTTP service with a Python client that also embeds text for you. Both ran fine on a `default` sandbox at our sizes; measure at yours.

### Can I make the Chroma URL public?

`sbx host $id 8000 --localhost --public` removes the token, so anyone with the URL can read, write and delete your collections. We kept it private.

## Related

- [Run Postgres with pgvector in a sandbox](/guides/run-postgres-with-pgvector-in-a-sandbox)
- [Give a sandbox port an HTTPS URL](/guides/give-a-sandbox-port-an-https-url)
- [Run a Qdrant vector database in a sandbox](/guides/run-qdrant-vector-database-in-a-sandbox)
- [Hosting](/docs/hosting)