Skip to content
sandboxer
Start free trial
Guides · Recipes

How to Run a Chroma Vector Database in a Cloud Sandbox

Updated October 27, 2026 .md

Short answer: Run the chromadb/chroma Docker image in a Sandboxer microVM with a named volume and --restart unless-stopped, host port 8000 on a private HTTPS URL, and give the Python HttpClient the URL's auth cookie as a header. Our query took about 2 ms inside the VM and 129 ms from outside.

What we ran#

Everything below ran on production on October 27, 2026, in one default sandbox (4 vCPU, 8 GB). The server was the chromadb/chroma image (latest at the time, which reported chroma 1.4.4). The Python client was chromadb 1.5.9, both in the sandbox and on our client machine. The two versions worked together for everything on this page.

The data is real text: the 27 Markdown pages of the Sandboxer docs, split into 430 paragraphs. Chroma embedded them with its default model (all-MiniLM-L6-v2, run on the CPU through ONNX). There is no GPU in this setup.

Step Result
Pull the image and start Chroma 7.3 s
First add (downloads the 79 MB default model) 6.1 s
Embed and add 429 paragraphs on the sandbox CPU 12.6 s (34 paragraphs per second)
Embed one query on the sandbox CPU 248 to 302 ms
Top-5 query inside the VM (vector given) 1.8 to 2.1 ms median over 20 runs
Same query from outside, over the private hosted URL 128 to 129 ms median over 20 runs
Private URL without the token 401
sbx stop --wait then sbx resume Memory restored: Chroma kept running, same URL and token worked
Delete and re-create the container on the same volume Ready in 0.2 s, all 430 records there

If you want vectors next to relational data in SQL, see Postgres with pgvector in a sandbox instead. For a vector server where you send your own embeddings and set an API key, see Qdrant in a sandbox. This page is about Chroma's own HTTP server and its Python client.

1. Start Chroma in a sandbox#

Chroma documents a one-line Docker install (Chroma Docker guide). Docker is preinstalled in every sandbox. We added a restart policy and used a named volume instead of a bind mount:

sbx new --name chroma --type default --no-env
id=sb_m8h798mxvr    # the id that sbx new printed

sbx exec $id --timeout 300 "docker run -d --name chroma --restart unless-stopped \
  -p 127.0.0.1:8000:8000 -v chroma-data:/data chromadb/chroma"
sbx exec $id "until curl -sf localhost:8000/api/v2/heartbeat; do sleep 0.5; done; echo; docker exec chroma chroma --version"
Status: Downloaded newer image for chromadb/chroma:latest
{"nanosecond heartbeat":1793095888453411535}
chroma 1.4.4

Chroma stores everything under /data in the container, in a SQLite file (chroma.sqlite3) plus index files. After loading our 430 paragraphs, the chroma-data volume was 2.4 MB. For anything you keep, pin the version you tested (chromadb/chroma:1.4.4 here) instead of latest.

2. Load documents, embedding them in the sandbox#

Chroma's Python client computes embeddings on the client side. With no embedding_function set, it uses all-MiniLM-L6-v2, which it downloads to ~/.cache/chroma on first use. Running the loader inside the sandbox means the model download and the CPU work happen there, not on your laptop.

Install the client with uv (preinstalled):

sbx exec $id --timeout 600 "bash -lc 'uv venv -q -p 3.12 ~/venv && VIRTUAL_ENV=~/venv uv pip install -q chromadb==1.5.9'"

This is ingest.py. It splits each Markdown file on blank lines, skips code blocks and short lines, and adds the rest in batches of 100. The configuration argument sets cosine distance for the collection's HNSW index:

import glob, os, time
import chromadb

paras = []
for path in sorted(glob.glob("docs/*.md")):
    text = open(path).read().split("\n---\n", 1)[-1]  # drop frontmatter
    for i, p in enumerate(text.split("\n\n")):
        p = p.strip()
        if len(p) > 80 and not p.startswith("```"):
            paras.append((f"{os.path.basename(path)[:-3]}#{i}", p))

client = chromadb.HttpClient(host="localhost", port=8000)
col = client.get_or_create_collection("docs", configuration={"hnsw": {"space": "cosine"}})

t0 = time.perf_counter()
col.add(ids=[paras[0][0]], documents=[paras[0][1]])  # first call loads the default model
print(f"first add (model load): {time.perf_counter() - t0:.1f} s")

t0 = time.perf_counter()
for i in range(1, len(paras), 100):
    batch = paras[i:i + 100]
    col.add(ids=[b[0] for b in batch], documents=[b[1] for b in batch])
dt = time.perf_counter() - t0
print(f"added {len(paras) - 1} paragraphs in {dt:.1f} s ({(len(paras) - 1) / dt:.0f}/s)")
print("count:", col.count())
sbx scp ./docs.tgz $id:/home/user/      # the docs folder and ingest.py
sbx exec $id --timeout 600 "tar xzf docs.tgz && ~/venv/bin/python ingest.py"
first add (model load): 6.1 s
added 429 paragraphs in 12.6 s (34/s)
count: 430

The 6.1 seconds include the 79 MB model download. The 34 paragraphs per second is the embedding model on 4 vCPU; the database writes are a small part of it. The download progress bar goes to stderr, so add 2>/dev/null if it clutters your logs.

To see what the server itself costs, we timed a query with the vector already computed (lat.py, run in the sandbox):

import statistics, time
import chromadb

col = chromadb.HttpClient(host="localhost", port=8000).get_collection("docs")
t0 = time.perf_counter()
q = col._embedding_function(["how do I keep a process running after the sandbox resumes"])
print(f"embed one query on the sandbox CPU: {(time.perf_counter() - t0) * 1000:.0f} ms")
times = []
for _ in range(20):
    t0 = time.perf_counter()
    col.query(query_embeddings=q, n_results=5)
    times.append((time.perf_counter() - t0) * 1000)
print(f"query inside the VM: median {statistics.median(times):.1f} ms, min {min(times):.1f} ms (20 runs)")
embed one query on the sandbox CPU: 302 ms
query inside the VM: median 2.1 ms, min 1.7 ms (20 runs)

A second run gave 248 ms and 1.8 ms. At this size, embedding the question takes about 100 times longer than searching for it. _embedding_function is a private attribute; we used it only to split the timing. In your own code, pass query_texts and let Chroma embed.

3. Query it from your machine over a private URL#

Chroma speaks HTTP, so unlike Postgres it can go on a hosted URL instead of an SSH tunnel. Hosted URLs are private by default:

sbx host $id 8000 --localhost --title Chroma
curl -s -o /dev/null -w "%{http_code}\n" https://fine-civic-aspen-8000.box.sandboxer.com/api/v2/heartbeat
https://fine-civic-aspen-8000.box.sandboxer.com?_token=<token>
401

A browser handles the ?_token= for you: the first request answers 302 and sets an __sbx_auth cookie. Chroma's HttpClient takes a host and port, not a URL with a query string, and it doesn't keep cookies. Without the cookie, every call gets the proxy's 401 page, which the client reports as a ValueError:

ValueError <!doctype html><html><head><meta charset="utf-8">…<title>Access token required</title>

So trade the token for the cookie once, then pass the cookie as a header on every request. This is query.py; it reads the full private URL from an environment variable so the token stays out of your code:

# CHROMA_URL="https://…-8000.box.sandboxer.com?_token=…" python query.py
import os, statistics, time
from urllib.parse import urlsplit
import requests, chromadb

url = urlsplit(os.environ["CHROMA_URL"])
r = requests.get(os.environ["CHROMA_URL"], allow_redirects=False)  # trade ?_token= for the cookie
cookie = r.headers["set-cookie"].split(";")[0]

client = chromadb.HttpClient(host=url.hostname, port=443, ssl=True, headers={"Cookie": cookie})
print("heartbeat ok:", client.heartbeat() > 0)
col = client.get_collection("docs")
print("count:", col.count())

res = col.query(query_texts=["how do I keep a process running after the sandbox resumes"], n_results=3)
for i, d in zip(res["ids"][0], res["distances"][0]):
    print(f"{d:.3f}  {i}")

q = col._embedding_function(["how do I keep a process running after the sandbox resumes"])
times = []
for _ in range(20):
    t0 = time.perf_counter()
    col.query(query_embeddings=q, n_results=5)
    times.append((time.perf_counter() - t0) * 1000)
print(f"query over hosted URL: median {statistics.median(times):.0f} ms, min {min(times):.0f} ms (20 runs)")
heartbeat ok: True
count: 430
0.251  openai-agents#22
0.297  snapshots#22
0.320  faq#3
query over hosted URL: median 129 ms, min 128 ms (20 runs)

The three hits are on topic. The third one, from the FAQ, starts: "Stop freezes the filesystem, writes the VM's memory and device state to disk, and shuts it down. Resume restores that memory, so processes keep running…". Filters work over the same connection too: col.get(where_document={"$contains": "memoryRestored"}) returned the three paragraphs of the snapshots page that mention that field.

Two things to know about the client side:

  • The client embeds the question. query_texts runs all-MiniLM-L6-v2 on your machine, so the first query there downloads the same 79 MB model. Whatever embeds your queries must be the same model that embedded your documents, or the distances mean nothing.
  • Most of the 129 ms is the network. The search takes about 2 ms in the VM; the rest is the round trip from our client to the sandbox through the HTTPS proxy, and it depends on where you are. An agent that queries in a loop should run in the same sandbox and use localhost:8000.

The cookie is the route's access token, so treat it like the URL: keep it out of logs. To cut off everyone who has the link, run sbx host $id 8000 --localhost --public and then the same with --private; that mints a new token. Keep --localhost on every sbx host call for this port, since each call sets the relay mode again. sbx host hide $id 8000 takes the URL down and leaves Chroma running.

Chroma also has its own server configuration file (Chroma Docker guide, mounted at /config.yaml). We didn't configure Chroma's own auth here: the private URL was the only gate we tested.

A note on the published port#

The commands above publish Chroma as -p 127.0.0.1:8000:8000 and host it with --localhost, so the hosted URL relays to the VM's loopback. The sandbox also has a public IPv6 address. In our check on October 27, 2026, its firewall (ufw, closed except SSH) blocked a port Docker published on all interfaces, but Docker can bypass ufw rules on other setups. Binding to 127.0.0.1 keeps the private URL the only way in, whatever the firewall does. Our test run used -p 8000:8000 and a plain sbx host; the timings above come from that run.

4. What survives a stop, a resume and an upgrade#

sbx stop $id --wait
sbx resume $id
sbx exec $id "docker ps --format '{{.Names}} {{.Status}}'"
chroma Up About a minute

The stop took 2.2 seconds and the resume 1.4 seconds, with memoryRestored: true. The container was never restarted. The hosted URL kept the same token, and query.py returned the same three hits and a 128 ms median.

When a resume boots from disk instead (no machine with the same CPU type is free, or you change the size), running processes are gone. --restart unless-stopped makes Docker start Chroma again, and the data is on the chroma-data volume. See snapshots for when each path happens.

The same volume is how you upgrade. We removed the container and started a new one with the same command:

sbx exec $id --timeout 120 "docker rm -f chroma >/dev/null; docker run -d --name chroma --restart unless-stopped \
  -p 127.0.0.1:8000:8000 -v chroma-data:/data chromadb/chroma >/dev/null; \
  until curl -sf localhost:8000/api/v2/heartbeat >/dev/null; do sleep 0.2; done; \
  ~/venv/bin/python -c 'import chromadb; print(chromadb.HttpClient().get_collection(\"docs\").count())'"
430

The new container answered in 0.2 seconds with every record in place. To move to a new Chroma version, change the image tag in that command. Check Chroma's release notes first, and fork the sandbox before you try it (fork copies the disk), so you can go back.

5. Clean up#

sbx delete $id --yes

A default sandbox costs $0.036/hour while it runs and nothing while it is stopped.

FAQ#

Why does chromadb.HttpClient fail with an "Access token required" ValueError?#

The hosted URL is private and the client sent neither the token nor the cookie. HttpClient can't carry ?_token= in its host, so fetch the URL once with redirects off, take the __sbx_auth cookie from the Set-Cookie header, and pass it with headers={"Cookie": cookie}.

Where do the embeddings get computed?#

In the client, not the Chroma server. With the default embedding function, the Python client runs all-MiniLM-L6-v2 with ONNX wherever the client runs. On a default sandbox, that was 34 paragraphs per second for loading and about 250 to 300 ms per query.

Will my collections survive if the sandbox auto-stops?#

Yes. An auto-stop is a normal stop: disk and memory are saved and sbx resume brings them back. The data also lives on the chroma-data Docker volume, so it survives a resume that boots from disk and a re-created container.

Should I use Chroma or pgvector?#

Use pgvector if your data is already in Postgres or you want SQL joins and transactions over the same rows. Use Chroma if you want a small HTTP service with a Python client that also embeds text for you. Both ran fine on a default sandbox at our sizes; measure at yours.

Can I make the Chroma URL public?#

sbx host $id 8000 --localhost --public removes the token, so anyone with the URL can read, write and delete your collections. We kept it private.

More guides

Top AI sandbox platforms for code execution in 2026, ranked

For running AI-written code we rank Sandboxer, Freestyle and E2B first: each gives code its own VM with an API or SDK and can pause with memory. Pick Daytona for Windows or GPUs, Modal or Northflank for GPUs at scale, E2B or Northflank to run in your own cloud, and Vercel Sandbox if your app lives on Vercel.

Read the guide
Best sandboxes for running coding agents (Claude Code, Codex, Gemini CLI) in 2026, ranked

For coding agents we rank Sandboxer first: a full Ubuntu VM with Claude Code and Codex preinstalled, Docker, and stop-and-resume that keeps the agent's session in memory. Freestyle and Amp Orbs also keep memory across pauses. Pick E2B for prebuilt agent templates on open source, Docker Sandboxes for free local runs, and Codespaces if you live in GitHub.

Read the guide
Best Daytona sandbox alternatives in 2026, ranked

The best Daytona alternatives depend on why you're switching. For a full Linux microVM on every sandbox with memory-saving stop and Docker, we rank Sandboxer first. E2B suits open source and self-hosting, Modal massive short-job fleets, Northflank bring-your-own-cloud with GPUs, and Vercel Sandbox apps on Vercel.

Read the guide
Best E2B alternatives in 2026, ranked

If you're leaving E2B for longer sessions or a fuller machine, we rank Sandboxer first, then Freestyle: both give each sandbox a full Linux VM that keeps memory across pauses, with Docker and no short session cap. Pick Daytona for Windows or GPUs, Northflank for bring-your-own-cloud, and stay on E2B if you need open source.

Read the guide