Self-hosting a RAG vector database (ChromaDB & pgvector)
read · 7 min
Retrieval-augmented generation needs somewhere to store embeddings. Self-hosting that vector store on your VPS keeps your documents private and your costs fixed. Two solid options: ChromaDB (simple) or PostgreSQL with pgvector (production-grade).
Steps
Option A — ChromaDB
ChromaDB is the fastest way to start. Install it with pip and run the server with a persistent data directory.
$ python3 -m venv venv && source venv/bin/activate
$ pip install chromadb
$ chroma run --path ~/chroma-data --port 8000 Option B — PostgreSQL + pgvector
For a production setup, add the pgvector extension to PostgreSQL and store embeddings alongside your relational data.
$ sudo apt-get install -y postgresql postgresql-16-pgvector
$ sudo -u postgres psql -c "CREATE EXTENSION vector;" Create a vector column
In pgvector, store embeddings as a vector column and add an index for fast similarity search.
CREATE TABLE docs (id bigserial PRIMARY KEY, content text, embedding vector(1536));
CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops); Persist and back up the data
Keep the data directory on a volume that survives restarts, and schedule regular backups so your index is never lost.
$ pg_dump -Fc ragdb > ~/backups/rag-$(date +%F).dump Keep it private with Tailscale
A vector store should not be exposed to the public internet. Bind it to your Tailscale interface so only your devices can reach it.
$ sudo ufw allow in on tailscale0 to any port 8000 proto tcp Frequently asked
ChromaDB or pgvector — which should I choose? +
ChromaDB is quickest for prototypes and pure-vector workloads. Choose pgvector when you already use PostgreSQL or want vectors and relational data in one ACID database.
How much disk do embeddings need? +
Roughly the dimension count times 4 bytes per vector, plus index overhead. A 1536-dim embedding is about 6 KB; a million of them is a few GB — easily within a VPS disk.
How do I back it up? +
For pgvector use pg_dump on a schedule; for ChromaDB snapshot the persist directory. Store copies off the server.
Related guides
Host your own RAG stack
A VPS with the disk and memory for a private vector database.
See VPS plans →