Local RAG Pipeline
Your data, your answers — without a per-token bill.
RAG (retrieval-augmented generation) lets an LLM answer from YOUR documents instead of its training data. Done locally, your client contracts, SOPs, and research become chat-queryable — with zero cloud, zero per-token bill, zero upload of your IP to a vendor.
€49 once, not $40–$500 a month
Hosted RAG chatbots are metered and tiered: Chatbase runs from $40 a month (Hobby) to $500 a month (Pro), plus $99 a month just to remove their branding from the widget on your client's site — and your documents sit on their servers. This stack runs the same retrieval on your own machine, once, for €49.
The stack (four pieces, one compose file)
- Ollama — serves qwen3:8b for answers + nomic-embed-text for vectors
- Chroma — vector store (docker), holds your embedded chunks
- ingest.py — splits docs, embeds via Ollama, writes to Chroma
- query.py — embeds your question, retrieves top-k, prompts Ollama
What's included
- 📄 The Guide (24-page PDF) — architecture, the stack, 5-step launch, hardware needs, troubleshooting
- ⚙️ docker-compose.yml — Chroma, ready to run
- ⚙️ ingest.py + query.py — the real RAG scripts (no placeholders)
- ⚙️ .env.example — ports, model tags, paths — fill yours
- 📊 XLSX (2 tabs) — file index + hardware matrix
- 🖨️ Cheatsheet (2-page PDF) — 1-page launch ref
Hardware needs (honest)
- 8GB VRAM — works, slow; ingest in small batches
- 16GB VRAM — smooth (recommended)
- 24GB VRAM — fast; big corpora
- No GPU — CPU only, fine for <500 docs
Your data stays on your machine
Documents never leave your machine. Ollama embeds and answers locally. No OpenAI key, no per-token bill. Chroma runs in docker on your box. You control chunking and retrieval. Portable — runs on any GPU machine.
One-time price · no subscription · editable · unwatermarked
€49 once, free to run. The compose file and scripts are plain files you own and edit. Nothing is paywalled, nothing is watermarked.
Who this is for
Freelancers/agencies sitting on client docs, SOPs, or research they want to query privately — without uploading IP to a vendor. You'll run Docker + Ollama (both free).
Who it's NOT for
People who want a hosted RAG SaaS. This is self-hosted by design.
Why trust this?
This is the private-knowledge pattern behind Operator Co: client docs ingested locally, queried by our own Ollama. No contract ever touches a cloud LLM. The compose file and scripts are the real ones, trimmed of our paths.