Real Costs of Embeddings, Vector Databases, and Fine-Tuning
This series has referenced real cost trade-offs throughout without fully itemizing them. This part covers the complete, real cost picture across embeddings, vector databases, and fine-tuning — genuinely necessary for making an honest, informed decision, not a vague sense that "AI is expensive."
Real embedding generation costs
Real, typical pricing structure: charged per token of TEXT EMBEDDED,
usually meaningfully cheaper per token than text generation (the
AI Fundamentals series' own token-cost coverage)Bright Leaf Coffee's FAQ (part 6): ~50 documents, a few hundred
tokens each → a genuinely trivial, one-time real embedding cost
GreenDesk's knowledge base (part 15): hundreds of documents,
re-embedded on every real content change (part 16) → a real,
small but genuinely recurring cost, worth tracking over timeEmbedding cost is real but genuinely modest compared to generation cost — the practical, real budget concern in a RAG system is usually the ongoing generation cost per query (per the AI Fundamentals series' own coverage), not the one-time or periodic embedding step.
Real vector database hosting costs
Chroma (part 4): genuinely free to run locally or self-hosted —
the real cost is your own infrastructure (a server, or none at
all for local development)
Pinecone (part 4): a real, ongoing, usage-based cost — genuinely
scales with the number of real vectors stored and real queries run
pgvector (part 4): genuinely no additional real cost beyond an
existing PostgreSQL instance already being paid forThis directly connects back to part 4's decision framework with real, concrete numbers attached: Bright Leaf Coffee's small FAQ genuinely never justifies Pinecone's real, ongoing cost; GreenDesk's larger, growing knowledge base may genuinely reach a scale where Pinecone's managed reliability is worth its real, measured cost — an honest, calculated trade-off, not a default choice either way.
Real fine-tuning costs
Real, typical cost structure: a real, one-time training cost based
on dataset size and training duration, PLUS often a genuinely
higher real per-token cost for using the fine-tuned model
afterward compared to the base modelGreenDesk's real email-formatting fine-tune (part 9, part 10): a
real, one-time training cost for the initial run, PLUS a real,
ongoing per-token premium every time the fine-tuned model actually
generates an email afterwardThis is directly why part 10's "exhaust prompting first" guidance matters financially, not just as engineering best practice — a real, well-constructed prompt has zero training cost and no ongoing per-token premium, while fine-tuning carries both real costs, justified only when the measured compliance gap (part 10's evaluation) genuinely demands it.
A real, complete cost comparison table
Technique Upfront cost Ongoing cost
Better prompting None Standard generation cost
RAG (Chroma) Minimal (embedding) Standard generation + minimal embedding
RAG (Pinecone) Minimal (embedding) Generation + real, ongoing hosting fee
Fine-tuning Real training run Often a real, per-token premiumThis table directly reflects the real, honest ordering this series has implicitly built toward — from cheapest and simplest (prompting) to most expensive and complex (fine-tuning), with RAG occupying real, genuine middle ground.
A real, practical budgeting exercise
def estimate_monthly_rag_cost(queries_per_day: int, avg_tokens_per_query: int, cost_per_1k_tokens: float) -> float:
monthly_queries = queries_per_day * 30
monthly_tokens = monthly_queries * avg_tokens_per_query
return (monthly_tokens / 1000) * cost_per_1k_tokens
# Bright Leaf Coffee: ~50 real support questions/day, ~500 tokens each
print(estimate_monthly_rag_cost(50, 500, cost_per_1k_tokens=0.003))Running a real, concrete calculation like this — using actual, current per-token pricing from a real provider's documentation (per the AI Fundamentals series' own guidance on reading real API documentation) — turns an abstract cost concern into a real, specific number worth actually budgeting for before deploying any of this series' systems to real production.
The real, honest bottom line
For MOST real, practical use cases at Bright Leaf Coffee and
GreenDesk's actual scale: better prompting and RAG with Chroma
cover the real, genuine need at genuinely modest cost — fine-
tuning and managed vector databases become worthwhile only at a
real, specific, measured scale or compliance requirement neither
business has hit yetNext, and last: the capstone — bringing every technique in this series together into one real, complete decision framework.