Build a Knowledge-Base Chatbot
Bright Leaf Coffee's real FAQ (part 6) had a few dozen documents. GreenDesk's real, internal employee knowledge base is genuinely different — hundreds of documents, multiple document types, and real access-control needs. This part builds the chatbot for that actual scale.
The real, genuine difference from part 6
Bright Leaf Coffee's FAQ: a few dozen real documents, all publicly
answerable, no real access restrictions
GreenDesk's internal knowledge base: hundreds of real documents —
onboarding guides, engineering runbooks, HR policies — where some
content is genuinely restricted to specific real employee rolesReal, metadata-based filtering
collection.add(
documents=["To reset a customer's password, use the admin panel..."],
ids=["runbook-42"],
metadatas=[{"category": "engineering", "min_role": "engineer"}],
)
collection.add(
documents=["Vacation requests must be submitted 2 weeks in advance..."],
ids=["hr-policy-7"],
metadatas=[{"category": "hr", "min_role": "employee"}],
)Chroma's real metadatas parameter attaches structured, real data to each document alongside its embedding — genuinely necessary at this scale, where retrieval needs to respect real, actual access boundaries, not just semantic similarity.
Real, role-aware retrieval
def answer_question_for_employee(question: str, employee_role: str) -> dict:
allowed_categories = get_allowed_categories(employee_role)
results = collection.query(
query_texts=[question],
n_results=5,
where={"category": {"$in": allowed_categories}},
)
retrieved_docs = results["documents"][0]
if not retrieved_docs:
return {"answer": "I don't have accessible information about that.", "sources": []}
# ...same real grounded-generation pattern as part 6The where clause here is a real, genuine access-control mechanism — an employee without engineering-level real access structurally never receives engineering-only content in their retrieved context, regardless of how semantically relevant a specific runbook might be to their real question.
This is a genuinely important, real security principle worth being precise about: access control belongs in the RETRIEVAL step, filtering what's even eligible to be retrieved — not as an instruction telling the model to "not share restricted information" in the prompt. A real instruction can, in principle, be circumvented; content that was never retrieved in the first place structurally cannot leak.
Real, practical document ingestion at this larger scale
import os
def ingest_directory(directory: str) -> None:
for filename in os.listdir(directory):
filepath = os.path.join(directory, filename)
category = infer_category_from_path(filepath) # e.g. from folder name
text = extract_text_smart(filepath) # part 12's real, combined extractor
chunks = chunk_document(text) # part 7's real chunking
collection.add(
documents=chunks,
ids=[f"{filename}-{i}" for i in range(len(chunks))],
metadatas=[{"category": category} for _ in chunks],
)This real, batch ingestion function is what makes indexing GreenDesk's actual, hundreds-of-documents knowledge base practical — a genuine, direct application of parts 7 and 12's chunking and extraction work at real, meaningful scale, rather than the manual, one-document-at-a-time approach from part 6's small example.
The real, concrete signal to migrate off Chroma
Per part 4's decision framework: GreenDesk's knowledge base growing
into the thousands of real documents, with genuine production
reliability requirements (real uptime SLAs for employee-facing
tooling), is exactly the real, concrete trigger for migrating to
Pinecone or pgvectorThis directly closes the loop on part 4's earlier, honest framing — this is the actual, real moment that decision framework was built for, not a hypothetical.
A real, practical addition: usage analytics
def log_query(question: str, employee_role: str, retrieved_count: int) -> None:
# real, structured logging — which real questions get asked most,
# which return zero results (a genuine content gap signal)
logger.info("kb_query", extra={
"question": question,
"role": employee_role,
"results_found": retrieved_count,
})Real, structured logging of every query — especially ones returning zero real results — is a genuinely practical way to discover actual, real gaps in GreenDesk's knowledge base: a pattern of employees asking about something with no matching document is a direct, real signal for what to write next.
Next: keeping a knowledge-base chatbot up to date — a real, practical re-indexing pipeline for when GreenDesk's actual documents change.