Build an AI Question-Answering App
Part 2's chatbot answered from a small, hardcoded block of plan data. This part builds something genuinely more scalable — a real question-answering app that searches across Bright Leaf Coffee's actual FAQ and policy documents first, then answers strictly from what it finds, the practical, hands-on version of the AI Fundamentals series' own RAG explanation.
The real, actual knowledge base
KNOWLEDGE_BASE = [
{
"id": "shipping-1",
"text": "Orders ship within 24 hours of roasting. Standard shipping within the US takes 3-5 business days.",
},
{
"id": "refund-1",
"text": "If your coffee arrives stale or damaged, contact us within 14 days for a full refund or free replacement.",
},
{
"id": "subscription-1",
"text": "You can change your roast selection or pause your subscription anytime before the next billing cycle.",
},
]This is a genuine, real simplification of what a production knowledge base would be (typically a real database or vector store, not a Python list) — but the actual retrieval and answering logic below works identically regardless of where the real documents are stored.
Real, practical keyword-based retrieval
def find_relevant_docs(question: str, knowledge_base: list[dict], top_n: int = 2) -> list[dict]:
question_words = set(question.lower().split())
scored = []
for doc in knowledge_base:
doc_words = set(doc["text"].lower().split())
overlap = len(question_words & doc_words)
scored.append((overlap, doc))
scored.sort(key=lambda x: x[0], reverse=True)
return [doc for score, doc in scored[:top_n] if score > 0]This is a real, deliberately simple retrieval technique — genuine keyword overlap, not the embedding-based semantic search the AI Fundamentals series described conceptually. It's a real, honest starting point: functional and easy to understand, with a clearly-named upgrade path (embeddings) for when keyword matching genuinely isn't precise enough.
The real, complete Q&A function
def answer_question(question: str) -> str:
relevant_docs = find_relevant_docs(question, KNOWLEDGE_BASE)
if not relevant_docs:
return "I don't have information about that — please contact support@brightleafcoffee.com directly."
context = "\n\n".join(doc["text"] for doc in relevant_docs)
client = get_client()
response = client.messages.create(
model="claude-sonnet-5",
max_tokens=512,
system=(
f"Answer the customer's question using ONLY this real, "
f"retrieved information:\n\n{context}\n\nIf this "
f"information doesn't fully answer the question, say so "
f"directly rather than guessing."
),
messages=[{"role": "user", "content": question}],
)
return response.content[0].textNotice the real, explicit early return when find_relevant_docs finds nothing — this is a genuine, deliberate hallucination guard, directly implementing the AI Fundamentals series' own grounding technique: rather than sending an empty or irrelevant context and hoping the model admits uncertainty on its own, the application itself detects the no-match case and responds without ever calling the model at all.
This retrieve-then-answer pattern is genuinely why this project scales past part 2's chatbot — adding a hundredth real FAQ entry to KNOWLEDGE_BASE costs nothing extra per request, since only the specific, relevant entries actually get included in each real prompt. Part 2's approach of stuffing everything into the system prompt would instead grow every single request's token cost linearly with the knowledge base's real size.
A real, concrete test
print(answer_question("What if my coffee arrives broken?"))
# → correctly retrieves refund-1, answers grounded in that real policy
print(answer_question("Do you ship internationally?"))
# → no real doc matches "internationally" — correctly returns the
# "I don't have information" fallback, rather than guessingThe second real case is exactly the honest behavior part 13 of the AI Fundamentals series was built around — genuinely missing information produces a genuine, honest "I don't know," not a plausible-sounding fabrication.
The real, honest limit of keyword retrieval
Works well: "refund" question matching a doc containing "refund"
Fails: "my order arrived damaged" not matching a doc phrased as
"stale or damaged," if the exact word overlap is too sparseThis is a genuine, real limitation worth knowing rather than glossing over — keyword matching misses genuinely relevant documents phrased differently than the question, exactly the semantic gap embeddings solve, covered as the real, practical next step for this project once keyword matching's limits are actually hit in production.
Next: adding source citations to this Q&A app — showing customers exactly which real document an answer came from, for genuine transparency and trust.