Adding Source Citations to Your QA App
Part 12's Q&A app answers correctly, grounded in real, retrieved documents — but a customer reading the answer has no way to verify it against the actual source. This part adds real citations, directly tied to each retrieved document's ID.
Why citations are a genuine trust mechanism, not decoration
Without citations: "Orders ship within 24 hours of roasting." — a
real, correct answer, but the customer has to simply trust it
With citations: "Orders ship within 24 hours of roasting. [Source:
shipping-1]" — the exact same real answer, now independently
verifiable against the actual, real source documentThis directly extends the AI Fundamentals series' own honest framing of hallucination risk — grounding reduces the risk of a wrong answer, but citations let a real reader independently verify a specific answer, a genuinely different, complementary safeguard.
The real, updated Q&A function
def answer_question_with_citations(question: str) -> dict:
relevant_docs = find_relevant_docs(question, KNOWLEDGE_BASE)
if not relevant_docs:
return {
"answer": "I don't have information about that — please contact support@brightleafcoffee.com directly.",
"sources": [],
}
context = "\n\n".join(f"[{doc['id']}] {doc['text']}" for doc in relevant_docs)
client = get_client()
response = client.messages.create(
model="claude-sonnet-5",
max_tokens=512,
system=(
f"Answer using ONLY this real information:\n\n{context}\n\n"
f"After your answer, on a new line, list the real source "
f"IDs you actually used, in this exact format: "
f"Sources: [id1, id2]"
),
messages=[{"role": "user", "content": question}],
)
full_text = response.content[0].text
answer, sources = parse_answer_and_sources(full_text)
return {"answer": answer, "sources": sources}Real, reliable parsing of the model's cited sources
import re
def parse_answer_and_sources(text: str) -> tuple[str, list[str]]:
match = re.search(r"Sources:\s*\[(.*?)\]", text)
if not match:
return text.strip(), []
sources = [s.strip() for s in match.group(1).split(",")]
answer = text[:match.start()].strip()
return answer, sourcesThis real parsing step directly relies on the exact output format the system prompt requested — re.search looks specifically for the literal Sources: [...] pattern, which is exactly why part 12's technique of explicitly specifying output format (from the AI Fundamentals series) matters here concretely: reliable parsing genuinely depends on the model consistently following that exact, requested structure.
Notice the model itself decides which of the retrieved documents it actually used to answer — find_relevant_docs might retrieve two real candidate documents, but the model may have only genuinely needed one of them, and the citation output reflects that real, actual usage rather than just echoing back everything that was retrieved.
A real, honest limitation: the model could still cite incorrectly
This technique reduces the risk of an UNGROUNDED answer, but it
doesn't structurally GUARANTEE the model's self-reported citation
is genuinely accurate — it's still the model's own real, generated
claim about what it used, not independently verifiedA genuinely more rigorous real implementation would programmatically check that claimed facts in the answer actually appear in the cited source document's text — a real, additional verification step beyond this part's scope, but worth knowing as the honest next level of rigor for a use case where citation accuracy is genuinely critical.
Displaying real citations to an actual user
result = answer_question_with_citations("What if my coffee arrives broken?")
print(result["answer"])
for source_id in result["sources"]:
source_doc = next(d for d in KNOWLEDGE_BASE if d["id"] == source_id)
print(f"— {source_doc['text']}")In a real, web-facing version of this app (built the same way as part 5's Django chatbot), this becomes a real, visible "sources" section under the answer — genuinely useful for a support agent reviewing the AI's answer before relaying it to a real customer, not just for the customer directly.
Next: building a real AI content generator with Python — a genuinely creative task, distinct from this series' factual, grounded projects so far.