The Capstone: A Real Production-Ready Agent Checklist

Twenty-one parts have built agents from a first, single loop through a complete multi-agent onboarding system. This capstone assembles all of it into one checklist — 16 items across 7 categories, each with a concrete way to check it rather than just a box to tick, and what it looks like in production when that item gets skipped.

Design (parts 1-5)

• [ ] Confirmed this task genuinely needs an agent's loop, not a simpler chatbot or RAG call
• [ ] Every tool has one narrow responsibility with a clear, specific description
• [ ] Tool count is scoped — not overloaded with dozens of overlapping options

How to verify: For the first item, describe the task as a fixed sequence of steps known in advance — if that description holds, it doesn't need a loop; a single prompted call handles it more cheaply. For the other two, read each tool's description out loud: if it needs "and" to explain what it does, it's probably two tools, and if two tool names could plausibly handle the same request, the model will sometimes pick wrong.

Skipped, this looks like: an order-status lookup wrapped in a full agent loop — three or four model round trips for one fact a direct function call would return in one. Part 1 and part 2 cover the actual decision criteria; the companion mistakes page has the code for this exact failure.

Memory (parts 7-8)

• [ ] Correctly scoped to working, session, or long-term memory based on actual need
• [ ] If long-term memory is used, it has an explicit staleness consideration

How to verify: for every piece of state the agent holds, ask "does this need to exist after this conversation ends?" If no, it belongs in working memory only, not a database row. For anything that is persisted, find the code path that updates or expires it — a stored fact this agent will still serve as current a year from now, with no way to correct it, fails this item.

Skipped, this looks like: every chat message logged to a persistent table nobody ever reads back, while a customer's actual stated preference from months ago quietly goes stale and keeps getting served. Part 7 and part 8 cover choosing correctly.

External tools (parts 9-10)

• [ ] Every external API call has a timeout and retry logic
• [ ] Every tool response is translated into a clean, minimal format before returning to the model
• [ ] Every tool's safety boundary is enforced in code, not just a prompt instruction

How to verify: search the codebase for requests.get( or requests.post( calls with no timeout= argument — each one is a hang waiting to happen. For the second item, log the raw tool response next to what actually gets sent to the model; if they're identical for a large API payload, context is being spent on fields the model never uses. For the third, take every safety claim in a system prompt ("only read-only," "never delete") and find the line of code that would actually stop the unsafe action if the model tried it anyway — no such line means it isn't enforced.

Skipped, this looks like: a hung third-party API call stalling the entire agent loop with no way out, or a prompt that says "read-only" while the database connection underneath it has full write access. Part 9 covers the timeout/retry pattern (the same one detailed in the Python requests library guide); part 10 and the mistakes page's Mistake 5 cover the safety-boundary failure directly.

Execution safety (part 18)

• [ ] An explicit max_iterations limit is set
• [ ] Errors are surfaced to the model as visible text it can reason about, not silent failures
• [ ] Unresolved failures escalate to a human, rather than producing a confident-sounding fabricated answer

How to verify: search the agent loop for while True — if it exists without a paired counter and an explicit exit condition, this item fails outright. For error handling, check every except block in tool-calling code: does it return the actual error text to the model, or does it swallow the exception and return a placeholder result? For escalation, define what happens after N consecutive tool failures — "it just tries again forever" or "it answers anyway" both fail this item.

Skipped, this looks like: an unbounded retry loop against a permanently failing tool generating unlimited API cost, or an agent that reports a task complete when the underlying action silently errored. Part 18 covers the full pattern.

Observability (part 19)

• [ ] Every agent run produces a structured, reviewable trace
• [ ] Aggregate metrics — iteration counts, tool error rates — are tracked across production runs

How to verify: pull up the trace for the most recent production run right now. Can you tell which tool was called, with what arguments, and what it returned, without reading raw application logs line by line? For the second item, is there a dashboard or a saved query that answers "what's our tool error rate this week" — or does answering that require someone manually grepping logs?

Skipped, this looks like: an agent that took an unexpected path on one specific run, with no way to reconstruct why, because nothing was logged in a structured, queryable form. Part 19 covers building this properly.

Security (part 20)

• [ ] Any agent processing untrusted external content has no access to consequential tools
• [ ] Consequential actions require explicit human confirmation before executing

How to verify: for every agent with a tool that reads external content — a webpage, an email, a file someone else uploaded — list its other tools. If any of them sends, writes, or deletes anything, that's a structural risk regardless of how the prompt is worded. For the second item, trace one consequential action from the tool call to execution: is there a real point where a human has to approve it, or does the code path run straight through?

Skipped, this looks like: a research agent that can both read arbitrary web pages and send emails, giving a prompt-injected instruction hidden in a fetched page a direct path to real, unauthorized action. Part 20 and the mistakes page's Mistake 13 cover this threat model directly.

Evaluation (referencing the LLM & Advanced AI series)

• [ ] A concrete evaluation set exists, not just anecdotal hand-picked examples

How to verify: name the file or dataset containing real test cases with expected outcomes that this agent is scored against before any change ships. "We tried it a few times and it seemed fine" does not satisfy this item.

How to score this

16 items, 7 categories. Security and Execution safety are non-negotiable — any unchecked item in either category blocks shipping, regardless of the total score. Outside those two categories, 12 or more of the remaining 11 items checked is a reasonable ship threshold; below that, the gaps are usually concentrated enough (all in Memory, say, or all in Observability) that it's faster to fix them than to argue about the exact number.

Mapping this checklist onto GreenDesk's onboarding system

Design: three narrowly-scoped sub-agents (part 15) instead of one
  overloaded agent
Memory: the orchestrator uses working memory only — no persistent
  state needed for a single onboarding task
External tools: create_workspace and schedule_kickoff_call both wrap
  external APIs with part 9's timeout and retry discipline
Execution safety: account provisioning and scheduling are genuinely
  consequential — both gate on human confirmation before executing,
  per part 18's pattern
Observability: every onboarding run produces a full structured trace
  across all three sub-agents (part 19)
Security: the Welcome Content sub-agent (which processes external
  brand-voice reference material) has zero access to
  account-provisioning or scheduling tools (part 20)

The complete arc across this site's four AI series

AI Fundamentals: what AI, ML, and LLMs are
AI Projects with Python: ten working applications
LLM & Advanced AI: embeddings, RAG, fine-tuning — the deeper
  techniques for production-scale knowledge and behavior
Building AI Agents (this series): autonomy, tools, memory, and
  multi-agent coordination — the most advanced capability this
  site's AI curriculum covers

This capstone closes the loop on all four — every technique from every prior series (grounding, RAG, evaluation, cost accounting) reappears here, applied to the more complex problem of autonomous, tool-using systems.

FAQ

Is 16 items too few for a "production-ready" checklist? It's deliberately scoped to what this series actually covers in depth — deployment infrastructure, load testing, and cost-monitoring dashboards are real production concerns too, but they're general software-engineering practices rather than agent-specific ones, so they're out of scope here.

What if an item is checked but I'm not confident it would hold up under real load? Don't check it. Every item above has a concrete verification step precisely so "probably fine" doesn't get treated the same as "confirmed."

Does every agent need all three sub-agents like GreenDesk's example? No — GreenDesk's three-sub-agent design is what the task actually needed (per part 15), not a template to copy. A single-agent system that passes all 16 items is more production-ready than a multi-agent one that doesn't.

How does this relate to the common-mistakes page? That page is a post-mortem — 13 failure patterns with the code that causes and fixes each one. This checklist is the pre-launch gate. Several items here link directly to the matching mistake for the code-level detail.

That's the complete Building AI Agents series — from a precise definition of what an agent is, through a working, guardrailed multi-agent production system. Combined with AI Fundamentals, AI Projects with Python, and LLM & Advanced AI, this closes out the arc from a single token prediction to coordinating multiple autonomous, production-grade AI systems together.