# Building Agentic Systems
All Building Agentic Systems notes →1. Why Multi-Agent Systems
Covers the concrete failure modes of single-agent systems, such as context overload, tool sprawl, and conflicting objectives, that motivate splitting work across multiple specialized agents.
2. Collaboration Models
Splitting one investigation agent into metrics, logs, and traces specialists — tool isolation and prompt specialization as the design levers that make each one reliable.
3. Communication Protocols
Agent-to-agent protocols, shared memory, message passing, coordination patterns, and how a multi-agent system recovers when one agent in the chain fails.
4. Task Decomposition
Covers strategies for breaking a complex goal into subtasks that can be assigned to different agents, and how decomposition granularity affects coordination overhead.
5. Agent Negotiation
Covers how agents with different objectives or partial information reach agreement on a shared action, including bidding and argumentation-based negotiation protocols.
6. Consensus Mechanisms
Covers how multi-agent systems reach agreement on a single output or decision when individual agents disagree, drawing on voting, quorum, and distributed-consensus analogies.
7. Swarm Intelligence
Covers decentralized multi-agent patterns where global behavior emerges from simple local rules rather than centralized planning, and where that tradeoff pays off for agentic systems.
8. Distributed Coordination
Covers coordinating agent state and actions across distributed processes, including the partial failure, message loss, and race condition modes borrowed from distributed systems theory.
9. Supervisor Architectures
A supervisor agent that delegates to the specialist agents, aggregates their results, resolves conflicting conclusions, and generates the final incident report.
10. Agent Meshes
Covers service-mesh-inspired architectures for agent-to-agent discovery, routing, and observability at the scale of dozens of interacting agents.
11. Agent Lifecycle Management
Covers how agents are added to, updated or reconfigured within, and retired from an already-running multi-agent workflow -- versioning agent definitions, draining vs. hard-cutting over in-flight runs, and preserving auditability without breaking workflow continuity, using GitHub Copilot's custom agent files as the reference implementation.
1. Agent Architecture
Covers: LLM, Tools, Memory, Planning, Execution Loop
2. Planner–Executor Pattern
How to wire a planner role and an executor role as two distinct LLM-call shapes inside a single agent process, and when that in-process split stops being the right call.
3. Router Pattern
Shows what the router pattern looks like wired into a single agent process -- one classification call that picks a specialized system prompt and tool set, with no sub-agent spawned and no service boundary crossed.
4. Workflow Agents
Covers agents that follow a predefined, deterministic sequence of steps rather than freely choosing their own next action, and when that constraint is the right engineering tradeoff.
5. Autonomous Agents
Covers agents that independently decide their own sequence of actions toward a goal, including the loop-control and stopping-condition problems that make them harder to bound than workflow agents.
6. Event-Driven Agents
Covers agents triggered by external events such as webhooks, message queues, or alerts rather than direct user prompts, and the design implications for statelessness and idempotency.
7. Human-in-the-Loop Systems
Covers patterns for pausing an agent to request human input or confirmation mid-task, and how to design the handoff so the agent resumes with full context.
8. Approval Workflows
Covers how to gate high-risk agent actions behind explicit human approval steps, including timeout handling and audit trail requirements.
9. Production-Ready Agent Design
Covers the checklist that separates a demo agent from a production one: retries, timeouts, cost controls, observability hooks, and graceful degradation under failure.
1. AI Evaluation Frameworks
The metrics that actually define a good agent -- task success rate, cost per successful task, groundedness, and tool-call correctness -- why generic LLM benchmarks don't transfer to agent evaluation, and the LLM-as-judge pattern's known failure modes.
2. Benchmarks
Building a standing benchmark suite that runs against every model or prompt change, isolating a provider-side regression from one you introduced with a pinned-model control, and the composition, staleness, and cadence tradeoffs that keep the suite discriminating over time.
3. Online Evaluation
Continuously scoring live production traffic — LLM-as-judge scoring applied to real conversations, implicit feedback signals as cheaper proxies, shadow-mode comparison, and the sampling strategy that makes any of this affordable at production scale.
4. Offline Evaluation
Running a held-out golden dataset through a candidate agent version before deploy as a CI regression gate — golden dataset construction and versioning, hard-threshold versus regression-from-baseline pass criteria, and the coverage limit that makes online evaluation a necessary complement, not a redundant check.
1. Evaluation Criteria
Covers the axes, including orchestration model, state management, observability, and ecosystem maturity, used to evaluate and compare agent frameworks before adopting one.
2. OpenAI Agents SDK
Covers OpenAI's Agents SDK primitives, agents, handoffs, guardrails, and sessions, and where it fits versus building an orchestration layer from scratch.
3. LangGraph
Covers LangGraph's graph-based state machine model for agent orchestration — nodes, edges, and conditional routing — and why that model, not a simple DAG, is what makes cycles, checkpointing, and human-in-the-loop interrupts first-class instead of bolted on.
4. CrewAI
CrewAI's role-based multi-agent orchestration model — crews, tasks, and processes — and where its opinionated defaults help you ship fast versus where they become a ceiling on custom control flow.
5. AutoGen
Covers Microsoft AutoGen's conversational multi-agent model — agents coordinate through group-chat message exchange and a speaker-selection policy instead of an explicit graph — and where that buys flexibility versus where it costs control.
6. Semantic Kernel
Covers Microsoft Semantic Kernel's plugin-and-planner model for embedding agentic behavior into existing enterprise .NET and Python applications, rather than building a new agent-first service from scratch.
7. Google ADK
Covers Google's Agent Development Kit — its workflow/dynamic-routing composition model, native tool-integration story, and the deployment path onto Vertex AI Agent Engine that is the actual site of vendor coupling, not the framework code itself.
8. LlamaIndex Workflows
Covers LlamaIndex's event-driven Workflows abstraction — steps wired by typed events instead of an explicit graph — and why the framework's RAG-first origins make it the natural home for retrieval-heavy agents.
9. Haystack Agents
Covers Haystack's pipeline-based approach to building agents on top of its retrieval and NLP component graph, aimed at production search and RAG use cases.
10. Choosing the Right Framework
Covers a decision framework for picking among the agent frameworks surveyed in this part, based on team skillset, orchestration complexity, and production observability needs.
Building & Evaluating Agents
A book-shaped table of contents for Building & Evaluating Agents: the architectural core of agent design — building single-agent systems, multi-agent systems, evaluation, and the agent framework landscape. Book 3 of the AI Systems Engineering series.