9. Production-Ready Agent Design

14. Safe Execution Paths & Error Handling

Covers the error-handling taxonomy, retry design, rollback mechanics, escalation paths, and traceability record that let an agent operate safely when a tool call fails -- grounded in GitHub Copilot coding agent's CI-driven retry loop and git-native audit trail.

What is Harness Engineering

The discipline of designing everything that wraps a raw LLM into a reliable agent — tool contracts, system prompt architecture, context/memory management, permission gates, and feedback loops. Distinct from prompt engineering and model training.

1. AI Evaluation Frameworks

The metrics that actually define a good agent -- task success rate, cost per successful task, groundedness, and tool-call correctness -- why generic LLM benchmarks don't transfer to agent evaluation, and the LLM-as-judge pattern's known failure modes.

Building & Evaluating Agents

A book-shaped table of contents for Building & Evaluating Agents: the architectural core of agent design — building single-agent systems, multi-agent systems, evaluation, and the agent framework landscape. Book 3 of the AI Systems Engineering series.

1. Agent Architecture

Covers: LLM, Tools, Memory, Planning, Execution Loop

8. Approval Workflows

Covers how to gate high-risk agent actions behind explicit human approval steps, including timeout handling and audit trail requirements.

5. Autonomous Agents

Covers agents that independently decide their own sequence of actions toward a goal, including the loop-control and stopping-condition problems that make them harder to bound than workflow agents.

6. Event-Driven Agents

Covers agents triggered by external events such as webhooks, message queues, or alerts rather than direct user prompts, and the design implications for statelessness and idempotency.