Harness engineering is the discipline of designing the system that wraps a raw LLM so it behaves as a dependable, steerable agent — the tool-calling loop, system prompt architecture, context and memory management, permission gates, sandboxing, sub-agent orchestration, and the feedback loops that catch drift. The model supplies reasoning; the harness turns that reasoning into software you can actually run in production.
It’s a distinct layer from two things it’s often confused with:
- Prompt engineering — optimizing the text of a single instruction or few-shot example.
- Model training / fine-tuning — optimizing the weights themselves.
The components of a harness
| Component | What it does | Failure mode if missing |
|---|---|---|
| Tool contracts | Typed schemas for every action the agent can take (file edit, shell exec, API call) | Agent invents arguments, calls undefined tools, silent no-ops |
| System prompt architecture | Durable behavior rules layered over per-turn instructions | Instructions drift or get overridden turn to turn |
| Context/memory management | Compaction, retrieval, summarization once the window fills | Silent context loss, contradictory actions after compaction |
| Permission model | Approve/deny gates scoped to blast radius (read vs. destructive) | Irreversible actions (force-push, rm -rf, prod writes) run unsupervised |
| Sandboxing / execution environment | Isolates what a tool call can actually reach | A shell tool becomes a full RCE surface |
| Orchestration | Sub-agent spawning, parallel fan-out, workflow sequencing | Monolithic context window, no separation of concerns |
| Feedback loops | Hooks, telemetry, evals, guardrails that catch bad behavior | Regressions ship silently; no signal until a user complains |
Concrete example: this session is a harness
The Claude Code session you’re reading this in is a harness around an LLM: typed tool schemas
(Bash, Edit, Read, Agent, Workflow), a layered system prompt (global CLAUDE.md → project CLAUDE.md →
session context), a permission-mode gate on risky tool calls, hooks for pre/post tool-call behavior,
a memory system for cross-session state, and sub-agent orchestration via the Agent and Workflow
tools. None of that is the model — it’s the engineering that makes the model trustworthy enough to
run git commit on your behalf.
Why it’s its own discipline
Harness engineering brings systems-design rigor to a layer that’s often treated as an afterthought bolted onto a prompt:
- Reliability framing — the harness needs the same posture as any distributed system: failure modes enumerated, blast radius bounded, degraded modes defined. A tool call without a permission gate is the agentic equivalent of an ungated destructive API endpoint.
- Observability framing — every tool call, every context compaction, every sub-agent spawn is an event worth being able to trace and audit after the fact, the same way a request is traced through a distributed system.
- Guardrails as circuit breakers — an evaluator that halts a bad agent loop plays the same role a circuit breaker plays in a service mesh: fail fast, contain the blast radius, don’t let one bad decision cascade.
This is why What is Google ADK, What is CrewAI, What is Hermes Agent, and What is OpenClaw are all, underneath their differing APIs, competing answers to the same harness design questions: how is memory scoped, how are tools sandboxed, how does orchestration compose. Comparing them productively means comparing harness decisions, not comparing prompts.
Where this shows up in practice
- The Build an AI SRE Assistant hands-on build (Part 00) is a harness engineering exercise in miniature: the tool layer wiring to Grafana/Loki/Tempo, the approval checkpoints before remediation, and the RCA-confidence scoring are harness decisions, not prompt decisions — see also Production-Ready Agent Design (Part 00 of Building & Evaluating Agents) for the checklist this generalizes into.
- MCP servers like What is MCP Toolbox and What is Grafana MCP are harness inputs — they define the tool contract surface an agent can be granted, with their own permission scoping (OAuth2, RBAC) layered underneath.
The practical takeaway: when an agent misbehaves, the fix is rarely “reword the prompt.” It’s almost always a harness gap — a missing tool contract, an ungated destructive action, or a context window that silently dropped the constraint that mattered.
Local graph
Linked from 2 notes
11. Claude Code: Architecture Case Study
A documentation-grounded analysis of Claude Code's architecture — the gather/act/verify agentic loop against a real filesystem and shell, the allow/deny/ask tool-permission model, and subagent delegation with isolated context — distinguishing Anthropic's own documented mechanics from reasonable architectural inference.
Agentic AI: Projects & Engineering Mastery
A book-shaped table of contents for Agentic AI: Projects & Engineering Mastery: hands-on practitioner builds, Principal/Staff-level technical leadership, and the lookup appendices and vendor/framework reference notes for the whole series. Book 6 of the AI Systems Engineering series.
Related notes
What is Hermes Agent
Nous Research's open-source, self-hosted AI agent that runs continuously on your own server, builds persistent memory over time, and distills its own reusable 'skills' from completed tasks.
What is Mem0
Universal memory layer for AI agents — combines vector search, a knowledge graph, and key-value caching behind one API, so any framework can bolt on persistent, cross-session memory in under a day.
What is Azure SRE Agent (Microsoft)
Microsoft's AI agent embedded in Azure for autonomous incident response — acknowledges alerts from PagerDuty/ServiceNow/Azure Monitor, investigates via Azure Monitor/App Insights/Kusto, executes configurable Incident Response Plans with tunable autonomy, and learns across incidents via Session Insights.
What is Azure AI Services
Microsoft's Azure AI service catalog — account models (single- vs multi-service), Azure OpenAI's deployment-based access pattern, Azure AI Search as the RAG grounding layer, and the single-service capability catalog (Vision, Language, Speech, Document Intelligence).