Notes / tag / observability-and-evaluation

#observability-and-evaluation

8 notes

# Production Agent Systems

All Production Agent Systems notes →

1. AI Observability Fundamentals

Building the metrics, logs, traces, dashboards, and alerting an agent needs so its own operators can tell when it is misbehaving, not just when it is down.

production-agent-systems observability-and-evaluation book

2. Agent Tracing

What AI observability adds on top of standard OTel instrumentation — spans around LLM calls and tool calls, tracing full agent execution, and capturing token usage as a first-class attribute.

production-agent-systems observability-and-evaluation book

3. Token Metrics

Treating input, output, and cached token counts as first-class SLIs — per-request, per-tenant, and per-model dashboards, the token-to-cost conversion, and alerting on token-count anomalies as an early signal of prompt drift or a runaway loop.

production-agent-systems observability-and-evaluation book

4. Prompt Observability

Capturing and versioning the exact prompt (system, few-shot, and injected context) sent on every call so a regression can be traced to a specific prompt-template change, with redaction rules for what's safe to log versus what must be hashed or dropped.

production-agent-systems observability-and-evaluation book

5. Memory Observability

Instrumenting what an agent actually retrieved from long-term memory on each turn — retrieval hit rate, relevance and similarity score distributions, and staleness of cached embeddings — so memory-driven hallucinations can be traced to a specific bad retrieval instead of guessed at.

production-agent-systems observability-and-evaluation book

6. Tool Invocation Metrics

Per-tool latency, error rate, and call-volume dashboards plus argument-validation failure tracking, so a misbehaving tool integration shows up as a metrics anomaly before it shows up as a user-facing failure.

production-agent-systems observability-and-evaluation book

7. AI Logging

Structured logging conventions for an agent's reasoning trace (thought, action, observation) that balance debuggability against the cost and privacy risk of logging full prompts and completions at high volume.

production-agent-systems observability-and-evaluation book

12. AI SLOs

SLOs, error budgets, incident response, and cost optimization applied to an agent workload — including token cost as a first-class SLI and degrade-to-human-handoff as an error-budget policy.

production-agent-systems observability-and-evaluation book