Chapter 1 — Observability Architecture
Part 08 of the System Design curriculum. The full treatment lives in the Observability book’s Foundations part — this chapter is the curriculum-level summary and the interview framing on top of it.
An observability architecture is built around one property, not a shopping list of tools: can an engineer explain why a system behaved a certain way using only telemetry it already emits — without shipping a code change first and waiting to find out. That distinction — monitoring vs. observability — is one an L6/L7 candidate should be able to state precisely, not just gesture at “we have Grafana.”
The four correlated signals
Four signal types cover almost every telemetry question, and each earns its cost by deliberately discarding a different dimension of information:
| Signal | Answers | Discards |
|---|---|---|
| Metric | ”Is this getting worse, and since when?” | Per-request identity |
| Log | ”What did this one component actually say?” | Aggregate/trend view |
| Trace | ”Which hop in this request was slow?” | Cheap, complete (100%) coverage |
| Profile | ”Which function is burning the CPU right now?” | Request-level narrative |
See The Signals for the full five-signal treatment (metrics, logs, traces, profiles, and events) and what each is built to capture.
Why “three pillars” is the wrong frame for an interview answer
Naming metrics/logs/traces as three separate tools is a monitoring answer, not an architecture answer. The signals have to share a correlation key — a trace ID propagated across every hop, an exemplar linking a metric spike back to one representative trace — or three well-instrumented pillars are still three disconnected monitoring tools bolted together. This is the single most common gap between a senior-level and a principal-level design: a senior candidate lists the three pillars; a principal candidate explains how a metric spike gets a responder to the one trace and log line that explains it, in under a minute.
What this means for a system design interview
When a design calls for “add observability,” the L6/L7 answer names the specific signal for the specific question at hand, states the correlation mechanism tying signals together, and is explicit about what gets sampled or dropped and why. Reliability: SLI, SLO, SLA & Error Budgets covers the number this architecture ultimately feeds; Alerting Systems covers what happens once that number crosses a threshold.
Where to go deeper
Metadata
| Author | Amit Singh |
| Scope | system-design |
Local graph
Linked from 2 notes
Observability Architecture: Questions to Ask
A chronological sequence of 216 questions an architect asks when designing a production-grade observability platform — from business context through multi-tenancy, SLOs, onboarding, and validation.
System Design
Principal/Staff-level system design reference collection for MAANG interview preparation — observability pipelines, distributed systems, reliability engineering, and beyond.
Related notes
Chapter 4 — Alerting Systems
Multi-window burn-rate alerts, recording rules, routing, and deduplication as the difference between an actionable page and noise.
Chapter 3 — Monitoring at Scale
Prometheus, Mimir, Cortex, and Thanos as the horizontally-scaled answer to a single Prometheus instance running out of room.
Chapter 2 — Telemetry Pipelines
OpenTelemetry, OTLP, collectors, sampling, and aggregation as the pipeline that gets a signal from emission to storage without becoming the outage itself.
Chapter 4 — Distributed Tracing Backend
Trace assembly from spans, tail-based vs. head-based sampling.