Tempo is Grafana Labs’ distributed tracing backend, built around one deliberate simplification: unlike What is Jaeger‘s classic architecture (which requires Cassandra or Elasticsearch to index span tags for search), Tempo maintains no secondary index at all. The only lookup it needs is trace ID → storage block, which means the only infrastructure dependency is object storage — no index cluster to size, operate, or pay for.
Ingest: multi-protocol by design
OTLP ──┐
Jaeger ─┼──▶ Tempo distributor ──▶ Ingester ──▶ Object storage (blocks)
Zipkin ─┘ │
▼
Compactor
Tempo accepts OTLP, Jaeger, and Zipkin wire protocols natively at the same distributor — a service or vendor agent that “sends to Jaeger” doesn’t need a separate Jaeger deployment; pointing its Jaeger-protocol exporter at Tempo’s Jaeger receiver is a drop-in replacement.
Finding a trace without an index
| Access path | How it works |
|---|---|
| Trace ID lookup | Direct block lookup — the one operation Tempo was built to make cheap |
| Exemplars from Mimir | A Prometheus/Mimir histogram bucket carries a sampled trace ID linking straight to the request that produced that latency value |
| Log-to-trace correlation | A structured log line in What is Loki carrying trace_id links directly into Tempo |
| TraceQL search | Structured query over span attributes/duration/status, added later to close the gap with tag-indexed search |
{ span.http.status_code = 500 && duration > 800ms }
TraceQL search still has to scan recent blocks rather than hit a pre-built tag index, which is the real tradeoff behind Tempo’s cost model: cheap storage and ingestion, in exchange for search being scoped to recent time windows rather than instant across all history the way an ES-backed Jaeger index would be.
Exemplars: the concrete metrics-to-trace bridge
Mimir histogram: http_request_duration_seconds_bucket{le="1.0"}
│
│ exemplar: trace_id="a1b2c3...", value=0.94
▼
Tempo trace a1b2c3... (the actual request that produced that sample)
This is the mechanism What is Cardinality (in observability) points to as the alternative to keeping high-churn fields (request IDs, user IDs) as metric labels: instead of a label multiplying series count, a sampled exemplar carries just enough of a pointer to jump from an aggregate metric straight to one concrete trace — no series growth, full request-level detail on demand.
Tempo vs Jaeger
| Concern | Tempo | Jaeger |
|---|---|---|
| Storage requirement | Object storage only | Cassandra / Elasticsearch (indexed) or Badger (local) |
| Operational overhead | Low — no index cluster to run | Higher — index cluster sizing, retention, upgrades |
| Tag/attribute search | TraceQL, scoped to recent blocks | Full tag index, historically stronger ad hoc search |
| Ingest protocols | OTLP, Jaeger, Zipkin — all native | OTLP-native since Jaeger v2 (rebuilt on OTel Collector) |
Why it matters here: Tempo is the trace-storage tier in the ShipSolid Grafana Cloud stack, and exemplars are the standing answer to “how do I get request-level detail without adding a label” — whenever a proposed metric label is actually about identifying one specific request, the exemplar path into Tempo is the correct redirect, not a What is Cardinality (in observability) exception.
Local graph
Linked from 17 notes
7 — Distributed Tracing Backend
How spans that arrive out of order, from different services, get assembled into one trace — and the two competing storage models (indexed search vs. object storage plus a trace-ID lookup) that trade query flexibility for cost.
What is Jaeger
CNCF-graduated distributed tracing system built at Uber in 2015, Dapper-lineage like Zipkin before it — and, since Jaeger v2, rebuilt on top of the OpenTelemetry Collector rather than bespoke ingestion code.
Grafana Cloud Deployment
How Alloy exports traces, metrics, and logs to Grafana Cloud, including credential architecture, Azure Key Vault setup, and troubleshooting.
7. Component Map (What Exists in the Wild)
OSS and managed-SaaS options for every layer of the telemetry ingestion pipeline, mapped against ShipSolid's own production experience.
Telemetry Gateways: Protocol-Specific Ingestion Points
The ingestion frontier is a fleet of protocol-specific gateways — OTLP, Prometheus remote-write, Syslog, Kafka, and legacy tracing/metrics protocols — each terminating a different producer's wire format before a shared auth/rate-limit/tenant-routing layer.
Agentic AI: Projects & Engineering Mastery
A book-shaped table of contents for Agentic AI: Projects & Engineering Mastery: hands-on practitioner builds, Principal/Staff-level technical leadership, and the lookup appendices and vendor/framework reference notes for the whole series. Book 6 of the AI Systems Engineering series.
Grafana Cloud
A book-shaped table of contents for Grafana Cloud: platform foundations through telemetry collection, Mimir/Loki/Tempo/Pyroscope, visualization, application observability, reliability tooling, developer experience, governance, and enterprise reference architectures — cross-linking existing notes instead of duplicating them.
2 — The Signals
Metrics, logs, traces, profiles, and events — what each is built to capture, what it costs, and which question it actually answers vs. which one people mistakenly ask it.
7 — Multi-Tenancy
Two separate guarantees hiding under one name — data isolation and performance fairness — and the tenant identification, quota enforcement, and selective backpressure that make both hold under shared infrastructure.
1 — OpenTelemetry SDKs & Semantic Conventions
OpenTelemetry is a specification and an API/SDK, not a backend — the pieces that make it up, and the semantic-convention vocabulary that lets two unrelated teams' telemetry be queried the same way.
9 — OTel Collector Pipeline Design
Receivers, processors, and exporters chained into a pipeline; why a platform runs more than one; and the agent/gateway topology that tail sampling specifically forces on that design.
Observability Engineering
A book-shaped table of contents for observability engineering: foundations through architecture, metrics, logging, tracing, profiling, OpenTelemetry, instrumentation, Kubernetes/cloud, data platforms, visualization, alerting, SRE integration, cost, security, platform engineering, AI-driven operations, and MAANG interview preparation — cross-linking existing prometheus/grafana-cloud/kubernetes/sre/platform-engineering notes instead of duplicating them.
Related notes
What is Jaeger
CNCF-graduated distributed tracing system built at Uber in 2015, Dapper-lineage like Zipkin before it — and, since Jaeger v2, rebuilt on top of the OpenTelemetry Collector rather than bespoke ingestion code.
What is Loki
Grafana Labs' log aggregation system — 'like Prometheus, but for logs': index only labels, store compressed chunks in object storage, query with LogQL. Shares its distributor/ingester/compactor architecture with Mimir and Tempo.
What is Mimir
Grafana Labs' horizontally-scalable, multi-tenant long-term storage for Prometheus metrics — the 2022 successor to Cortex, and the actual system serving every PromQL query and remote-write in a Grafana Cloud metrics stack.
What is Cortex (cortexproject)
CNCF Incubating, horizontally-scalable multi-tenant long-term storage for Prometheus — the project Grafana Mimir forked from in 2022, still maintained as the vendor-neutral, community-governed alternative once Grafana Labs redirected engineering effort to Mimir.