How a 'Consistency Fix' in Program.cs Silently Broke Label Promotion Across Mimir and Loki
Renaming an OTel resource attribute from deployment.environment to deployment_environment in application code looked like the fix for inconsistent labels across Grafana Cloud's three signals. It's backwards: Mimir and Loki both promote and convert dotted attribute names on ingestion, so a pre-converted attribute silently fails to match and drops out of both label sets. The real fix belongs in the Alloy pipeline, scoped to traces only.
The Three Pillars of Observability Are a Storage Detail, Not a Strategy
Metrics, logs, and traces name three storage engines with different index models and cost curves — not three things to instrument separately. Teams that organize around the pillars build three disconnected silos and still can't say why one request was slow. The unit that matters is the correlated event: one trace_id and an exemplar that stitches a histogram bucket to the trace to the logs.