Unbounded Cardinality, Zero Alerts, and 14-Second Dashboard Loads: The Structural Limits of the Grafana Azure Monitor Plugin
The Grafana Azure Monitor datasource plugin connects directly to Azure Monitor at query time. On a 12-panel dashboard, that means 12 sequential ARM API calls on every open — 14 seconds before the first chart renders. It cannot feed Mimir alerting rules. It cannot attach environment labels without custom queries per panel. And promoting Azure tags as Prometheus labels explodes cardinality to thousands of series per resource group. This post documents the five structural gaps and the specific design decisions in a push-based exporter that close each one.
In FinTech, Your Trace Attributes Are a Compliance Liability
The observability data that makes a payments system debuggable — full request bodies in logs, account numbers on spans, card data in error messages — is the same data that turns your telemetry backend into an unregulated copy of your system of record. Trust in fintech isn't a status page; it's provable reliability plus a guarantee that sensitive data never left the boundary it was supposed to stay inside.
Hybrid Observability Scales on the Label Schema, Not the Collector Count
The hard part of hybrid and multi-cloud observability isn't running collectors in every environment — it's making one query resolve identically whether the data came from AKS, an on-prem plant VM, or SAP RISE. That needs a single enforced label schema, set at the collector layer, with no segment optional. Without it, cross-environment queries become per-source OR branches and the cost dashboard becomes an unattributable blob.
Why Cardinality Kills Observability Platforms (and How to Stop It)
Cardinality is the silent killer of Prometheus-based observability platforms. Here's how it happens, how to detect it early, and the label schema discipline that keeps ingestion costs sane at scale.