# Prometheus
All Prometheus notes →2 — Long-Term Storage
Why single-node Prometheus has no built-in long-term-storage or HA story, and how remote_write receivers like Thanos, Cortex, Mimir, and VictoriaMetrics fill that gap at a horizontally-scaled, multi-tenant layer.
1 — Scaling Prometheus
Federation, functional and horizontal sharding, HA server pairs, and remote_write as Prometheus's own answers to 'this one server can't hold it all anymore.'
3 — Performance Tuning
Tuning Prometheus's own resource footprint and query performance — memory, CPU, WAL replay time, compaction cadence, scrape interval, and retention as levers, with cardinality as the dominant cost driver.
4 — High Availability
Running Prometheus HA pairs, the duplicate-sample problem that creates, deduplication strategies, and failover/DR posture for a monitoring system that is itself a dependency.
# Kubernetes
All Kubernetes notes →1 — High Availability
Running three or more control-plane replicas behind a load balancer only buys availability if etcd quorum, not just the API server, survives the loss of any single node.
2 — Disaster Recovery
Disaster recovery is defined by RTO and RPO targets negotiated before an outage, not by how fast a runbook can be executed after one.
3 — Backup & Restore
Backing up etcd snapshots without also capturing PV data and CRDs restores a control plane that boots but manages nothing.
4 — Multi-Tenancy
Namespace isolation alone is not a security boundary; without NetworkPolicies, ResourceQuotas, and PodSecurityAdmission, one hostile tenant can starve or reach every other tenant on the same node.
5 — Cost Optimization
Most Kubernetes clusters waste money on the requested-vs-used gap, not on compute price: pods routinely request two to three times what they actually consume, so right-sizing requests beats chasing spot-instance discounts.
6 — Reliability Engineering
Kubernetes self-healing masks the symptoms of reliability problems, not the causes, so an SLO-driven error budget is what actually tells you whether the system is healthy.
7 — Production Anti-Patterns
Missing resource requests/limits, floating 'latest' image tags, and skipped liveness/readiness probes are the three anti-patterns responsible for the majority of production Kubernetes incidents.
8 — Kubernetes Failure Modes
The most dangerous Kubernetes failures are control-plane and etcd degradations, not pod crashes, because they fail silently — the API server keeps serving stale state while nothing can actually be scheduled or reconciled.
9 — Real Production Case Studies
Post-incident reviews from real Kubernetes outages consistently trace root cause to a control-plane or DNS bottleneck — CoreDNS, etcd, API server throttling — rather than the workload code itself.