# Data Engineering
All Data Engineering notes →1 — Performance Optimization
Performance optimization for distributed pipelines — parallelism, partitioning strategy, data skew, shuffle optimization, and caching.
2 — Cost Optimization
Cost optimization for data platforms — storage and compute cost drivers, compression, autoscaling, and spot instance strategies.
3 — Capacity Planning
Capacity planning for data systems — throughput estimation, scaling strategy, benchmarking, and load testing.
# Dbms
All Dbms notes →1 — Query Performance
Covers reading EXPLAIN plans, diagnosing slow queries, and choosing the right index.
2 — Database Tuning
Covers connection pooling, buffer pool sizing, caching, statistics, and vacuum/analyze maintenance.
3 — Common Bottlenecks
Covers the most common production bottlenecks — lock contention, hot partitions, index bloat, and deadlocks.
# Kubernetes
All Kubernetes notes →1 — Resource Optimization
Why requests should track real p95 usage while limits stay loose — tight CPU limits throttle a container even when the node has idle capacity sitting unused right next to it.
2 — Scheduler Performance
Why scheduling throughput degrades non-linearly past a few thousand nodes unless percentageOfNodesToScore is tuned down from its default of scoring every feasible node.
3 — Cluster Autoscaler
Why Cluster Autoscaler scales purely on unschedulable pending pods rather than utilization metrics, making it reactive by design and blind to a burst until pods have already failed to schedule.
4 — Karpenter
Why Karpenter provisions right-sized nodes directly from pending pod shape instead of scaling pre-defined node groups, collapsing the ASG-and-node-group abstraction Cluster Autoscaler depends on.
5 — Vertical Pod Autoscaler
Why VPA's Auto and Recreate update modes still evict and restart a pod to resize it — true in-place resize without disruption only lands with the still-maturing InPlacePodVerticalScaling feature.
6 — Horizontal Pod Autoscaler
Why HPA's polling-interval and stabilization-window defaults make it structurally too slow for sub-minute traffic spikes, forcing teams toward custom metrics or KEDA to react in time.
7 — Network Performance
Why the CNI's choice between an overlay (VXLAN/IPIP encapsulation) and native BGP routing is usually the single biggest lever on pod-to-pod latency and throughput, ahead of kube-proxy mode.
8 — Storage Performance
Why local NVMe (local-path or a CSI ephemeral volume) beats network-attached PVs on latency, but only by trading away the pod-to-node decoupling that makes rescheduling safe.
9 — Large Cluster Design
Why Kubernetes' official node-count ceiling is really an etcd write-throughput and API server watch-fanout limit, which is why hyperscalers split fleets into many smaller clusters instead of pushing past it.