# Patterns
All Patterns notes →Observability KPIs for the Fan-out / Fan-in Pattern
Observability KPIs for the Fan-out / Fan-in Pattern
Q1 Answer — Search Fan-Out Design
Worked answer to Fan-Out/Fan-In Practice Q1: partitioning, deadline propagation, partial-result policy, and instrumentation priority for a 200-shard search API at 150ms P99.
Q2 Answer — Hedging Trade-off
Worked answer to Fan-Out/Fan-In Practice Q2: the load-vs-latency math of hedged requests, when to enable them, and what to instrument first to justify the decision.
Q3 Answer — Context Cancellation Leak
Worked answer to Fan-Out/Fan-In Practice Q3: diagnosing a ghost-request leak where client-visible errors look healthy but infra cost and downstream CPU are elevated.
Q4 Answer — Aggregator Bottleneck
Worked answer to Fan-Out/Fan-In Practice Q4: min-heap merge strategy for a 500-shard top-K aggregation, its complexity, and how to keep aggregator latency from contaminating per-shard dashboards.
Q5 Answer — Sizing the Fan-Out Width
Worked answer to Fan-Out/Fan-In Practice Q5: the questions to ask and safeguards to add before accepting a design that fans out to all 8,000 tenant shards in prod.
Q6 Answer — Retry Storm
Worked answer to Fan-Out/Fan-In Practice Q6: how uncapped per-worker retries turn a transient blip into a full outage, and the retry policy that prevents it.
Q7 Answer — Backpressure and Load Shedding
Worked answer to Fan-Out/Fan-In Practice Q7: why an aggregate latency average hides a single overloaded shard, and where the fix belongs — dispatcher, worker, or shard.
Q8 Answer — Hierarchical Fan-Out
Worked answer to Fan-Out/Fan-In Practice Q8: budgeting a deadline across two nested fan-out levels and preventing partial failure from silently compounding across hops.
Q9 Answer — Validating Hedging and Deadline Propagation
Worked answer to Fan-Out/Fan-In Practice Q9: fault-injection, load testing, and canary comparison for validating a deadline-propagation and hedging rewrite before it reaches production.
01 — Monolith — Modular and Majestic
A single deployable unit with well-defined internal module boundaries. Underrated at MAANG interviews — the right answer when decomposition cost exceeds the benefit.
02 — Strangler Fig
Incrementally replace a legacy system by routing new functionality to a new implementation while the old system continues to run. Named after the fig tree that grows around and eventually replaces its host.
04 — Fan-Out / Fan-In
Decompose a request into parallel sub-tasks (fan-out), execute concurrently, then merge results (fan-in). The foundational pattern for latency-bound aggregation.
05 — Backpressure
Signal from a slow consumer to a fast producer to slow down. Prevents unbounded queue growth, OOM, and cascading overload. The foundational flow-control pattern.
07 — Circuit Breaker
A state machine (Closed → Open → Half-Open) that stops calls to a failing dependency before they cascade. The foundational resilience pattern for distributed systems.
08 — Retry with Exponential Backoff and Jitter
Retry transient failures with exponentially increasing wait times and randomised jitter to prevent thundering-herd recovery storms. The foundational pattern for resilient RPC.
09 — Bulkhead
Partition resources (thread pools, connection pools, semaphores) so that a failure or overload in one partition cannot exhaust resources for others. Limits blast radius.
10 — Hedged Requests
A tail-latency optimization that issues the same idempotent request to multiple replicas and uses whichever responds first, trading extra compute for dramatically lower P99/P999.
12 — CQRS — Command Query Responsibility Segregation
Separate the write model (commands) from the read model (queries). Unlocks independent scaling, optimised projections, and eventual-consistency trade-offs at scale.
13 — Event Sourcing
Store state as an immutable, append-only sequence of domain events. Current state is derived by replaying the log. Gives audit trail, temporal queries, and projection flexibility for free.
14 — Transactional Outbox
Atomically write to the database and publish a message by using a single local transaction. The outbox table is polled or tailed by CDC to publish reliably. Solves the dual-write problem.
15 — Saga
Manage distributed transactions across multiple services using a sequence of local transactions with compensating actions on failure. The microservices answer to 2PC.
01 — Sidecar
Co-locate a helper container with the application container to handle cross-cutting concerns — TLS, observability, auth, retries — without modifying application code.
01 — What Is a Pattern?
The history and vocabulary of pattern thinking — Christopher Alexander's pattern language, the Gang of Four's catalog, anti-patterns, and how a pattern differs from a one-off design decision.
02 — Pattern Selection & Trade-offs
How to choose between competing patterns under real forces: naming the context, weighing trade-offs and consequences, composing multiple patterns together, and recognizing when a pattern has outlived its fit.
03 — SOLID Revisited — Principal-Level Framing
SRP, OCP, LSP, ISP, and DIP reframed at architecture scale — stable dependencies and stable abstractions as the load-bearing idea, not the five-bullet mnemonic.
01 — Creational Patterns
Controlling object instantiation at the composition-root level: Singleton, Factory Method, Abstract Factory, Builder, and Prototype — when each earns its complexity and how they compose in a real dependency graph.
02 — Structural Patterns
Composing objects and classes into larger structures without duplicating behavior: Adapter, Bridge, Composite, Decorator, Facade, Flyweight, and Proxy.
03 — Behavioral Patterns
Distributing responsibility and communication between objects: Strategy, Observer, Command, Chain of Responsibility, Mediator, Memento, Interpreter, Iterator, State, Template Method, and Visitor.
01 — Layering Patterns
Layered Architecture, Hexagonal (Ports & Adapters), Onion, Clean Architecture, and Screaming Architecture — how each draws the boundary between domain logic and infrastructure, and what that boundary costs.
02 — Domain Modeling Patterns
DDD's tactical toolkit: Rich Domain Model vs. Anemic Model, Aggregate, Entity, Value Object, Repository, Factory, and Specification — the vocabulary a domain model needs to stay consistent under concurrent writes.
03 — Transaction Patterns
Unit of Work, Identity Map, Lazy Loading, and Optimistic vs. Pessimistic Locking — the patterns that keep an object graph and its persisted state from drifting apart.
04 — Integration Patterns
DTO, Gateway, Data Mapper, Service Layer, Table Data Gateway, and Active Record — the patterns that mediate between a domain model and everything outside it (databases, external services, the wire).
01 — Dependency Management Patterns
Constructor, Method, and Property Injection, Service Locator, and IoC Containers — how a dependency graph gets assembled and why constructor injection is the default the others are exceptions to.
02 — Extensibility Patterns
Plugin Architecture, Module Pattern, Strategy Registration, Reflection, and Dynamic Loading — how a system stays open to new behavior without recompiling its core.
03 — Service Decomposition
Decomposing by business capability, domain, or bounded context, and building self-contained systems — the remaining decomposition axes beyond the Monolith-vs-Strangler-Fig choice already covered in this book.
06 — Service Communication
Request-Response, Async Messaging, Event Streaming, Pub/Sub, and RPC/gRPC — the communication styles a service boundary can choose between, beyond the Fan-Out/Fan-In and Backpressure patterns already covered in this book.
11 — Reliability Patterns (Microservice Building Blocks)
Timeout, Rate Limiter, Fallback, and Adaptive Concurrency — the remaining resilience building blocks beyond Circuit Breaker, Retry, Bulkhead, and Hedged Requests, which already have their own chapters in this book.
16 — Data Patterns (Microservice)
Database-per-Service, Shared Database, and Materialized View — the remaining data-ownership patterns beyond CQRS, Event Sourcing, Outbox, and Saga, which already have their own chapters in this book.
01 — Consensus Patterns
Raft, Paxos, Leader Election, and Quorum — how a distributed system agrees on a single value or leader despite node failures and network partitions.
02 — Coordination Patterns
Distributed Lock, Lease, Heartbeat, Membership, and Gossip — the primitives nodes use to coordinate without a single point of failure.
03 — Replication Patterns
Leader-Follower, Leaderless, Multi-Leader, and Read Replica replication — how copies of the same data stay available and how they diverge under partition.
04 — Consistency Patterns
Strong, Eventual, Causal, Read-Your-Writes, and Monotonic Reads consistency models — what guarantee a client actually gets, and what it costs in latency and availability.
01 — Message Broker Patterns
Queue, Topic, Fan-out, Dead Letter Queue, and Delayed Queue — the delivery topologies a message broker offers and when each one fits.
02 — Event Patterns
Event Notification vs. Event-Carried State Transfer, and Choreography vs. Orchestration — how services stay decoupled through events, building on the Event Sourcing and Saga patterns already covered in this book.
03 — Streaming Patterns
Windowing, Watermarks, and Exactly-Once vs. At-Least-Once vs. At-Most-Once delivery semantics — the vocabulary for reasoning about correctness in a stream processing pipeline.
01 — REST Patterns
Resource modeling, pagination, filtering, HATEOAS, and versioning — the design decisions that separate a well-behaved REST API from an RPC call wearing HTTP verbs.
02 — RPC Patterns
gRPC and Protobuf, and the unary, streaming, and bidirectional-streaming call shapes — when RPC's tighter contract and lower overhead beat REST's looser one.
03 — API Gateway Patterns
Gateway, Backend for Frontend, Aggregation, and Federation — where cross-cutting API concerns (auth, rate limiting, fan-out) belong relative to the services behind them.
01 — Database Design Patterns
Normalization, denormalization, sharding, partitioning, and archiving — the structural decisions that determine how a database scales and how expensive its queries stay.
02 — Caching Patterns
Cache-Aside, Read-Through, Write-Through, Write-Back, and Refresh-Ahead — the five ways an application and its cache can disagree about who owns writing to the source of truth.
03 — Search Patterns
Inverted Index, Secondary Index, Bloom Filter, and Skip List — the data structures that make search and existence-checks fast at scale.
02 — Kubernetes Patterns
Ambassador, Adapter, Init Container, and Operator — the remaining multi-container and control-plane patterns beyond Sidecar, which already has its own chapter in this book.
03 — Cloud Infrastructure Patterns
Immutable Infrastructure, Auto Scaling, and the Blue-Green, Canary, and Rolling Update deployment strategies — how infrastructure changes roll out without a full-stop cutover.
04 — Multi-Region Patterns
Active-Passive, Active-Active, Geo-Replication, and Traffic Steering — the patterns that keep a system available when an entire region fails.
01 — Monitoring Patterns
RED, USE, the Four Golden Signals, and Saturation — the metric frameworks that decide what to measure on a service before an incident forces the question.
02 — Logging Patterns
Structured Logging, Correlation IDs, Log Sampling, and Log Aggregation — how logs stay searchable and affordable at scale instead of becoming a second, worse metrics system.
03 — Tracing Patterns
Distributed Tracing, Context Propagation, and Tail vs. Head Sampling — how a single request's path across services becomes reconstructable instead of a pile of disconnected spans.
04 — Alerting Patterns
Multi-window Burn Rate alerts, SLO Alerts, Composite Alerts, and Noise Reduction — the alerting design that pages on user-facing pain instead of every internal wobble.
01 — Resilience Patterns
Graceful Degradation, Load Shedding, Fail Fast, and Self-Healing — the system-level resilience postures that sit above any single pattern like Circuit Breaker or Bulkhead.
02 — Availability Patterns
High Availability, Disaster Recovery, Backup/Restore, and Chaos Engineering — the practices that turn an availability target into something actually tested, not just assumed.
03 — Scalability Patterns
Horizontal vs. Vertical Scaling, Elasticity, and Partitioning — the levers for handling more load, and why horizontal scaling is usually the one worth designing for first.
01 — Authentication Patterns
OAuth, OpenID Connect, API Keys, and Mutual TLS — proving identity between a client and a service, and between services themselves.
02 — Authorization Patterns
RBAC, ABAC, ReBAC, and Capability-Based Security — the models for deciding what an authenticated identity is allowed to do.
03 — Secure Communication Patterns
Zero Trust, Service Mesh, and Secrets Management — how a system stops trusting the network itself and starts verifying every call.
01 — Threading Patterns
Thread Pool, Producer-Consumer, and Reader-Writer — the foundational patterns for sharing work and data across threads without corrupting either.
02 — Lock-Free Programming
Compare-and-Swap, Atomic Variables, and Wait-Free vs. Lock-Free Queues — trading locks for retry loops to avoid blocking, priority inversion, and deadlock.
03 — Async Patterns
Futures, Promises, Reactive Streams, and the Actor Model — the abstractions for composing asynchronous work without callback-driven spaghetti.
01 — LLM System Patterns
RAG, Multi-Agent, Tool Calling, Planner-Executor, Reflection, and Memory — the pattern vocabulary for composing LLM calls into a system, at the design layer above any single framework.
02 — AI Infrastructure Patterns
Model Routing, Prompt Chaining, Guardrails, Human-in-the-Loop, and Evaluation Pipelines — the operational scaffolding that makes an LLM system reliable enough to run in production.
01 — Team Topologies
Stream-Aligned, Platform, Enabling, and Complicated-Subsystem teams — the four fundamental team types and the interaction modes between them.
02 — Conway's Law
Inverse Conway Maneuver, Team APIs, and Cognitive Load — why a system's architecture mirrors its org chart, and how to design the org chart on purpose instead of by accident.
03 — Engineering Leadership Patterns
RFC Process, ADRs, Design Reviews, Technical Governance, and Architecture Council — the decision-making scaffolding that lets an organization make architecture calls without every one becoming a meeting.
01 — Decision-Making Patterns
Buy vs. Build, Build vs. Platform, Sync vs. Async, SQL vs. NoSQL, and Monolith vs. Microservices — recurring architecture forks and the questions that actually resolve them.
02 — Trade-off Analysis Frameworks
CAP, PACELC, Latency vs. Throughput, Cost vs. Reliability, and Simplicity vs. Flexibility — the frameworks for making a trade-off explicit instead of leaving it implicit in the design.
01 — Combining Patterns
Pattern layering, pattern synergy, and pattern conflicts — how patterns interact once more than one is applied to the same system, and where two reasonable patterns pull against each other.
02 — Anti-Patterns
Distributed Monolith, Shared Database, God Object, Big Ball of Mud, Chatty Services, Golden Hammer, and Vendor Lock-in — the failure modes that look like patterns but are really a pattern applied without its context.
03 — Pattern Case Studies
How Amazon, Google, Netflix, Uber, Stripe, LinkedIn, and Cloudflare have combined these patterns in production — to be researched and written company by company, not as a single pass.
Patterns
A book-shaped table of contents for reusable engineering patterns spanning object-oriented design, enterprise architecture, distributed systems, messaging, APIs, cloud infrastructure, observability, security, concurrency, AI/agentic systems, and organizational design — grounded in production experience at scale.