8. Scale, Capacity, Performance, and Caching

Scalability is not a promise that “we can add more instances.” You need a load model, known bottlenecks, and safe limits on databases, queues, connection pools, external APIs, and other scarce resources. Caching can reduce latency and load, but it adds staleness and invalidation problems.

Evidence: S13, S20, S21, S31

Capacity math should answer a design question: required throughput, storage growth, partition count, connection limits, bandwidth, queue drain time, or cost. For production, confirm the assumptions against current service limits and load tests rather than treating estimates as proof.

Evidence: S31, S44

Checklist

• Normal, peak, and growth load are estimated.

• Critical latency and throughput targets are measurable.

• The slowest/scarcest dependency is identified.

• Connection/thread/worker pools are bounded.

• Autoscaling signals match the real bottleneck.

• Scaling consumers cannot overwhelm the next dependency.

• Queues are bounded operationally through backlog limits, admission control, or load shedding.

• Load and stress tests cover realistic data and dependency behavior.

• Caching solves a measured problem rather than being added by habit.

• Cache TTL/invalidation rules are explicit.

• Acceptable staleness is defined.

• Cache failure behavior is defined.

• Hot keys/partitions and uneven load are considered.

• Capacity alerts fire before users experience saturation.

Evidence: S13, S20, S21, S24, S31