Table of Contents

Return to the Software Development Career Playbook

System design is one way to show reasoning about requirements, scale, failure, security, cost, and change. Role expectations and interview formats vary by employer and level.

Core Concepts to Master

Scalability

Scalability is the ability of a system to handle increased load. Two fundamental approaches:

  • Vertical scaling (scale up): add more CPU, memory, or storage to a single machine. Simple but has a ceiling.
  • Horizontal scaling (scale out): add more machines. Limits still arise from coordination, data, networks, cost, and service quotas.

When one host no longer meets a defined load or availability requirement, a design might distribute work across hosts. This adds coordination, replication, routing, and failure questions.

Load Balancing

A load balancer distributes incoming requests across multiple backend servers. Common algorithms:

  • Round robin: each request goes to the next server in a cycle.
  • Least connections: routes to the server with the fewest active connections.
  • Consistent hashing: routes a given request key (user ID, session ID) to the same server deterministically, used for distributed caches.

Caching

Caching stores frequently accessed data in fast memory to reduce database load and response latency.

Cache TypeExampleWhat It Caches
In-process cacheLocal dict/mapPer-instance data, not shared
Distributed cacheRedis, MemcachedShared across services, network-accessed
CDNCloudflare, CloudFrontStatic assets and cacheable responses at the edge

Cache invalidation requires an explicit freshness policy. Choose an eviction and update strategy, such as a time-to-live, write-through, or cache-aside policy, before implementation.

Databases

Know the trade-offs between database categories:

TypeExamplesBest For
Relational (SQL)PostgreSQL, MySQLStructured data, complex queries, ACID transactions
DocumentMongoDB, DynamoDBFlexible schemas, hierarchical data, high write throughput
Key-valueRedis, DynamoDBSimple lookups, session storage, caching
Column-familyCassandra, HBaseTime-series, write-heavy, wide column data
SearchElasticsearchFull-text search, log aggregation
GraphNeo4j, NeptuneRelationship-heavy queries (social graphs, fraud detection)

The CAP Theorem

The CAP theorem describes a choice during a network partition: a distributed system cannot guarantee both availability for every request and linearizable consistency across the partition. The choice depends on the operation and system design.

In practice, network partitions happen. So the real choice is: when a partition occurs, do you prefer consistency (return an error or wait rather than serve stale data) or availability (serve possibly stale data rather than an error)?

  • A consistency-first operation might reject or delay a request until replicas agree.
  • An availability-first operation might accept or serve a request on one side of the partition and reconcile later.

Avoid assigning one permanent CAP label to a product. For example, DynamoDB offers configurable read consistency , and behavior depends on the operation, region, and failure.

Message Queues and Event Streaming

Message queues (RabbitMQ, SQS) decouple services, allow async processing, and absorb traffic spikes. Event streaming systems such as Kafka provide durable, replayable records. Kafka ordering is scoped to a partition, so partition-key design controls which events retain order.

Use queues and streams when:

  • The caller does not require the downstream result before responding.
  • Processing can be delayed without user impact.
  • You need to fan out the same event to multiple consumers.

Consistency Patterns

PatternDefinition
LinearizabilityAn operation appears to take effect at one point between its start and completion. A read which starts after a write completes observes that write or a later one.
Eventual consistencyReplicas converge over time, reads may see stale data
Read-your-writes consistencyWithin a defined client session or identity scope, later reads observe that client’s completed writes.

How to Approach a System Design Interview

  1. Clarify requirements: functional behavior, scale, latency objective, availability, privacy, abuse, cost, recovery, and constraints.
  2. Estimate scale: users, requests per second, storage per day, units, and overhead. One billion records at 100 bytes each equal 100 GB before indexes, replication, metadata, and storage overhead.
  3. Define the API: show endpoints or messages, authentication, authorization, validation, and error behavior.
  4. High-level design: draw components and data flows. Explain each boundary.
  5. Deep dive: explore the data model, failures, scaling, consistency, observability, cost, and recovery.
  6. Trade-offs: state the operation and requirement. Cassandra offers tunable per-operation consistency. A valid choice explains replica count, consistency level, failure behavior, latency, and which guarantees the query needs.

Primary sources, checked 2026-10-10: Apache Cassandra tunable consistency , Apache Kafka documentation , and Amazon DynamoDB read consistency .

Resources for Learning System Design

ResourceFormatLevel
Designing Data-Intensive Applications, Martin KleppmannBookIntermediate to advanced, the canonical deep reference
System Design Interview, Alex Xu (Vol 1 and 2)BookPractical interview prep with worked examples
ByteByteGo (blog and newsletter)WebAccessible visual explanations, covers common interview questions
High Scalability (blog)WebReal-world architecture case studies
Grokking the System Design Interview (educative.io)Interactive courseStructured interview-style preparation

Next Steps

Practice and Verify

Artifact: Scored system design. Design a URL service with explicit users, request rate, record size, retention, privacy, abuse controls, availability goal, recovery target, and budget. Show unit math, API, data model, failure modes, observability, and trade-offs. Ask one reviewer to score the design.

Expected result: The arithmetic reconciles, each requirement maps to a component, and the reviewer records one unresolved trade-off.

REDACTED COMPLETED EXAMPLE
Raw records: 100 GB. Replication and index overhead: stated separately. Reviewer gap: no abuse-rate limit. Revision added quota.

Pass checklist

  • The artifact uses dated, role-specific evidence
  • Claims, ownership, team credit, and unknown values are truthful
  • Privacy, secrets, licenses, accessibility, cost, and safety received review where relevant
  • A reviewer or repeatable check produced feedback
  • The record names a timed next action and checkpoint

Source rule: Use primary product or protocol documentation for each claimed behavior.