System Design for Software Engineers: Concepts, Patterns, and Interview Prep
Table of Contents
Click Here to Return To the Software Development Career Playbook
System design is the skill that separates mid-level developers from senior engineers. The ability to decompose a complex problem, evaluate architectural trade-offs, and design systems that are scalable, reliable, and maintainable is what the senior-and-above interview loop tests. System design is not memorization. It is a framework for reasoning about scale, failure, and change applied to novel problems.
Core Concepts to Master
Scalability
Scalability is the ability of a system to handle increased load. Two fundamental approaches:
- Vertical scaling (scale up): add more CPU, memory, or storage to a single machine. Simple but has a ceiling.
- Horizontal scaling (scale out): add more machines. More complex but nearly unlimited.
Most modern systems scale horizontally, which introduces distributed systems problems: how do the machines coordinate, how is data replicated, and how do clients know which machine to talk to?
Load Balancing
A load balancer distributes incoming requests across multiple backend servers. Common algorithms:
- Round robin: each request goes to the next server in a cycle.
- Least connections: routes to the server with the fewest active connections.
- Consistent hashing: routes a given request key (user ID, session ID) to the same server deterministically, used for distributed caches.
Caching
Caching stores frequently accessed data in fast memory to reduce database load and response latency.
| Cache Type | Example | What It Caches |
|---|---|---|
| In-process cache | Local dict/map | Per-instance data; not shared |
| Distributed cache | Redis, Memcached | Shared across services; network-accessed |
| CDN | Cloudflare, CloudFront | Static assets and cacheable responses at the edge |
Cache invalidation is the hardest caching problem. Design your eviction strategy (TTL vs. write-through vs. cache-aside) before implementing.
Databases
Know the trade-offs between database categories:
| Type | Examples | Best For |
|---|---|---|
| Relational (SQL) | PostgreSQL, MySQL | Structured data, complex queries, ACID transactions |
| Document | MongoDB, DynamoDB | Flexible schemas, hierarchical data, high write throughput |
| Key-value | Redis, DynamoDB | Simple lookups, session storage, caching |
| Column-family | Cassandra, HBase | Time-series, write-heavy, wide column data |
| Search | Elasticsearch | Full-text search, log aggregation |
| Graph | Neo4j, Neptune | Relationship-heavy queries (social graphs, fraud detection) |
The CAP Theorem
The CAP theorem states that a distributed system can guarantee at most two of three properties: Consistency, Availability, and Partition tolerance.
In practice, network partitions happen. So the real choice is: when a partition occurs, do you prefer consistency (return an error or wait rather than serve stale data) or availability (serve possibly stale data rather than an error)?
- CP systems (consistency + partition tolerance): ZooKeeper, HBase. Correct data or nothing.
- AP systems (availability + partition tolerance): Cassandra, DynamoDB. Always respond, possibly with stale data.
Message Queues and Event Streaming
Message queues (RabbitMQ, SQS) decouple services, allow async processing, and absorb traffic spikes. Event streaming (Kafka) provides durable, replayable, ordered event logs at scale.
Use queues and streams when:
- One service should not wait synchronously for another to respond.
- Processing can be delayed without user impact.
- You need to fan out the same event to multiple consumers.
Consistency Patterns
| Pattern | Definition |
|---|---|
| Strong consistency | Every read sees the most recent write |
| Eventual consistency | Replicas converge over time; reads may see stale data |
| Read-after-write consistency | A user always reads their own most recent writes |
How to Approach a System Design Interview
- Clarify requirements (5 minutes): functional requirements (what the system does), non-functional requirements (scale, latency SLA, availability), and constraints (read/write ratio, data size).
- Estimate scale (3 minutes): users, requests per second, storage per day. Back-of-the-envelope math. A URL shortener at 1 billion URLs stores roughly 1 TB if each URL averages 100 bytes.
- Define the API (3 minutes): what endpoints does the system expose? What do request and response shapes look like?
- High-level design (10 minutes): draw boxes for services, databases, caches, load balancers. Narrate what each does.
- Deep dive (15 minutes): the interviewer picks a component to explore. Data model, failure modes, scaling strategy, consistency guarantees.
- Trade-offs (throughout): every choice has a trade-off. Articulate it. “I chose Cassandra because we need high write throughput and can tolerate eventual consistency, but this means queries requiring strong consistency are not supported.”
Resources for Learning System Design
| Resource | Format | Level |
|---|---|---|
| Designing Data-Intensive Applications — Martin Kleppmann | Book | Intermediate to advanced; the canonical deep reference |
| System Design Interview — Alex Xu (Vol 1 and 2) | Book | Practical interview prep with worked examples |
| ByteByteGo (blog and newsletter) | Web | Accessible visual explanations; covers common interview questions |
| High Scalability (blog) | Web | Real-world architecture case studies |
| Grokking the System Design Interview (educative.io) | Interactive course | Structured interview-style preparation |
Next Steps
- Advancing from Junior to Senior Developer
- Software Engineering Technical Tracks
- Software Development Career Playbook Home
