Limited Time Offer:Up to 20% off Hello Interview Premium
Up to 20% off Hello Interview Premium 🎉
Hello Interview
Learn System Design
Introduction
How to Prepare
Delivery Framework
Core Concepts
Key Technologies
Common Patterns
Question Breakdowns
Networking Essentials
API Design
Data Modeling
Caching
Sharding
Consistent Hashing
CAP Theorem
Database Indexing
Numbers to Know
Bitly
Dropbox
Local Delivery Service
Ticketmaster
FB News Feed
Tinder
LeetCode
WhatsApp
Rate Limiter
FB Live Comments
FB Post Search
YouTube Top K
Uber
YouTube
Web Crawler
Ad Click Aggregator
News Aggregator
Yelp
Strava
Online Auction
Price Tracking Service
Instagram
Robinhood
Google Docs
Distributed Cache
Job Scheduler
Payment System
Metrics Monitoring
ChatGPT
Real-time Updates
Dealing with Contention
Multi-step Processes
Scaling Reads
Scaling Writes
Handling Large Blobs
Managing Long Running Tasks
Redis
Elasticsearch
Kafka
API Gateway
Cassandra
DynamoDB
PostgreSQL
Flink
ZooKeeper
Time Series Databases
Data Structures for Big Data
Vector Databases
Vote For New Content
Pricing
Sign in / Sign up
Search
⌘K
Pricing

Tutor

Common Problems

Distributed Cache

Scaling Reads
Published
ByEvan King·
hard

Try This Problem Yourself

Practice with guided hints and real-time feedback

Understanding the Problem

💾 What is a Distributed Cache? A distributed cache is a system that stores data as key-value pairs in memory across multiple machines in a network. Unlike single-node caches that are limited by the resources of one machine, distributed caches scale horizontally across many nodes to handle massive workloads. The cache cluster works together to partition and replicate data, ensuring high availability and fault tolerance when individual nodes fail.

Functional Requirements

Core Requirements
  1. Users should be able to set, get, and delete key-value pairs.
  2. Users should be able to configure the expiration time for key-value pairs.
  3. Data should be evicted according to Least Recently Used (LRU) policy.
Below the line (out of scope)
  • Users should be able to configure the cache size.
We opted for an LRU eviction policy, but you'll want to ask your interviewer what they're looking for if they weren't explicitly upfront. There are, of course, other eviction policies you could implement, like LFU, FIFO, and custom policies.

Non-Functional Requirements

At this point in the interview, you should ask the interviewer what sort of scale we are expecting. This will have a big impact on your design, starting with how you define the non-functional requirements.
If I were your interviewer, I would say we need to store up to 1TB of data and expect to handle a peak of up to 100k requests per second.
Core Requirements
  1. The system should be highly available. Eventual consistency is acceptable.
  2. The system should support low latency operations (< 10ms for get and set requests).
  3. The system should be scalable to support the expected 1TB of data and 100k requests per second.
Below the line (out of scope)
  • Durability (data persistence across restarts)
  • Strong consistency guarantees
  • Complex querying capabilities
  • Transaction support
Note that I'm making quite a few strong assumptions about what we care about here. Make sure you're confirming this with your interviewer. Chances are you've used a cache before, so you know the plethora of potential trade-offs. Some interviewers might care about durability, for example, just ask.

The Set Up

Planning the Approach

Defining the Core Entities

The API

High-Level Design

1) Users should be able to set, get, and delete key-value pairs

2) Users should be able to configure the expiration time for key-value pairs

3) Data should be evicted according to LRU policy

Potential Deep Dives

1) How do we ensure our cache is highly available and fault tolerant?

2) How do we ensure our cache is scalable?

3) How can we ensure an even distribution of keys across our nodes?

4) What happens if you have a hot key that is being read from a lot?

5) What happens if you have a hot key that is being written to a lot?

6) How do we ensure our cache is performant?

Tying it all together

What is Expected at Each Level?

Mid-level

Senior

Staff

Purchase Premium to Keep Reading

Unlock this article and so much more with Hello Interview Premium
Buy Premium

Currently up to 20% off

Hello Interview Premium

System Design Guided Practice
Exclusive content
Recent interview questions
Learn More
Reading Progress

On This Page

Understanding the Problem

Functional Requirements

Non-Functional Requirements

The Set Up

Planning the Approach

Defining the Core Entities

The API

High-Level Design

1) Users should be able to set, get, and delete key-value pairs

2) Users should be able to configure the expiration time for key-value pairs

3) Data should be evicted according to LRU policy

Potential Deep Dives

1) How do we ensure our cache is highly available and fault tolerant?

2) How do we ensure our cache is scalable?

3) How can we ensure an even distribution of keys across our nodes?

4) What happens if you have a hot key that is being read from a lot?

5) What happens if you have a hot key that is being written to a lot?

6) How do we ensure our cache is performant?

Tying it all together

What is Expected at Each Level?

Mid-level

Senior

Staff

Questions
Meta SWE Interview QuestionsAmazon SWE Interview QuestionsGoogle SWE Interview QuestionsOpenAI SWE Interview QuestionsEngineering Manager (EM) Interview Questions
Learn
Learn System DesignLearn DSALearn BehavioralLearn ML System DesignLearn Low Level DesignGuided Practice
Links
FAQPricingGift PremiumHello Interview Premium
Legal
Terms and ConditionsPrivacy PolicySecurity
Contact
About UsProduct Support

7511 Greenwood Ave North Unit #4238 Seattle WA 98103


© 2026 Optick Labs Inc. All rights reserved.