20. System Design Case Study 4: Social Media News Feed (Twitter/Instagram) with Fan-out on Write/Read • High-Level System Design (HLD): Distributed Systems at Scale • Rishabh Ki Kaksha

High-Level System Design (HLD): Distributed Systems at Scale › 20. System Design Case Study 4: Social Media News Feed (Twitter/Instagram) with Fan-out on Write/Read

60 mins

← Prev Next →

Course Syllabus

Module 1: Distributed Systems Core & Scalability Fundamentals

01. Scalability Dimensions: Vertical Scaling vs Horizontal Scaling & Latency vs Throughput 02. CAP Theorem vs PACELC Theorem: Consistency, Availability & Partition Tolerance Tradeoffs 03. Consistency Models: Strong Consistency, Eventual Consistency, Read-Your-Writes & Monotonic Reads

Module 2: Networking, Load Balancing & Reverse Proxies

04. Layer 4 vs Layer 7 Load Balancing: Round Robin, Weighted, Least Connections & IP Hash 05. High-Availability Load Balancers: Keepalived, VRRP, Anycast DNS & Failover Clustering 06. API Gateway Pattern: Authentication, Rate Limiting, Request Routing & Circuit Breakers

Module 3: Database Scaling: Sharding, Replication & Distributed Transactions

07. Database Replication: Master-Slave vs Multi-Master & Synchronous vs Asynchronous Replication 08. Database Sharding: Horizontal Partitioning, Directory-Based Sharding & Rebalancing Hotspots 09. Distributed Transactions: Two-Phase Commit (2PC) vs Saga Pattern (Choreography vs Orchestration)

Module 4: Caching Architectures, CDN & Consistent Hashing

10. Caching Strategies: Cache-Aside, Read-Through, Write-Through, Write-Around & Write-Back 11. Cache Invalidation Nightmares: Cache Avalanche, Cache Stampede, Thundering Herd & Cache Penetration 12. Consistent Hashing with Virtual Nodes (Solving Cache Clustering & Dynamic Scaling) 13. Content Delivery Networks (CDN): Edge Caching, Dynamic Site Acceleration & Origin Shielding

Module 5: Message Brokers, Event-Driven Architecture & Stream Processing

14. Message Queues vs Event Streams: RabbitMQ vs Apache Kafka (Partitioning, Consumer Groups) 15. Exactly-Once Processing Semantics (EOS), Idempotency Keys & Deduplication Architecture 16. Distributed Unique ID Generators: Twitter Snowflake, UUIDv7 & Database Ticket Servers

Module 6: FAANG Classic HLD Architectural Case Studies

17. System Design Case Study 1: URL Shortener (TinyURL / Bitly) at 100M Daily Requests 18. System Design Case Study 2: Distributed Rate Limiter & Web Application Firewall 19. System Design Case Study 3: Real-Time Chat & Notification System (WhatsApp / Slack) 20. System Design Case Study 4: Social Media News Feed (Twitter/Instagram) with Fan-out on Write/Read 21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture 22. System Design Case Study 6: Ride Hailing Geospatial Dispatch Architecture (Uber / Ola)

Interactive Lesson Studio

# 20. System Design Case Study 4: Social Media News Feed (Twitter/Instagram) with Fan-out on Write/Read

> High-Level Design (HLD) Notes by Rishabh - Topic 20

---

### 1. The Fan-Out Dilemma: Push vs Pull Model 📰

*   **Fan-out on Write (Push Model):** User post karta hai $\rightarrow$ System uske saare followers ke pre-computed Redis Newsfeed Inbox me post ID push kar deta hai!
    *   *Read Latency:* ⚡ **$O(1)$ Instant** (Feed is pre-baked in RAM!).
    *   *Problem:* **Celebrity / Hotspot Flaw!** (Justin Bieber posts $\rightarrow$ 100 Million Redis writes in 1 second = System Crash!).
*   **Fan-out on Read (Pull Model):** Feed pre-compute nahi hoti; user jab feed open karta hai tab uske followed creators ke latest posts DB se fetch karke merge-sort hote hain.
    *   *Read Latency:* Slow $O(N \log N)$ disk query.

---

### 2. The Hybrid Industry Architecture (Twitter/Instagram Gold 👑)

1. **Normal Users (< 20,000 Followers):** **Fan-out on Write (Push Model)**.
2. **Celebrity Accounts (> 20,000 Followers):** **Fan-out on Read (Pull Model)**.
3. **Feed Generation:** User feed load karte waqt uske pre-computed Redis inbox me dynamically celebrity posts merge kar di jaati hain!

---

### 3. Master Comparison Matrix

| Model | Write Complexity | Read Complexity | Celebrity Handling |
| :--- | :--- | :--- | :--- |
| **Pure Push** | $O(\text{Followers})$ (Heavy) | ⚡ $O(1)$ (Direct Redis get) | ❌ Crashes on celebrities |
| **Pure Pull** | ⚡ $O(1)$ (Fast post) | $O(\text{Followed} \times \text{Posts})$ | Moderate |
| **Hybrid (Industry Standard)**| ⚡ **Balanced** | ⚡ **Blazing Fast (< 50ms)** | 🛡️ **Flawless** |

---

### 4. Interview Ke 3 Sawal Is Topic Se

1. **Feed ranking algorithm (ML score) cached inbox me kaise apply hota hai?** -> Redis me top 800 post IDs store hoti hain; Ranking service un 800 items ke feature vectors compute karke top 20 items sort karti hai.
2. **User feed pagination (Infinite Scroll) offset based kyu nahi hoti?** -> Cursor-based pagination (`cursor = last_post_id`) use hoti hai taaki naye posts aane par duplicates aur missed posts na hon.
3. **Inactive users (who haven't opened app in 30 days) ke liye Fan-out on write kaise optimize hota hai?** -> Inactive users ke feed inbox me write skip kar di jaati hai; unka feed sirf tab compute hota hai jab wo app login karein.

---
**Next Topic:** 21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture

Loading lesson content...

Python 3.12

Full IDE

Live Execution Terminal

Click "Execute Live" to run code.
                

Lesson Mastery AI Quiz

AI Doubt Solver

CERTIFICATE OF COMPLETION

Rishabh Gehlot

PHP 8+ & Laravel 11 Master Track