21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture • High-Level System Design (HLD): Distributed Systems at Scale • Rishabh Ki Kaksha

High-Level System Design (HLD): Distributed Systems at Scale › 21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture

60 mins

← Prev Next →

Course Syllabus

Module 1: Distributed Systems Core & Scalability Fundamentals

01. Scalability Dimensions: Vertical Scaling vs Horizontal Scaling & Latency vs Throughput 02. CAP Theorem vs PACELC Theorem: Consistency, Availability & Partition Tolerance Tradeoffs 03. Consistency Models: Strong Consistency, Eventual Consistency, Read-Your-Writes & Monotonic Reads

Module 2: Networking, Load Balancing & Reverse Proxies

04. Layer 4 vs Layer 7 Load Balancing: Round Robin, Weighted, Least Connections & IP Hash 05. High-Availability Load Balancers: Keepalived, VRRP, Anycast DNS & Failover Clustering 06. API Gateway Pattern: Authentication, Rate Limiting, Request Routing & Circuit Breakers

Module 3: Database Scaling: Sharding, Replication & Distributed Transactions

07. Database Replication: Master-Slave vs Multi-Master & Synchronous vs Asynchronous Replication 08. Database Sharding: Horizontal Partitioning, Directory-Based Sharding & Rebalancing Hotspots 09. Distributed Transactions: Two-Phase Commit (2PC) vs Saga Pattern (Choreography vs Orchestration)

Module 4: Caching Architectures, CDN & Consistent Hashing

10. Caching Strategies: Cache-Aside, Read-Through, Write-Through, Write-Around & Write-Back 11. Cache Invalidation Nightmares: Cache Avalanche, Cache Stampede, Thundering Herd & Cache Penetration 12. Consistent Hashing with Virtual Nodes (Solving Cache Clustering & Dynamic Scaling) 13. Content Delivery Networks (CDN): Edge Caching, Dynamic Site Acceleration & Origin Shielding

Module 5: Message Brokers, Event-Driven Architecture & Stream Processing

14. Message Queues vs Event Streams: RabbitMQ vs Apache Kafka (Partitioning, Consumer Groups) 15. Exactly-Once Processing Semantics (EOS), Idempotency Keys & Deduplication Architecture 16. Distributed Unique ID Generators: Twitter Snowflake, UUIDv7 & Database Ticket Servers

Module 6: FAANG Classic HLD Architectural Case Studies

17. System Design Case Study 1: URL Shortener (TinyURL / Bitly) at 100M Daily Requests 18. System Design Case Study 2: Distributed Rate Limiter & Web Application Firewall 19. System Design Case Study 3: Real-Time Chat & Notification System (WhatsApp / Slack) 20. System Design Case Study 4: Social Media News Feed (Twitter/Instagram) with Fan-out on Write/Read 21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture 22. System Design Case Study 6: Ride Hailing Geospatial Dispatch Architecture (Uber / Ola)

Interactive Lesson Studio

# 21. System Design Case Study 5: Video Streaming Platform (YouTube / Netflix) Transcoding Architecture

> High-Level Design (HLD) Notes by Rishabh - Topic 21

---

### 1. Video Ingestion & Adaptive Bitrate Streaming (ABR 🎬)

Raw 4K video file upload hoti hai (~50GB):
*   Client devices: 4K Smart TV, 1080p Laptop, 720p 4G Mobile, 360p 2G Mobile.
*   **Transcoding Pipeline:** Video ko multiple resolutions (1080p, 720p, 480p) aur codecs (H.264, AV1, VP9) me transcode karke **2-to-10 second Chunks** me slice karta hai (`.ts` / `.m4s` chunks).

---

### 2. ABR Protocols: HLS vs MPEG-DASH 📡

*   **HLS (HTTP Live Streaming - Apple):** `.m3u8` Master Playlist index file.
*   **MPEG-DASH:** `.mpd` XML Media Presentation Description.
*   *Player Mechanics:* Player network speed monitor karke automatically bina buffer kiye 1080p se 720p chunk par adapt kar leta hai!

---

### 3. Distributed Transcoding Worker Pipeline Diagram

```
[ User Uploads 4K Video ] ➔ [ Amazon S3 Raw Bucket ]
                                   ⬇
                       [ Kafka Task Queue ("transcode-jobs") ]
                                   ⬇
                 [ GPU Worker Pool (FFmpeg Chunk Transcoding) ]
                 (Splits video into 100 parallel chunks ➔ Encodes simultaneously)
                                   ⬇
                 [ Stitch & Generate .m3u8 Manifest ]
                                   ⬇
                 [ Global CDN Edge Storage (Cloudflare / Fastly) ]
```

---

### 4. Master Comparison Matrix

| Protocol | Manifest File | Chunk Format | Compatibility |
| :--- | :--- | :--- | :--- |
| **HLS** | `.m3u8` | MPEG-2 TS / fMP4 | 100% Apple iOS, Android, Browsers |
| **MPEG-DASH** | `.mpd` | Fragmented MP4 (fMP4) | Android, Smart TVs, Web browsers |

---

### 5. Interview Ke 3 Sawal Is Topic Se

1. **YouTube View Counter at Billions of views kaise scale karta hai?** -> In-memory distributed counters (Redis / Aggregation pipelines) jo 10-second batches me main database me update hoti hain (Eventual Consistency).
2. **Video chunking me Keyframe (I-Frame) alignment kyu zaroori hai?** -> Taaki user resolution switch karte waqt visual artifacts aur video freeze na ho.
3. **Netflix Open Connect CDN kya hai?** -> Netflix ke custom hardware storage appliances jo direct local ISP data centers me install hote hain taaki 95% traffic origin backbone ko touch hi na kare.

---
**Next Topic:** 22. System Design Case Study 6: Ride Hailing Geospatial Dispatch Architecture (Uber / Ola)

Loading lesson content...

Python 3.12

Full IDE

Live Execution Terminal

Click "Execute Live" to run code.
                

Lesson Mastery AI Quiz

AI Doubt Solver

CERTIFICATE OF COMPLETION

Rishabh Gehlot

PHP 8+ & Laravel 11 Master Track