Social MediaQuestion #1ADVANCED

Instagram: Hybrid Feed, Media Pipeline & Social Graph

System design of Instagram at scale (500M+ MAU, 1B+ DAU) covering Hybrid Feed generation, media processing pipeline, social graph, and stories.

Category: Category 1: Social Media•Updated: 2026-08-30
#Social Media#Hybrid Feed#Media Pipeline#Social Graph#Stories

System Design: Social Media App Like Instagram

Author: System Design Document | Date: 2026-08-28 Target Scale: 500M+ MAU, 1B+ DAU (Instagram-scale) Document Level: FAANG Interview / Production-Ready Thinking


Table of Contents

  1. Requirements Clarification
  2. Back-of-Envelope Calculations
  3. System Interface Design
  4. High-Level Architecture
  5. Core Component Design
  6. Data Model
  7. Key Design Decisions
  8. Scalability Strategy
  9. Curveball Scenarios
  10. Failure Modes and Mitigations

1. Requirements Clarification

1.1 Functional Requirements

Core Features (MVP)

  • User Authentication & Profiles

    • Sign up / login (email, phone, OAuth)
    • Profile management (bio, profile picture, username)
    • Private / public account toggle
  • Media Posting

    • Upload photos (single or carousel up to 10)
    • Upload short videos (up to 60s for MVP; Reels up to 90s)
    • Apply filters and basic edits (brightness, contrast, crop)
    • Add captions, hashtags, location tags, and mentions (@user)
  • Feed

    • Home Feed (Chronological + Algorithmic): Posts from followed users, ranked by relevance
    • Explore Feed: Discovery feed of recommended content from non-followed users
    • Pull-to-refresh and infinite scroll pagination
  • Social Graph

    • Follow / unfollow users (asymmetric relationship)
    • Mutual follow indicator
    • Suggested follows based on social graph and interests
  • Interactions

    • Like / unlike posts
    • Comment / reply to comments (nested, up to 2 levels)
    • Share posts (to DM or story)
    • Save / bookmark posts
  • Direct Messaging (DM)

    • 1:1 and group messaging
    • Text, photo, video, GIF support
    • Read receipts and typing indicators
    • Disappearing messages (vanish mode)
  • Stories

    • 24-hour ephemeral content (photos + short videos)
    • Viewers list, reply to stories
    • Story highlights (pinned to profile)
  • Notifications

    • Push notifications for likes, comments, follows, mentions, DMs
    • In-app notification center

Advanced Features (Post-MVP)

  • Reels: Vertical short-form video (TikTok-style), up to 90s
  • Live Streaming: Real-time video broadcasting with live comments
  • Shopping: Product tags, checkout integration
  • IGTV / Long-form Video: Videos up to 60 minutes
  • Collaborative Posts: Co-authored posts with multiple creators
  • Close Friends List: Restricted story/feed sharing
  • Archiving: Archive posts without deleting

1.2 Non-Functional Requirements

Requirement Target Metric Rationale
Availability 99.95% (26 min downtime/month) Social media is expected to be always-on
Read Latency (Feed) P99 < 200ms Feed is the most frequent user action
Write Latency (Post) P99 < 2s (upload + process) Users tolerate slightly longer for uploads
Media Delivery P99 < 1s first byte (CDN) Visual-first app; slow images = churn
Throughput 100K+ QPS (peak read), 5K+ QPS (peak write) See Section 2 for derivation
Data Durability 99.9999999% (11 nines) User photos are irreplaceable
Consistency Eventual for feed/likes; Strong for auth/DM Feed can be eventually consistent; DM needs ordering
Scalability Linear horizontal scaling for all services Must handle 10x growth without re-architecture
Security GDPR/CCPA compliant, encrypted at rest and in transit Regulatory + Cost Efficiency

1.3 Assumptions

Assumption Value Reasoning
MAU 500 million Instagram-scale
DAU 300 million (60% of MAU) Industry standard engagement
Posts per DAU per day 2 posts Conservative estimate
Feed views per DAU per day 100 views (scrolls) Heavy usage pattern
Avg. post size (compressed) 500 KB (photo), 5 MB (video) After compression + CDN optimization
Avg. followers per user 150 Power-law distribution; median is lower
Top 1% users 10K+ followers each Influencers / celebrities
Top 0.1% users 1M+ followers each Mega-influencers
Media retention Permanent (unless deleted) Unlike Stories (24h)
Geographic distribution Global, with concentration in NA, EU, LATAM, SEA Instagram’s actual distribution
Read:Write ratio 100:1 (feed reads vs. posts) Social media is read-heavy

2. Back-of-Envelope Calculations

2.1 Storage Estimates

Daily Media Ingestion

Daily posts = DAU × posts_per_dau = 300M × 2 = 600M posts/day

Photo posts (70%): 420M × 500 KB = 210 TB/day
Video posts (30%): 180M × 5 MB = 900 TB/day
Total raw media/day = ~1,110 TB/day

Annual Storage (with redundancy)

Raw storage/year = 1,110 TB × 365 = ~405 PB/year

With 3x replication (durability): 405 × 3 = ~1,215 PB/year

With thumbnails, multiple resolutions, and metadata (~2x overhead):
Total storage/year ≈ 2,400 PB (~2.4 EB/year)

Key Insight: At Instagram scale, storage is measured in exabytes per year. This drives the need for tiered storage (hot/warm/cold) and aggressive compression.

Metadata Storage

Per post metadata: ~2 KB (JSON: author, caption, tags, timestamps, etc.)
600M posts/day × 2 KB = 1.2 TB/day metadata
Annual: ~440 TB/year (negligible compared to media)

Social Graph Storage

500M users × 150 avg followers × 8 bytes (two user IDs) = 600 GB
But with power-law distribution and fan-out cache, actual stored edges:
~75B edges × 16 bytes = ~1.2 TB (still manageable)

2.2 Bandwidth Estimates

Daily Egress

Feed views/day = DAU × views_per_dau = 300M × 100 = 30B views/day

Each view loads ~20 images (feed scroll):
30B × 20 × 500 KB = 300,000 TB/day = 300 EB/day of image egress

But with CDN caching (80% hit rate on popular content):
Actual origin egress = 300 EB × 20% = 60 EB/day
CDN edge egress = 300 EB/day

Key Insight: CDN caching is NOT optional at this scale. Without it, origin bandwidth requirements are physically impossible to sustain.

Peak Bandwidth

Peak hour = 20% of daily traffic in 1 hour
Peak image egress = 300 EB / 24 × 20% = 2.5 EB/hour = 700 TB/s

This is why aggressive CDN caching, image optimization, and lazy loading are critical.

2.3 QPS Estimates

Operation Calculation QPS (Avg) QPS (Peak)
Feed Read 30B views/day ÷ 86,400s ~347K ~700K
Post Write 600M posts/day ÷ 86,400s ~7K ~14K
Like 3B likes/day ÷ 86,400s ~35K ~70K
Comment 600M comments/day ÷ 86,400s ~7K ~14K
Follow/Unfollow 30M/day ÷ 86,400s ~350 ~700
DM Send 3B messages/day ÷ 86,400s ~35K ~70K
Profile View 3B views/day ÷ 86,400s ~35K ~70K
Auth/Login 300M logins/day ÷ 86,400s ~3.5K ~7K

Total estimated peak QPS: ~1M+ across all services combined.


2.4 Infrastructure Estimates

Assuming each app server handles 1,000 QPS (read-heavy, cached):

Feed Service servers = 700K peak QPS / 1,000 = 700 instances
Media Service servers = 14K peak QPS / 500 (write-heavy) = 28 instances
Like Service servers = 70K peak QPS / 1,000 = 70 instances
User Service servers = 70K peak QPS / 1,000 = 70 instances

Total app servers: ~1,000+ instances (excluding CDN, DB, cache layers)

Database instances (sharded):
- 500M users / 10M per shard = 50 user shards
- 600M posts/day / 10M per shard = 60 media shards
- Each shard = 1 primary + 2 replicas = 3 instances
- Total DB instances: (50 + 60) × 3 = 330 instances

Redis cache clusters:
- Feed cache: 200 nodes (for fan-out cache)
- Session cache: 100 nodes
- Rate limiting: 50 nodes

3. System Interface Design

3.1 RESTful API Definitions

User Service

POST   /api/v1/users/register          # Register new user
POST   /api/v1/users/login               # Login, returns JWT
GET    /api/v1/users/{user_id}            # Get user profile
PUT    /api/v1/users/{user_id}              # Update profile
GET    /api/v1/users/{user_id}/followers    # List followers (paginated)
GET    /api/v1/users/{user_id}/following    # List following (paginated)
POST   /api/v1/users/{user_id}/follow       # Follow a user
DELETE /api/v1/users/{user_id}/follow       # Unfollow a user
GET    /api/v1/users/suggestions             # Suggested users to follow

Media / Post Service

POST   /api/v1/media/upload                 # Upload media (multipart)
GET    /api/v1/media/{media_id}             # Get media metadata
DELETE /api/v1/media/{media_id}             # Delete media
POST   /api/v1/posts                        # Create a post
GET    /api/v1/posts/{post_id}              # Get post details
PUT    /api/v1/posts/{post_id}                # Update post (caption)
DELETE /api/v1/posts/{post_id}              # Delete post
GET    /api/v1/posts/{post_id}/comments         # Get comments
POST   /api/v1/posts/{post_id}/comments           # Add comment
GET    /api/v1/posts/{post_id}/likes        # Get likers (paginated)
POST   /api/v1/posts/{post_id}/like              # Like a post
DELETE /api/v1/posts/{post_id}/like           # Unlike a post
POST   /api/v1/posts/{post_id}/save           # Save/bookmark a post
DELETE /api/v1/posts/{post_id}/save           # Unsave a post

Feed Service

GET    /api/v1/feed/home                    # Home feed (algorithmic)
GET    /api/v1/feed/home?mode=chronological # Chronological feed
GET    /api/v1/feed/explore                 # Explore/discovery feed
GET    /api/v1/feed/tagged/{hashtag}      # Hashtag feed
GET    /api/v1/feed/nearby               # Location-based feed
POST   /api/v1/feed/refresh                 # Pull-to-refresh

Stories Service

POST   /api/v1/stories                      # Create a story
GET    /api/v1/stories/{user_id}            # Get user's active stories
POST   /api/v1/stories/{story_id}/view      # Mark story as viewed
POST   /api/v1/stories/{story_id}/reply     # Reply to a story
GET    /api/v1/stories/highlights/{user_id} # Get story highlights
POST   /api/v1/stories/highlights             # Create story highlight

Direct Messaging Service

POST   /api/v1/dm/conversations             # Create conversation
GET    /api/v1/dm/conversations             # List conversations
GET    /api/v1/dm/conversations/{conv_id}   # Get conversation details
POST   /api/v1/dm/conversations/{conv_id}/messages  # Send message
GET    /api/v1/dm/conversations/{conv_id}/messages   # Get messages
POST   /api/v1/dm/conversations/{conv_id}/read       # Mark as read
WS     /ws/dm                                # WebSocket for real-time DM

Notification Service

GET    /api/v1/notifications               # Get notifications (paginated)
POST   /api/v1/notifications/{id}/read     # Mark as read
POST   /api/v1/notifications/read-all       # Mark all as read
POST   /api/v1/notifications/push/register   # Register push token

3.2 Key API Response Examples

Feed Item Response

{
  "data": {
    "post_id": "p_8f3k29d",
    "author": {
      "user_id": "u_12345",
      "username": "traveler_jane",
      "profile_pic_url": "https://cdn.instagram.com/profiles/u_12345.jpg",
      "is_verified": true,
      "is_following": false
    },
    "media": [
      {
        "media_id": "m_abc123",
        "type": "image",
        "url": "https://cdn.instagram.com/media/m_abc123_1080.jpg",
        "thumbnail_url": "https://cdn.instagram.com/media/m_abc123_300.jpg",
        "width": 1080,
        "height": 1350
      }
    ],
    "caption": "Beautiful sunset at Bali 🌅 #travel #bali",
    "like_count": 15234,
    "comment_count": 342,
    "is_liked": false,
    "is_saved": false,
    "created_at": "2026-08-28T14:30:00Z",
    "location": {
      "name": "Ubud, Bali",
      "lat": -8.5069,
      "lng": 115.2625
    }
  },
  "pagination": {
    "next_cursor": "eyJwb3N0X2lkIjogInBfOGYzazI5ZCJ9",
    "has_more": true
  }
}

4. High-Level Architecture

flowchart TB
    subgraph Client["Client Layer"]
        iOS[iOS App]
        Android[Android App]
        Web[Web Client]
    end

    subgraph Edge["Edge / CDN Layer"]
        CDN[CDN<br/>CloudFront / Akamai]
        WAF[WAF / DDoS Protection]
        LB[Global Load Balancer<br/>L7]
    end

    subgraph Gateway["API Gateway Layer"]
        GW[API Gateway<br/>Kong / Custom]
        Auth[Auth Middleware<br/>JWT Validation]
        Rate[Rate Limiter<br/>Token Bucket]
    end

    subgraph Services["Microservices Layer"]
        subgraph Core["Core Services"]
            FeedSvc[Feed Service]
            MediaSvc[Media Service]
            UserSvc[User Service]
            GraphSvc[Social Graph Service]
        end
        subgraph Engagement["Engagement Services"]
            LikeSvc[Like Service]
            CommentSvc[Comment Service]
            NotifSvc[Notification Service]
        end
        subgraph Realtime["Real-time Services"]
            DMSvc[DM Service]
            StorySvc[Stories Service]
            LiveSvc[Live Service]
        end
        subgraph Discovery["Discovery Services"]
            ExploreSvc[Explore Service]
            SuggestSvc[Suggestion Service]
            SearchSvc[Search Service]
        end
    end

    subgraph Async["Async / Messaging Layer"]
        Kafka[Kafka Cluster<br/>Event Bus]
        MQ[Message Queue<br/>RabbitMQ / SQS]
    end

    subgraph Cache["Cache Layer"]
        Redis[Redis Cluster<br/>Session, Feed Cache, Rate Limit]
        Memcached[Memcached<br/>Hot Data]
    end

    subgraph Storage["Data Storage Layer"]
        subgraph DB["Databases"]
            UserDB[(User DB<br/>MySQL Sharded)]
            PostDB[(Post DB<br/>MySQL Sharded)]
            GraphDB[(Graph DB<br/>Cassandra)]
            NotifDB[(Notification DB<br/>MongoDB)]
        end
        subgraph ObjectStore["Object Storage"]
            S3[S3 / OSS<br/>Media Storage]
            ThumbS3[Thumbnail Store]
        end
        subgraph SearchStore["Search & Analytics"]
            ES[Elasticsearch<br/>Search Index]
            Clickhouse[ClickHouse<br/>Analytics]
        end
    end

    subgraph Processing["Processing Layer"]
        ImgProc[Image Processing<br/>Resize, Filter, Compress]
        VidProc[Video Processing<br/>Transcode, Thumbnail]
        ML[ML Pipeline<br/>Ranking, Moderation]
        Moderation[Content Moderation<br/>AI + Human]
    end

    subgraph Infra["Infrastructure"]
        K8s[Kubernetes Cluster]
        Monitor[Monitoring<br/>Prometheus + Grafana]
        Log[Logging<br/>ELK Stack]
        Trace[Distributed Tracing<br/>Jaeger]
    end

    iOS --> CDN
    Android --> CDN
    Web --> CDN

    CDN --> WAF
    WAF --> LB
    LB --> GW

    GW --> Auth
    Auth --> Rate
    Rate --> FeedSvc
    Rate --> MediaSvc
    Rate --> UserSvc
    Rate --> GraphSvc
    Rate --> LikeSvc
    Rate --> CommentSvc
    Rate --> NotifSvc
    Rate --> DMSvc
    Rate --> StorySvc
    Rate --> ExploreSvc
    Rate --> SuggestSvc
    Rate --> SearchSvc

    FeedSvc --> Redis
    FeedSvc --> Kafka
    MediaSvc --> S3
    MediaSvc --> ThumbS3
    MediaSvc --> Kafka
    UserSvc --> UserDB
    GraphSvc --> GraphDB
    LikeSvc --> Redis
    LikeSvc --> Kafka
    CommentSvc --> PostDB
    NotifSvc --> NotifDB
    NotifSvc --> Kafka
    DMSvc --> Redis
    DMSvc --> Kafka
    StorySvc --> S3
    StorySvc --> Kafka
    ExploreSvc --> ES
    SuggestSvc --> GraphDB
    SearchSvc --> ES

    Kafka --> ImgProc
    Kafka --> VidProc
    Kafka --> ML
    Kafka --> Moderation

    ImgProc --> S3
    VidProc --> S3
    ML --> Clickhouse
    Moderation --> PostDB

    FeedSvc --> PostDB
    MediaSvc --> PostDB
    CommentSvc --> Redis

    K8s -.-> Services
    Monitor -.-> Services
    Log -.-> Services
    Trace -.-> Services

5. Core Component Design

5.1 Feed Service

The feed is the most critical component of Instagram. It determines user engagement and retention.

Feed Generation Strategies

Strategy How It Works Pros Cons Best For
Fan-out on Write (Push) When a user posts, push to all followers’ feed caches O(1) read latency O(N) write cost; explodes for celebrities Users with < 10K followers
Pull on Read (Pull) When user opens feed, query posts from followed users O(1) write cost O(N) read cost; slow for many follows Users with many follows
Hybrid (Push + Pull) Push for normal users; pull for celebrities; merge at read time Balanced read/write Complex merging logic Production choice
Pre-computed + Real-time Pre-rank feed periodically; inject real-time posts at read Best UX, fresh content Storage overhead, stale pre-computed data Instagram-scale
┌─────────────────────────────────────────────────────────────┐
│                    Feed Generation Pipeline                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  1. CANDIDATE COLLECTION                                    │
│     ┌──────────┐  ┌──────────┐  ┌──────────┐               │
│     │ Pushed   │  │ Pulled   │  │ Explore  │               │
│     │ Feed     │  │ Feed     │  │ Candidates│               │
│     │ (Cache)  │  │ (Query)  │  │ (ML)     │               │
│     └────┬─────┘  └────┬─────┘  └────┬─────┘               │
│          └──────────────┴─────────────┘                      │
│                       ▼                                      │
│  2. RANKING (ML Model)                                       │
│     ┌─────────────────────────────────┐                      │
│     │ Features: recency, relationship,│                      │
│     │   media type, predicted like,   │                      │
│     │   predicted comment, diversity  │                      │
│     └──────────────┬──────────────────┘                      │
│                    ▼                                         │
│  3. FILTERING & RE-RANKING                                   │
│     ┌─────────────────────────────────┐                      │
│     │ Remove: hidden posts, muted,    │                      │
│     │   duplicate creators, sensitive │                      │
│     │ Re-rank: diversity, freshness   │                      │
│     └──────────────┬──────────────────┘                      │
│                    ▼                                         │
│  4. CACHING                                                  │
│     ┌─────────────────────────────────┐                      │
│     │ Redis: feed_{user_id}_{cursor} │                      │
│     │ TTL: 5 minutes (hot users)      │                      │
│     │ TTL: 30 minutes (cold users)    │                      │
│     └─────────────────────────────────┘                      │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Fan-out Threshold Decision

Threshold = 10,000 followers

User has < 10K followers:
    → Fan-out on write (push to follower caches)
    → Write cost: O(followers) = O(10K) max
    → Read cost: O(1) = cache lookup

User has >= 10K followers:
    → No fan-out on write
    → Pull on read (query their posts at feed time)
    → Write cost: O(1) = just store the post
    → Read cost: O(1) = their posts are cached individually

Feed Ranking Features

Feature Category Features Weight
Recency Post age (exponential decay) High
Relationship Mutual follow, DM frequency, profile visits, tag frequency High
Interest Past engagement with this creator, media type preference Medium
Predicted Affinity ML-predicted like probability, comment probability, share probability High
Diversity Creator diversity (avoid same creator spam), media type diversity Medium
Social Proof Like count, comment count (velocity, not absolute) Low

5.2 Media Service

Handles upload, processing, storage, and delivery of all media content.

Upload Pipeline

sequenceDiagram
    participant Client
    participant GW as API Gateway
    participant MediaSvc as Media Service
    participant Presign as Presign Service
    participant S3 as Object Storage
    participant Kafka
    participant ImgProc as Image Processor
    participant VidProc as Video Processor
    participant Moderation as Content Moderation
    participant DB as Post DB

    Client->>GW: POST /media/upload (request)
    GW->>MediaSvc: Validate request
    MediaSvc->>Presign: Generate presigned URL
    Presign->>S3: Create presigned PUT URL
    Presign-->>MediaSvc: Presigned URL + media_id
    MediaSvc-->>Client: Return presigned URL

    Client->>S3: PUT media (direct upload)
    S3-->>Client: 200 OK

    Client->>GW: POST /posts (create post with media_id)
    GW->>MediaSvc: Create post
    MediaSvc->>DB: Store post metadata
    MediaSvc->>Kafka: Emit POST_CREATED event

    Kafka->>ImgProc: Process image
    ImgProc->>S3: Store thumbnails (multiple sizes)
    ImgProc->>Moderation: Check content

    Kafka->>VidProc: Process video (if applicable)
    VidProc->>S3: Store transcoded versions
    VidProc->>Moderation: Check content

    Moderation->>DB: Update moderation status

Media Processing Pipeline

Original Upload
       │
       ▼
┌─────────────────┐
│  Validation      │  ← Format, size, aspect ratio check
└────────┬────────┘
         ▼
┌─────────────────┐
│  Image Processing│
│  (if photo)      │
│                  │
│  • Resize:       │
│    - 1080px      │  ← Full resolution
│    - 640px       │  ← Feed display
│    - 300px       │  ← Thumbnail
│    - 150px       │  ← Grid view
│  • Compress:     │
│    - JPEG quality 85% │
│    - WebP format   │  ← Modern browsers
│  • EXIF strip    │  ← Privacy
└────────┬────────┘
         ▼
┌─────────────────┐
│  Video Processing│
│  (if video)      │
│                  │
│  • Transcode:    │
│    - 1080p H.264 │  ← High quality
│    - 720p H.264  │  ← Standard
│    - 480p H.264  │  ← Low bandwidth
│    - H.265/HEVC  │  ← Modern devices
│  • Thumbnail:    │
│    - Keyframe extraction │
│    - 3 thumbnails │
│  • Audio:        │
│    - AAC 128kbps │
└────────┬────────┘
         ▼
┌─────────────────┐
│  Content         │
│  Moderation      │
│                  │
│  • NSFW detection│  ← AI model
│  • Copyright    │  ← Audio fingerprint
│  • Policy       │  ← Text/caption scan
└────────┬────────┘
         ▼
   Store in S3 + Update DB

CDN Strategy

Tier 1: Edge CDN (CloudFront / Akamai)
    - 200+ PoPs globally
    - Caches: thumbnails, profile pics, popular media
    - TTL: 24h for media, 1h for profile pics
    - Hit rate target: 85%+

Tier 2: Regional CDN
    - Regional caching for less popular content
    - TTL: 1 week
    - Fallback to origin if edge miss

Tier 3: Origin (S3)
    - Single source of truth
    - All processed media versions stored here

5.3 User Service

Authentication Flow

sequenceDiagram
    participant Client
    participant AuthSvc as Auth Service
    participant Redis as Redis (Session)
    participant UserDB as User DB
    participant Push as Push Notification

    Client->>AuthSvc: POST /login (email + password)
    AuthSvc->>UserDB: Query user by email
    UserDB-->>AuthSvc: User record + hashed password
    AuthSvc->>AuthSvc: Verify password (bcrypt)
    AuthSvc->>AuthSvc: Generate JWT (access + refresh)
    AuthSvc->>Redis: Store session
    AuthSvc-->>Client: JWT tokens + user info

    Note over Client,Push: Subsequent requests
    Client->>AuthSvc: Request with Authorization: Bearer <JWT>
    AuthSvc->>AuthSvc: Validate JWT signature
    AuthSvc->>Redis: Check session active
    Redis-->>AuthSvc: Session valid
    AuthSvc-->>Client: 200 OK

JWT Token Design

Access Token:
    - TTL: 15 minutes
    - Contains: user_id, scope, issued_at
    - Signed with RS256 (asymmetric)
    - Stored in memory (not localStorage for XSS protection)

Refresh Token:
    - TTL: 30 days
    - Stored in HTTP-only cookie
    - Rotated on each use (prevent token theft)
    - Revoked on logout / password change

5.4 Social Graph Service

Manages follow relationships, suggestions, and graph traversals.

Graph Storage Design

Follow relationships stored in Cassandra (wide-column):

Table: follows
    Partition Key: user_id (follower)
    Clustering Key: followed_user_id
    Columns: followed_user_id, timestamp, status

    Allows:
    - O(1) lookup: "Who does user X follow?"
    - O(1) lookup: "Does A follow B?" (via secondary index)

Table: followers
    Partition Key: user_id (followed)
    Clustering Key: follower_user_id
    Columns: follower_user_id, timestamp

    Allows:
    - O(1) lookup: "Who follows user X?"
    - Fan-out on write uses this table

Suggestion Algorithm

Suggestion Score = w1 × Friend-of-Friends
                 + w2 × Mutual Followers
                 + w3 × Profile View Velocity
                 + w4 × Search Query Match
                 + w5 × Location Proximity
                 + w6 × Account Similarity (interests)

Candidate Pool:
    1. Friends of friends (2-hop neighbors) — highest weight
    2. Users with high mutual follower count
    3. Users recently viewed by the target user
    4. Users searched for by the target user
    5. Users in same location with similar interests

Filter:
    - Exclude already followed
    - Exclude blocked/muted
    - Exclude inactive accounts (>90 days)
    - Limit to 50 suggestions per request

5.5 Notification Service

Notification Types and Delivery

Type Trigger Delivery Channel Priority
Like User likes your post Push + In-app Low
Comment User comments on your post Push + In-app Medium
Follow New follower Push + In-app Medium
Mention @mentioned in post/comment Push + In-app High
DM New direct message Push + In-app + Badge High
Story View Someone views your story In-app only Low
Live Start Followed user goes live Push High
Trending Your post is trending Push + In-app Medium

Notification Pipeline

flowchart LR
    A[Event: Like/Comment/Follow] --> B[Kafka Topic: notifications]
    B --> C[Notification Worker]
    C --> D{Deduplication}
    D -->|New| E[Write to MongoDB]
    D -->|Duplicate| F[Drop]
    E --> G[Batch Aggregation]
    G --> H[Push Notification Service]
    H --> I[FCM / APNs]
    I --> J[User Device]
    G --> K[In-App Notification Cache]
    K --> L[Redis: user_notifications_{id}]

Notification Deduplication and Batching

Problem: If 100 people like a post in 1 minute, don't send 100 push notifications.

Solution: Batch + Deduplicate

1. Write each notification to Kafka
2. Worker consumes with 30-second window
3. Aggregate: "John, Jane, and 98 others liked your photo"
4. Send ONE push notification per aggregation window
5. Store individual notifications in MongoDB for in-app detail view

Deduplication Key: {actor_id, target_user_id, post_id, action_type}

5.6 Direct Messaging (DM) Service

Real-time Architecture

WebSocket Connection Pool
         │
         ▼
┌─────────────────┐
│  Connection      │
│  Manager         │
│  (per-user WS    │
│   connections)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Message Router  │
│                  │
│  • Route to      │
│    conversation  │
│  • Persist │
│    to DB         │
│  • Fan-out to    │
│    recipients    │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Redis Pub/Sub   │
│  (in-memory      │
│   message bus)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Recipient WS   │
│  Connections    │
└─────────────────┘

Message Ordering and Reliability

Guarantees:
    - Per-conversation FIFO ordering (sequence numbers)
    - At-least-once delivery (ack + retry)
    - Message persistence (even if recipient offline)

Message Schema:
    {
        "message_id": "uuid",
        "conversation_id": "conv_123",
        "sender_id": "u_456",
        "type": "text" | "image" | "video" | "gif",
        "content": "...",
        "timestamp": 1690598400000,
        "sequence_num": 42,
        "status": "sent" | "delivered" | "read",
        "ttl": null | "2026-09-28T00:00:00Z"  // for disappearing messages
    }

5.7 Stories Service

Ephemeral Content Lifecycle

1. User creates story → Upload to S3 with 24h TTL metadata
2. Fan-out to followers' "available stories" list in Redis
3. When follower views:
   - Record view in Redis Set: story_views_{story_id}
   - Remove from follower's "unviewed" list
   - Show progress bar + viewer position
4. After 24 hours:
   - Cron job marks story as expired
   - Remove from all followers' lists
   - Media remains in S3 (for highlights) but hidden from feed
5. Story Highlights:
   - User can pin stories to profile
   - Stored separately, no TTL

Redis Data Structures for Stories

Available stories for a user:
    ZSET: stories_available_{user_id}
    Score: story_expiry_timestamp
    Value: story_id

Viewed stories for a user:
    SET: stories_viewed_{user_id}
    Value: story_id

Story viewers:
    SET: story_viewers_{story_id}
    Value: viewer_user_id

Story viewer order (for progress bar):
    ZSET: story_viewer_order_{story_id}
    Score: view_timestamp
    Value: viewer_user_id

5.8 Explore / Discovery Service

Recommendation Pipeline

Candidate Generation (Retrieval) → Ranking → Filtering → Serving

1. CANDIDATE GENERATION (Broad Retrieval)
   Sources:
   a) Collaborative Filtering: Users similar to you liked these posts
   b) Content-Based: Posts similar to what you engaged with
   c) Trending: Posts with high engagement velocity
   d) Social: Posts from friends-of-friends
   e) Location-Based: Popular posts near you

   Output: ~1000 candidate posts

2. RANKING (ML Model)
   Features:
   - User engagement history with similar content
   - Post engagement velocity (likes/min, comments/min)
   - Creator relevance to user
   - Media type preference
   - Recency

   Model: Gradient Boosted Trees (XGBoost) or Two-Tower DNN

   Output: Ranked list of ~100 posts

3. FILTERING
   - Remove already seen posts
   - Remove from blocked/muted users
   - Ensure diversity (max 2 posts per creator)
   - Ensure media type diversity

4. SERVING
   - Cache ranked results in Redis (TTL: 10 minutes)
   - Serve via CDN for media assets

6. Data Model

6.1 Relational Schema (MySQL - Sharded)

Users Table

CREATE TABLE users (
    user_id         BIGINT UNSIGNED NOT NULL,  -- Snowflake ID
    username              VARCHAR(30) NOT NULL UNIQUE,
    email                 VARCHAR(255) NOT NULL UNIQUE,
    phone                 VARCHAR(20) UNIQUE,
    password_hash         VARCHAR(255) NOT NULL,
    full_name             VARCHAR(100),
    bio                 VARCHAR(150),
    profile_pic_url       VARCHAR(500),
    is_private            BOOLEAN DEFAULT FALSE,
    is_verified           BOOLEAN DEFAULT FALSE,
    is_business_account BOOLEAN DEFAULT FALSE,
    created_at            TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at            TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    last_login_at         TIMESTAMP NULL,
    status                ENUM('active', 'suspended', 'deleted') DEFAULT 'active',

    PRIMARY KEY (user_id),
    INDEX idx_username (username),
    INDEX idx_email (email),
    INDEX idx_status (status)
) ENGINE=InnoDB;

Posts Table

CREATE TABLE posts (
    post_id         BIGINT UNSIGNED NOT NULL,  -- Snowflake ID
    user_id         BIGINT UNSIGNED NOT NULL,
    caption         TEXT,
    media_type      ENUM('image', 'video', 'carousel') NOT NULL,
    media_count     INT DEFAULT 1,
    location_id     BIGINT UNSIGNED NULL,
    is_deleted      BOOLEAN DEFAULT FALSE,
    moderation_status ENUM('pending', 'approved', 'rejected', 'reviewing') DEFAULT 'pending',
    like_count      INT UNSIGNED DEFAULT 0,
    comment_count   INT UNSIGNED DEFAULT 0,
    share_count     INT UNSIGNED DEFAULT 0,
    view_count      BIGINT UNSIGNED DEFAULT 0,  -- For video/reels
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

    PRIMARY KEY (post_id),
    INDEX idx_user_id (user_id),
    INDEX idx_created_at (created_at DESC),
    INDEX idx_moderation (moderation_status),
    INDEX idx_user_created (user_id, created_at DESC)
) ENGINE=InnoDB;

Media Table

CREATE TABLE media (
    media_id        BIGINT UNSIGNED NOT NULL,
    post_id         BIGINT UNSIGNED NOT NULL,
    media_type      ENUM('image', 'video') NOT NULL,
    original_url    VARCHAR(1000) NOT NULL,
    thumbnail_url   VARCHAR(1000),
    width           INT,
    height          INT,
    duration_sec    INT NULL,  -- For video
    file_size_bytes BIGINT UNSIGNED,
    format          VARCHAR(10),  -- jpeg, mp4, etc.
    display_order   INT DEFAULT 0,  -- For carousel
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

    PRIMARY KEY (media_id),
    INDEX idx_post_id (post_id),
    FOREIGN KEY (post_id) REFERENCES posts(post_id) ON DELETE CASCADE
) ENGINE=InnoDB;

Hashtags Table

CREATE TABLE post_hashtags (
    post_id     BIGINT UNSIGNED NOT NULL,
    hashtag     VARCHAR(30) NOT NULL,
    created_at  TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

    PRIMARY KEY (post_id, hashtag),
    INDEX idx_hashtag (hashtag),
    FOREIGN KEY (post_id) REFERENCES posts(post_id) ON DELETE CASCADE
) ENGINE=InnoDB;

CREATE TABLE hashtags (
    hashtag         VARCHAR(30) NOT NULL UNIQUE,
    post_count      INT UNSIGNED DEFAULT 0,
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

    PRIMARY KEY (hashtag)
) ENGINE=InnoDB;

6.2 NoSQL Schema (Cassandra - Social Graph)

-- Follow relationships (fan-out on write)
CREATE TABLE follows (
    follower_id     UUID,
    following_id    UUID,
    created_at      TIMESTAMP,
    status          TEXT,  -- 'pending', 'active'
    PRIMARY KEY (follower_id, following_id)
) WITH CLUSTERING ORDER BY (following_id ASC);

-- Followers (for notification fan-out)
CREATE TABLE followers (
    user_id         UUID,
    follower_id     UUID,
    created_at      TIMESTAMP,
    PRIMARY KEY (user_id, follower_id)
) WITH CLUSTERING ORDER BY (follower_id ASC);

-- User timeline (pushed feed items)
CREATE TABLE user_timeline (
    user_id         UUID,
    post_id         UUID,
    author_id       UUID,
    created_at      TIMESTAMP,
    like_count      INT,
    media_type      TEXT,
    PRIMARY KEY (user_id, created_at, post_id)
) WITH CLUSTERING ORDER BY (created_at DESC);

6.3 MongoDB Schema (Notifications)

{
    "_id": "ObjectId",
    "notification_id": "n_abc123",
    "recipient_id": "u_789",
    "actor_id": "u_456",
    "type": "like",
    "post_id": "p_xyz789",
    "message": "traveler_jane liked your photo",
    "actor_username": "traveler_jane",
    "actor_profile_pic": "https://cdn.../u_456.jpg",
    "post_thumbnail": "https://cdn.../p_xyz789_thumb.jpg",
    "is_read": false,
    "created_at": "2026-08-28T14:30:00Z",
    "aggregated_with": ["n_def456", "n_ghi789"],  // For batched notifications
    "expires_at": "2026-11-28T14:30:00Z"  // 90-day retention
}

{
    "mappings": {
        "properties": {
            "user_id": { "type": "long" },
            "username": {
                "type": "text",
                "analyzer": "username_analyzer",
                "fields": {
                    "keyword": { "type": "keyword" }
                }
            },
            "full_name": { "type": "text" },
            "bio": { "type": "text" },
            "post_id": { "type": "long" },
            "caption": { "type": "text" },
            "hashtags": { "type": "keyword" },
            "mentions": { "type": "keyword" },
            "location": { "type": "geo_point" },
            "media_type": { "type": "keyword" },
            "like_count": { "type": "integer" },
            "created_at": { "type": "date" },
            "is_verified": { "type": "boolean" },
            "follower_count": { "type": "integer" }
        }
    }
}

6.5 Redis Data Structures

# Session Storage
SET session:{session_id} "{user_id, expires_at, device_info}" EX 86400

# Feed Cache
ZSET feed:{user_id}:home
    Score: ranking_score (higher = more relevant)
    Value: post_id

# Like Count (Atomic Counter)
INCR like_count:{post_id}
GET like_count:{post_id}

# Like Membership (Has user liked?)
SADD likers:{post_id} {user_id}
SISMEMBER likers:{post_id} {user_id}

# Rate Limiting
INCR rate_limit:{user_id}:{endpoint}:{minute}
EXPIRE rate_limit:{user_id}:{endpoint}:{minute} 60

# User Online Status
SET online:{user_id} "1" EX 300  (5-minute TTL, heartbeat refreshes)

# DM Conversation Unread Count
HINCRBY dm_unread:{conv_id}:{user_id} unread_count 1

# Story Availability
ZADD stories_available:{user_id} {expiry_timestamp} {story_id}

# Trending Hashtags
ZSET trending:hashtags
    Score: engagement_velocity (likes + comments per minute)
    Value: hashtag_name

7. Key Design Decisions

7.1 Feed: Push vs. Pull

Aspect Push (Fan-out on Write) Pull (Compute on Read) Hybrid (Our Choice)
Read Latency O(1) — instant O(N) — slow for many follows O(1) — cached
Write Latency O(followers) — slow for celebrities O(1) — instant O(1) for celebrities, O(followers) for others
Storage High — duplicate feed per follower Low — single source of truth Medium — selective caching
Consistency Eventual (fan-out may lag) Strong (always fresh) Eventual (acceptable)
Celebrity Problem ❌ Explodes at 1M+ followers ✅ Handles gracefully ✅ Threshold-based switch

Decision: Hybrid with a threshold of 10K followers. Below threshold: push. Above threshold: pull. This gives O(1) reads for 99.9% of users while avoiding the celebrity fan-out explosion.


7.2 Database: SQL vs. NoSQL

Component Choice Rationale
User Data MySQL (sharded) ACID needed for auth, billing, profile consistency
Posts MySQL (sharded) Relational integrity with media, comments, likes
Social Graph Cassandra Massive write throughput for follows, O(1) lookups, horizontal scaling
Notifications MongoDB Schema flexibility, high write volume, eventual consistency OK
Feed Cache Redis Sub-millisecond reads, sorted sets for ranking
Search Elasticsearch Full-text search, faceting, geo-search
Media Metadata MySQL + S3 Relational for queries, object storage for blobs

Decision: Polyglot persistence. No single database can handle all access patterns optimally. Each component uses the best-fit storage engine.


7.3 Media Storage: S3 vs. Custom

Aspect S3 / Cloud Object Store Custom Distributed FS
Durability 11 nines (built-in) Must build yourself
Scalability Unlimited Limited by cluster size
Cost $0.023/GB/month Lower at massive scale
CDN Integration Native (CloudFront) Requires integration
Operations Managed service Team overhead

Decision: S3 (or equivalent cloud object store) for MVP and growth phase. Transition to custom storage only when cost savings justify the engineering overhead (typically at 10+ EB scale).


7.4 ID Generation: UUID vs. Snowflake

Aspect UUID v4 Snowflake Auto-Increment
Predictability Random Time-ordered Sequential
DB Friendly Poor (random writes) Good (time-ordered) Best
Sharding Natural Range-based Difficult
Info Leakage None Timestamp embedded Sequence visible
Length 36 chars (string) 19 chars (int64) Variable

Decision: Snowflake IDs. They are:

  • Time-ordered (good for DB indexing and range queries)
  • Distributed (no central coordinator bottleneck)
  • Compact (64-bit integer, fits in BIGINT)
  • Embed timestamp (useful for debugging and routing)
Snowflake ID Structure (64 bits):
┌────────────┬─────────┬─────────┬──────────┐
│ Timestamp  │ Worker  │ Datacenter │ Sequence │
│   41 bits  │  5 bits │   5 bits  │  12 bits │
│ (ms since  │ (machine│ (region)  │ (per-ms  │
│  custom    │  ID)    │           │  counter)│
│   epoch)   │         │           │          │
└────────────┴─────────┴─────────┴──────────┘

7.5 API Style: REST vs. GraphQL

Aspect REST GraphQL Hybrid
Over-fetching Common Solved Partial
Caching Native (HTTP cache) Complex (need Relay) Native for core
Learning Curve Low High Medium
Mobile Efficiency Poor (multiple calls) Excellent (single call) Good
CDN Friendly Yes No (POST only) Yes for core

Decision: REST for public APIs and CDN-cacheable endpoints (feed, media). GraphQL for mobile app’s complex queries (profile + feed + suggestions in one call). This hybrid approach gives the best of both worlds.


7.6 Consistency Model

Data Type Consistency Level Rationale
Auth / Sessions Strong Security-critical; must be consistent
User Profile Strong Users expect immediate updates
Posts Strong (eventual to readers) Writer sees immediately; readers see after propagation
Likes / Comments Eventual Count can be slightly stale; no harm
Feed Eventual Stale feed is acceptable; freshness < availability
DM Messages Strong (per conversation) Message ordering is critical
Notifications Eventual Delayed notification is acceptable
Stories Eventual 24h TTL makes eventual consistency fine

8. Scalability Strategy

8.1 Horizontal Scaling by Component

Component Scaling Strategy Shard Key Max Before Re-shard
User Service Consistent hashing on user_id user_id 10M users/shard
Post Service Range partitioning by post_id (time-ordered) post_id (embedded timestamp) 100M posts/shard
Feed Service Stateless; scale instances behind LB N/A (state in Redis) Unlimited
Like Service Hash partitioning on post_id post_id 50M likes/shard
Comment Service Hash partitioning on post_id post_id 10M comments/shard
Graph Service Consistent hashing on user_id user_id 5M users/shard
DM Service Hash partitioning on conversation_id conversation_id 10M convos/shard
Notification Service Hash partitioning on recipient_id recipient_id 10M users/shard

8.2 Caching Strategy

Cache Hierarchy:

Level 0: Client Cache (App Memory)
    - Profile pics, recent feed items
    - TTL: 5 minutes
    - Size: ~50 MB per device

Level 1: CDN Edge Cache
    - Media files (images, videos, thumbnails)
    - Static assets (JS, CSS)
    - TTL: 24 hours (media), 1 hour (profile pics)
    - Hit rate target: 85%+

Level 2: API Response Cache (Redis)
    - Feed responses
    - Profile data
    - Post details (hot posts)
    - TTL: 1-5 minutes
    - Hit rate target: 60%+

Level 3: Database Read Replicas
    - Read-heavy queries routed to replicas
    - Write-heavy queries go to primary
    - Replication lag: < 1 second

Level 4: Primary Database
    - Source of truth
    - All writes go here

8.3 Database Sharding Strategy

Phase 1: Single Shard (0 - 10M users)
    - Single MySQL instance with read replicas
    - Simple, easy to manage

Phase 2: Vertical Split (10M - 50M users)
    - Separate databases for users, posts, comments
    - Each with own read replicas

Phase 3: Horizontal Sharding (50M - 500M users)
    - Shard by user_id (consistent hashing)
    - 50 shards, each with 1 primary + 2 replicas
    - Proxy layer (Vitess / ShardingSphere) for routing

Phase 4: Multi-Region (500M+ users)
    - Active-active multi-region deployment
    - Each region has full shard set
    - Cross-region replication for disaster recovery
    - User routed to nearest region

8.4 Auto-Scaling Triggers

CPU Utilization > 70% for 2 minutes → Scale up
Memory Utilization > 80% for 1 minute → Scale up
Request Latency P99 > 500ms for 30 seconds → Scale up
Queue Depth > 1000 messages → Scale up consumers
Connection Count > 80% of max → Scale up

Scale Down:
CPU Utilization < 30% for 10 minutes → Scale down
Min instances: 3 (availability)
Max instances: 100 (cost control)
Cooldown: 5 minutes between scale events

9. Curveball Scenarios

Curveball 1: “What if reads are 100x writes?”

Context: This is actually Instagram’s real ratio. Feed reads vastly outnumber post writes.

Impact Analysis:

  • Feed read QPS: ~700K peak
  • Post write QPS: ~14K peak
  • Ratio: ~50:1 (even higher during peak hours)

Design Adaptations:

  1. Read-Optimized Architecture

    • Heavy caching at every layer (CDN → Redis → DB replicas)
    • Feed pre-computation: rank feeds in background, serve from cache
    • Read replicas: 10:1 replica-to-primary ratio for hot data
  2. Write-Optimized Paths

    • Async processing: accept write immediately, process in background
    • Eventual consistency: likes/comments update asynchronously
    • Batch writes: aggregate notifications, batch DB writes
  3. CDN as Primary Delivery

    • 85%+ of all reads served from CDN edge
    • Origin only handles cache misses and writes
    • Predictive preloading: preload likely-to-be-viewed content to CDN
  4. Feed Cache Invalidation Strategy

    Problem: How to invalidate feed cache when new posts arrive?
    
    Solution: Version-based cache with soft TTL
    - Each user's feed has a version number
    - New post from followed user → increment version
    - Cache key: feed:{user_id}:{version}
    - Old cache entries expire naturally (TTL)
    - No explicit invalidation needed

Result: The system is already designed for this ratio. The hybrid feed model, aggressive caching, and CDN strategy all optimize for read-heavy workloads.


Curveball 2: “What if we need to support Reels / Short-Form Video?”

Context: Reels is Instagram’s answer to TikTok. Short-form vertical video with music, effects, and algorithmic discovery.

New Requirements:

  • Video upload up to 90 seconds
  • Auto-play in feed (muted by default)
  • Music/audio licensing and attribution
  • Advanced video effects and filters
  • Separate Reels feed (algorithmic, not social graph)
  • Higher bandwidth requirements

Architecture Changes:

  1. Video Processing Pipeline Enhancement

    New processing steps:
    - Audio extraction and fingerprinting (for copyright)
    - Multiple bitrate laddering (288p, 360p, 480p, 720p, 1080p)
    - HLS/DASH adaptive streaming manifest generation
    - Thumbnail generation (3-5 keyframes)
    - Audio normalization and loudness matching
    - Effect/filter rendering (AR effects, transitions)
  2. Storage Impact

    Per Reel storage:
    - Original: 50 MB
    - Transcoded versions (5 bitrates): 20 MB × 5 = 100 MB
    - Thumbnails: 5 × 50 KB = 250 KB
    - Audio track: 2 MB
    - Metadata + manifests: 100 KB
    Total per Reel: ~152 MB
    
    If 30% of posts become Reels:
    180M Reels/day × 152 MB = 27.4 TB/day (vs. 900 TB for full video)
    Actually LESS than full video due to shorter duration
  3. New Service: Reels Service

    Responsibilities:
    - Reels feed generation (separate from main feed)
    - Music library management and licensing
    - Effect/filter catalog
    - Reels-specific ranking algorithm
    - Audio trend detection
    
    Ranking features (different from main feed):
    - Completion rate (did user watch till end?)
    - Rewatch rate
    - Share rate
    - Audio trend score
    - Effect novelty score
  4. Bandwidth Impact

    Reels auto-play in feed:
    - Each Reel preview: 2-3 seconds at 480p = ~1 MB
    - User scrolls 100 items/day, 30% are Reels
    - 30 Reels × 1 MB = 30 MB/user/day for Reels previews
    - 300M DAU × 30 MB = 9 EB/day for Reels previews alone
    
    Mitigation:
    - WiFi-only auto-play (option)
    - Lower bitrate for preview (288p)
    - Aggressive CDN caching of popular Reels
    - Preload next 2-3 Reels during watch
  5. Music Licensing Integration

    New components:
    - Music Catalog Service (licensed tracks)
    - Audio Fingerprinting (Shazam-like matching)
    - Royalty Tracking (per-play counting)
    - Takedown Pipeline (DMCA compliance)
    
    Flow:
    User selects music → Check license → Embed track ID →
    On play → Increment play count → Monthly royalty settlement

Curveball 3: “What if we need real-time collaboration (co-authored posts)?”

Context: Multiple creators collaborate on a single post, sharing ownership, credits, and analytics.

New Requirements:

  • Invite collaborators to a post
  • Real-time co-editing (caption, media selection)
  • Shared ownership and attribution
  • Combined follower reach
  • Split analytics and monetization

Architecture Changes:

  1. Data Model Extensions

    CREATE TABLE post_collaborators (
        post_id         BIGINT UNSIGNED NOT NULL,
        user_id         BIGINT UNSIGNED NOT NULL,
        role            ENUM('owner', 'collaborator', 'editor') NOT NULL,
        invited_by    BIGINT UNSIGNED NOT NULL,
        status          ENUM('pending', 'accepted', 'rejected') DEFAULT 'pending',
        added_at        TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
        PRIMARY KEY (post_id, user_id),
        FOREIGN KEY (post_id) REFERENCES posts(post_id) ON DELETE CASCADE,
        FOREIGN KEY (user_id) REFERENCES users(user_id)
    );
  2. Real-Time Co-Editing

    Technology: WebSocket + Operational Transformation (OT)
    
    Flow:
    1. Owner creates draft post
    2. Invites collaborators via WebSocket channel: collab_{post_id}
    3. Each editor connects to the channel
    4. Edges as OT operations (insert, delete, format)
    5. Server merges operations and broadcasts to all editors
    6. On publish: all collaborators notified
    
    Conflict Resolution:
    - Last-write-wins for simple fields (caption)
    - OT for rich text editing
    - Media additions are append-only (no conflicts)
  3. Feed and Discovery Impact

    Problem: A collaborative post appears in ALL collaborators' feeds.
    
    Solution:
    - Post is associated with multiple user_ids
    - Fan-out to ALL collaborators' followers
    - Feed shows all collaborators' profile pics
    - Clicking any collaborator navigates to their profile
    
    Storage Impact:
    - post_users table maps posts to all associated users
    - Feed fan-out multiplies by number of collaborators
    - Mitigation: Cap at 5 collaborators per post
  4. Analytics Splitting

    Each collaborator gets:
    - Their share of impressions (based on whose followers saw it)
    - Combined engagement metrics
    - Attribution breakdown
    
    Implementation:
    - impression_events table records (post_id, viewer_id, source_user_id)
    - source_user_id = whose feed the viewer was scrolling
    - Aggregated per collaborator for their analytics dashboard

Curveball 4: “What if a celebrity posts and gets 10M likes in an hour?”

Context: A mega-influencer (e.g., Selena Gomez with 400M followers) posts a photo.

Impact Analysis:

Fan-out cost: 400M followers × cache write = 400M writes
If each write takes 1ms: 400 seconds = 6.7 minutes of fan-out
During this time, followers see stale feeds

Mitigations:

  1. Threshold-Based Fan-out (Already Designed)

    • Users with > 10K followers use pull model
    • No fan-out explosion for celebrities
    • Their posts are cached individually with high replication
  2. Like Counter Handling

    Problem: 10M likes in 1 hour = 2,778 likes/second
    Single DB row update bottleneck
    
    Solution: Counter Sharding
    - Split like count across 100 virtual shards
    - Each shard handles ~28 likes/second (trivial)
    - Read: sum all shards (100 reads, parallelized)
    - Display: approximate count (good enough for UI)
    
    Redis Implementation:
    SET like_count:{post_id}:shard_001 123456
    SET like_count:{post_id}:shard_002 234567
    ...
    GETSUM like_count:{post_id}:shard_* → total
  3. Comment Handling

    Problem: 1M comments in 1 hour = 278 comments/second
    
    Solution:
    - Comments stored in Cassandra (high write throughput)
    - Paginated display (show top 50 + "load more")
    - Top comments pre-sorted by engagement
    - New comments appear after a short delay (eventual consistency)
  4. Notification Storm Prevention

    Problem: If the celebrity follows you back, you get a notification.
    If 1M people follow them simultaneously...
    
    Solution:
    - Rate-limit notifications per user (max 100/hour)
    - Batch notifications aggressively
    - Deprioritize low-importance notifications during storms
    - Queue overflow → discard lowest priority

Curveball 5: “What if we need to support live video streaming?”

Context: Instagram Live allows real-time video broadcasting with live comments and reactions.

Architecture Additions:

  1. Streaming Infrastructure

    Ingest → Processing → Distribution → Playback
    
    Ingest:
    - RTMP ingest servers (auto-scaled)
    - Each stream gets dedicated ingest instance
    - Geo-distributed ingest points
    
    Processing:
    - Real-time transcoding (multiple bitrates)
    - Low-latency HLS or WebRTC
    - Audio processing (noise reduction, normalization)
    
    Distribution:
    - CDN with live streaming optimization
    - Edge caching of recent segments
    - Adaptive bitrate switching
    
    Playback:
    - HLS for broad compatibility
    - WebRTC for lowest latency (newer browsers)
    - DASH for Android
  2. Live Comments Service

    Requirements:
    - Sub-second latency for comments
    - Handle 10K+ concurrent commenters
    - Comment moderation in real-time
    - Pinned comments, emoji reactions
    
    Architecture:
    - WebSocket connection per viewer
    - Comments published to Kafka topic: live_{stream_id}
    - Moderation worker filters toxic content
    - Clean comments broadcast to all viewers via WebSocket
    - Comment history stored in Cassandra for replay
  3. Scalability

    Concurrent streams: 100K (peak)
    Viewers per stream: 1M (top stream)
    Total concurrent viewers: 100M
    
    WebSocket connections: 100M concurrent
    Connection managers: 100M / 10K per instance = 10,000 instances
    
    Mitigation:
    - Connection multiplexing (one WS per device, not per stream)
    - Comment relay via server-sent events (SSE) for non-interactive viewers
    - Only active commenters get full WS connection

10. Failure Modes and Mitigations

10.1 Failure Mode Matrix

Failure Mode Probability Impact Detection Mitigation
CDN outage Low Critical CDN health checks, error rate spike Multi-CDN failover, direct origin fallback
Database primary failure Medium Critical Replication lag, write failures Automatic failover to replica, read-only mode
Redis cluster partition Medium High Cache miss rate spike, latency increase Local cache fallback, degraded mode
Kafka broker failure Low High Consumer lag, message loss Multi-broker replication, exactly-once semantics
S3 outage Very Low Critical Upload failures, media 404s Multi-region replication, CDN cache sustains reads
ML model degradation Medium Medium Engagement drop, A/B test failure Model rollback, rule-based fallback
DDoS attack Medium Critical Latency spike, resource exhaustion WAF, rate limiting, auto-scaling, scrubbing center
Data breach Low Critical Security monitoring, anomaly detection Encryption, access controls, incident response
Cascade failure Low Critical Cross-service error rate correlation Circuit breakers, bulkheads, graceful degradation
Clock skew Medium Low Duplicate IDs, ordering issues NTP synchronization, logical clocks

10.2 Detailed Mitigation Strategies

Circuit Breaker Pattern

Every inter-service call has a circuit breaker:

States:
    CLOSED   → Normal operation, requests flow through
    OPEN     → Failure threshold reached, requests fail fast
    HALF-OPEN → Test requests allowed, monitor success rate

Configuration:
    Failure threshold: 5 failures in 10 seconds → OPEN
    Recovery timeout: 30 seconds → HALF-OPEN
    Success threshold: 5 successes in HALF-OPEN → CLOSED
    Failure in HALF-OPEN → OPEN

Fallback behaviors:
    Feed Service: Return cached feed (stale but available)
    Like Service: Queue like for later, return optimistic count
    Notification Service: Queue notification, deliver when recovered
    Search Service: Return empty results with "try again" message

Bulkhead Pattern

Isolate failures by partitioning resources:

Thread Pool Bulkheads:
    Feed pool:     200 threads
    Media pool:       100 threads
    User pool:       50 threads
    Notification pool: 50 threads

    If Media pool is exhausted (video processing backlog),
    Feed and User pools remain unaffected.

Connection Pool Bulkheads:
    DB pool per service:
        User DB:     50 connections
        Post DB:     100 connections
        Graph DB:        50 connections

    If Post DB is slow, User DB connections are unaffected.

Graceful Degradation

When components fail, degrade gracefully instead of crashing:

Priority 1 (Always Available):
    - User authentication
    - Feed reading (cached)
    - Profile viewing

Priority 2 (Degrade if Needed):
    - Post creation (queue if processing is down)
    - Likes/comments (accept but delay processing)
    - Notifications (batch and delay)

Priority 3 (Disable if Needed):
    - Explore feed (return empty or cached)
    - Suggestions (return empty)
    - Stories (disable creation, allow viewing cached)
    - Live streaming (disable new streams)

User Communication:
    - Show "Some features may be slow" banner
    - Retry buttons for failed actions
    - Offline mode for cached content

Disaster Recovery

RTO (Recovery Time Objective): 1 hour
RPO (Recovery Point Objective): 5 minutes

Strategy:
    1. Multi-region active-passive deployment
       - Primary region: handles all traffic
       - Secondary region: warm standby, replicated data
       - Failover: DNS switch + traffic reroute (15 minutes)

    2. Data replication
       - MySQL: Cross-region replication (5-minute lag)
       - S3: Cross-region replication (continuous)
       - Redis: Cross-region async replication
       - Kafka: MirrorMaker for cross-region topic replication

    3. Regular DR drills
       - Monthly failover test (secondary region)
       - Quarterly full DR exercise
       - Annual chaos engineering game day

Chaos Engineering

Regularly inject failures to validate resilience:

Tests:
    - Kill random service instances → Verify auto-recovery
    - Simulate network partition → Verify split-brain handling
    - Inject latency (500ms-5s) → Verify timeout handling
    - Fill disk space → Verify graceful degradation
    - Corrupt cache entries → Verify cache miss handling
    - Simulate clock skew → Verify ID generation correctness

Tools:
    - Chaos Monkey (random instance termination)
    - Latency Monkey (random latency injection)
    - Custom fault injection framework

10.3 Monitoring and Alerting

Key Metrics

Availability:
    - Service uptime percentage (per service)
    - Error rate (5xx responses / total requests)
    - Circuit breaker state (per service pair)

Performance:
    - Request latency (P50, P95, P99) per endpoint
    - Throughput (QPS) per service
    - Cache hit rate (CDN, Redis, local)
    - Database query latency (P99)

Infrastructure:
    - CPU, memory, disk, network utilization
    - Connection pool usage
    - Queue depth (Kafka consumer lag)
    - Active WebSocket connections

Business:
    - Daily active users (DAU)
    - Posts created per hour
    - Feed views per hour
    - Upload success rate
    - Time to first byte (TTFB) for media

Alerting Thresholds

Metric Warning Critical Action
Error Rate > 1% for 5 min > 5% for 2 min Page on-call engineer
P99 Latency > 500ms for 5 min > 2s for 2 min Auto-scale + page
Cache Hit Rate < 50% for 10 min < 30% for 5 min Investigate cache invalidation
DB Replication Lag > 5s for 5 min > 30s for 2 min Failover to replica
Kafka Consumer Lag > 10K messages > 100K messages Scale consumers
CDN Error Rate > 0.1% for 5 min > 1% for 2 min Switch to backup CDN

Appendix A: Technology Stack Summary

Layer Technology Alternative
Frontend React Native (mobile), React (web) Flutter, Swift/Kotlin native
API Gateway Kong / Custom Go gateway AWS API Gateway, Envoy
Backend Services Go (performance), Java (complexity) Python, Rust
Message Queue Apache Kafka RabbitMQ, AWS SQS
Relational DB MySQL (Vitess sharding) PostgreSQL (Citus)
NoSQL DB Cassandra (graph), MongoDB (notifications) DynamoDB, ScyllaDB
Cache Redis Cluster Memcached, Dragonfly
Search Elasticsearch OpenSearch, Meilisearch
Object Storage AWS S3 GCS, Azure Blob, MinIO
CDN CloudFront + Akamai (multi-CDN) Fastly, Cloudflare
Container Orchestration Kubernetes ECS, Nomad
Service Mesh Istio Linkerd, Consul
Monitoring Prometheus + Grafana + ELK Datadog, New Relic
CI/CD GitLab CI / GitHub Actions Jenkins, ArgoCD
Infrastructure as Code Terraform Pulumi, CloudFormation

Appendix B: Cost Estimates at Scale

Component Monthly Cost (500M MAU) Notes
CDN Egress $15-25M 300 EB/month at $0.05/GB
Object Storage $5-8M 2.4 EB at $0.023/GB
Compute (K8s) $3-5M ~10,000 instances
Databases $2-3M Managed DB instances
Cache (Redis) $0.5-1M Large Redis clusters
Search (ES) $0.5-1M Large ES clusters
ML Inference $1-2M GPU instances for ranking
Monitoring/Logging $0.5-1M Data ingestion and storage
Total $27.5-46M/month ~$330-550M/year

Cost Optimization Levers:

  • Image compression (WebP/AVIF) → 30-50% bandwidth savings
  • CDN multi-provider negotiation → 20-30% cost reduction
  • Tiered storage (hot/warm/cold) → 40% storage savings
  • Spot instances for batch processing → 60-70% compute savings
  • Data lifecycle policies → Auto-delete old logs, compress old data

Appendix C: Security Considerations

Threat Mitigation
Account Takeover MFA, device fingerprinting, anomaly detection
DDoS WAF, rate limiting, CDN absorption, auto-scaling
Data Breach Encryption at rest (AES-256), in transit (TLS 1.3), key rotation
Content Abuse AI moderation + human review + user reporting
CSRF/XSS SameSite cookies, CSP headers, input sanitization
API Abuse Rate limiting per user/IP, API key rotation, anomaly detection
Insider Threat RBAC, audit logging, least privilege access
GDPR Compliance Data export/delete APIs, consent management, DPA
Child Safety Age verification, content filtering, reporting mechanisms

Summary

This design document covers a production-ready, billion-user-scale social media platform inspired by Instagram. The key architectural principles are:

  1. Read-optimized by default — Feed is the most frequent operation; cache aggressively at every layer
  2. Hybrid feed model — Push for normal users, pull for celebrities; avoids the fan-out explosion
  3. Polyglot persistence — Each data type uses the optimal storage engine
  4. Event-driven architecture — Kafka decouples services and enables async processing
  5. Graceful degradation — Circuit breakers, bulkheads, and priority-based fallbacks
  6. Horizontal scalability — Every component scales linearly with sharding and stateless design
  7. Multi-layer caching — CDN → Redis → DB replicas → Primary DB

The system is designed to handle 500M+ MAU with 99.95% availability, sub-200ms feed latency, and linear cost scaling. Each curveball scenario demonstrates how the architecture adapts to new requirements without fundamental re-architecture.


Document Version: 1.0 | Last Updated: 2026-08-28