E-commerceQuestion #9ADVANCED

Amazon: Catalog, Search, Cart, Orders & Inventory

End-to-end e-commerce platform architecture covering multi-warehouse inventory reservation, tiered product search, distributed shopping cart, and asynchronous order pipelines.

Category: Category 6: E-commerce•Updated: 2026-08-30
#E-commerce#Catalog#Search#Cart#Orders#Inventory#Recommendations

Design an E-commerce App like Amazon

Author: System Design Expert | Date: 2026-08-28 Scope: Multi-seller marketplace with product catalog, search, cart, orders, inventory, recommendations, and payments.


Table of Contents

  1. Requirements Clarification
  2. Back-of-Envelope Calculations
  3. System Interface Design
  4. High-Level Architecture
  5. Core Component Design
  6. Data Model
  7. Key Design Decisions
  8. Scalability Strategy
  9. Curveball Scenarios
  10. Failure Modes and Mitigations

1. Requirements Clarification

1.1 Functional Requirements

ID Requirement Priority
FR-01 User Management: Registration, login, profile, address book, order history P0
FR-02 Product Catalog: Browse, search, filter, sort, category navigation, product detail pages P0
FR-03 Search: Full-text search with autocomplete, faceted filtering, relevance ranking P0
FR-04 Shopping Cart: Add/remove items, quantity update, persist across sessions P0
FR-05 Checkout & Orders: Multi-step checkout, payment processing, order confirmation, tracking P0
FR-06 Inventory Management: Real-time stock tracking, reserve on cart/checkout, prevent overselling P0
FR-07 Multi-Seller Marketplace: Third-party sellers can list products, manage inventory, fulfill orders P1
FR-08 Recommendations: Personalized product recommendations (homepage, detail page, post-purchase) P1
FR-09 Reviews & Ratings: Users can write reviews, rate products, upload images P1
FR-10 Wishlist: Save items for later, share wishlists P2
FR-11 Notifications: Order status updates, price drops, back-in-stock alerts via email/push/SMS P1
FR-12 Returns & Refunds: Initiate return, RMA generation, refund processing P1
FR-13 Pricing & Promotions: Dynamic pricing, coupons, flash sales, bundle deals P1

1.2 Non-Functional Requirements

ID Requirement Target
NFR-01 Availability 99.99% uptime for core paths (browse, search, checkout)
NFR-02 Latency P95 < 200ms for catalog browse, < 500ms for search, < 2s for checkout
NFR-03 Throughput Support 10K+ QPS steady-state, 100K+ QPS peak (Prime Day / Black Friday)
NFR-04 Consistency Strong consistency for inventory deduction; eventual consistency for catalog/reviews
NFR-05 Durability Zero data loss for orders and payments (RPO = 0)
NFR-06 Scalability Horizontal scaling for all stateless services; sharded stateful services
NFR-07 Security PCI-DSS compliance for payments, GDPR/CCPA for PII, OWASP Top 10 protections
NFR-08 Internationalization Multi-currency, multi-language, multi-region deployment
NFR-09 Observability Distributed tracing, structured logging, real-time dashboards, alerting

2. Back-of-Envelope Calculations

2.1 Assumption Sets

We reason from first principles with three assumption sets to bound the design:

Parameter Conservative (A) Base (B) Aggressive (C)
Monthly Active Users (MAU) 100M 300M 500M
Daily Active Users (DAU) 20M 60M 100M
Avg. page views / session 8 12 15
Sessions / DAU 1.5 2 2.5
Avg. orders / DAU 0.05 0.08 0.10
Products in catalog 10M 50M 100M
Avg. product image size 100 KB 150 KB 200 KB
Images per product 5 8 10
Peak-to-steady ratio 5x 10x 15x

2.2 Traffic Estimates (Base Case B)

Daily page views = DAU × sessions/DAU × pages/session
                 = 60M × 2 × 12 = 1,440M page views/day

Daily orders = DAU × orders/DAU = 60M × 0.08 = 4.8M orders/day

Steady-state QPS (read-heavy):
  Total requests/day ≈ page views × 5 API calls/page = 7,200M
  Steady QPS = 7,200M / 86,400s ≈ 83,333 QPS

Peak QPS = 83,333 × 10 = 833,333 QPS

Read/Write Ratio: ~100:1 (browse/search dominate; checkout is a small fraction)

2.3 Storage Estimates

Component Calculation (Base) Size
Product metadata 50M products × 5 KB ~250 GB
Product images 50M × 8 images × 150 KB ~60 TB
Orders 4.8M/day × 365 × 1 KB ~1.75 TB/year
Order items 4.8M/day × 365 × 2 items × 500 B ~1.75 TB/year
User profiles 300M × 2 KB ~600 MB
Reviews 500M/year × 2 KB ~1 GB/year
Search index 50M docs × 10 KB ~500 GB
Total (year 1) ~65 TB

2.4 Bandwidth Estimates

Outbound bandwidth/day:
  Page views × 500 KB/page (HTML + images + JS) = 1,440M × 500 KB = 720 TB/day
  With CDN caching (80% hit rate): 720 TB × 0.2 = 144 TB/day origin egress

  Average: 144 TB / 86,400s ≈ 1.7 GB/s sustained origin egress
  Peak: ~17 GB/s

3. System Interface Design

3.1 API Definitions

User Service

POST   /api/v1/users/register          — Register new user
POST   /api/v1/users/login               — Authenticate, return JWT
GET    /api/v1/users/{userId}              — Get user profile
PUT    /api/v1/users/{userId}/profile      — Update profile
GET    /api/v1/users/{userId}/addresses    — List addresses
POST   /api/v1/users/{userId}/addresses    — Add address

Catalog Service

GET    /api/v1/categories                  — List category tree
GET    /api/v1/categories/{id}/products         — Browse products in category
GET    /api/v1/products/{productId}             — Product detail
GET    /api/v1/products/{productId}/reviews      — Product reviews
POST   /api/v1/products/{productId}/reviews      — Submit review
GET    /api/v1/products?sellerId={id}          — Seller's product listing

Search Service

GET    /api/v1/search?q={query}&category={cat}&page={n}&sort={field}
GET    /api/v1/search/suggest?q={prefix}          — Autocomplete suggestions

Cart Service

GET    /api/v1/cart/{userId}                      — Get cart
POST   /api/v1/cart/{userId}/items                — Add item to cart
PUT    /api/v1/cart/{userId}/items/{itemId}        — Update quantity
DELETE /api/v1/cart/{userId}/items/{itemId}        — Remove item
POST   /api/v1/cart/{userId}/merge                — Merge guest cart

Order Service

POST   /api/v1/orders                              — Create order (checkout)
GET    /api/v1/orders/{orderId}                     — Order detail
GET    /api/v1/users/{userId}/orders                — User order history
POST   /api/v1/orders/{orderId}/cancel              — Cancel order
POST   /api/v1/orders/{orderId}/return                — Initiate return
GET    /api/v1/orders/{orderId}/tracking              — Shipment tracking

Inventory Service

POST   /api/v1/inventory/{sku}/reserve              — Reserve stock (checkout)
POST   /api/v1/inventory/{sku}/commit               — Commit reservation
POST   /api/v1/inventory/{sku}/release               — Release reservation
GET    /api/v1/inventory/{sku}                       — Stock level
PUT    /api/v1/inventory/{sku}/adjust                — Seller stock adjustment

Payment Service

POST   /api/v1/payments/intent                      — Create payment intent
POST   /api/v1/payments/{intentId}/confirm              — Confirm payment
GET    /api/v1/payments/{intentId}                       — Payment status
POST   /api/v1/payments/{intentId}/refund                — Refund

Recommendation Service

GET    /api/v1/recommendations/home?userId={id}          — Homepage recs
GET    /api/v1/recommendations/detail?productId={id}     — Detail page recs
GET    /api/v1/recommendations/cross-sell?orderId={id}   — Post-purchase recs

3.2 Internal gRPC Interfaces

// Inventory gRPC
service InventoryService {
  rpc Reserve(ReserveRequest) returns (ReserveResponse);
  rpc Commit(CommitRequest) returns (CommitResponse);
  rpc Release(ReleaseRequest) returns (ReleaseResponse);
}

// Order gRPC
service OrderService {
  rpc CreateOrder(CreateOrderRequest) returns (CreateOrderResponse);
  rpc UpdateStatus(UpdateStatusRequest) returns (UpdateStatusResponse);
}

// Notification gRPC
service NotificationService {
  rpc Send(SendNotificationRequest) returns (SendNotificationResponse);
}

4. High-Level Architecture

flowchart TB
    subgraph Client["Client Layer"]
        Web[Web App]
        Mobile[Mobile App]
        Seller[Seller Portal]
    end

    subgraph Edge["Edge / CDN Layer"]
        CDN[CDN<br/>(Static Assets + Images)]
        WAF[WAF / DDoS Protection]
        GSLB[Global Load Balancer]
    end

    subgraph APIGateway["API Gateway Layer"]
        GW[API Gateway<br/>(Rate Limiting, Auth, Routing)]
    end

    subgraph Services["Microservices Layer"]
        subgraph UserSvc["User Service"]
        end
        subgraph CatalogSvc["Catalog Service"]
        end
        subgraph SearchSvc["Search Service"]
        end
        subgraph CartSvc["Cart Service"]
        end
        subgraph OrderSvc["Order Service"]
        end
        subgraph InvSvc["Inventory Service"]
        end
        subgraph PaySvc["Payment Service"]
        end
        subgraph RecSvc["Recommendation Service"]
        end
        subgraph ReviewSvc["Review Service"]
        end
        subgraph NotifSvc["Notification Service"]
        end
        subgraph SellerSvc["Seller Service"]
        end
    end

    subgraph Messaging["Messaging Layer"]
        K[Apache Kafka<br/>(Event Bus)]
    end

    subgraph Data["Data Layer"]
        subgraph Databases["Databases"]
            MySQL[(MySQL<br/>Orders, Users)]
            Dynamo[(DynamoDB<br/>Cart, Sessions)]
            Mongo[(MongoDB<br/>Catalog)]
        end
        subgraph Cache["Cache"]
            Redis[Redis Cluster<br/>(Cart, Sessions, Hot Data)]
        end
        subgraph SearchStore["Search"]
            ES[Elasticsearch<br/>Cluster]
        end
        subgraph Storage["Object Storage"]
            S3[S3 / OSS<br/>(Images, Logs)]
        end
        subgraph MLStore["ML Storage"]
            DataLake[Data Lake<br/>(Clickstream, Features)]
        end
    end

    subgraph Infra["Infrastructure"]
        Monitor[Monitoring & Alerting]
        Trace[Distributed Tracing]
        CI[CI/CD Pipeline]
    end

    Web --> CDN
    Mobile --> CDN
    Seller --> CDN
    CDN --> WAF
    WAF --> GSLB
    GSLB --> GW

    GW --> UserSvc
    GW --> CatalogSvc
    GW --> SearchSvc
    GW --> CartSvc
    GW --> OrderSvc
    GW --> InvSvc
    GW --> PaySvc
    GW --> RecSvc
    GW --> ReviewSvc
    GW --> SellerSvc

    UserSvc --> K
    CatalogSvc --> K
    OrderSvc --> K
    InvSvc --> K
    PaySvc --> K

    K --> NotifSvc
    K --> RecSvc
    K --> SearchSvc

    UserSvc --> MySQL
    CatalogSvc --> Mongo
    CatalogSvc --> Redis
    SearchSvc --> ES
    CartSvc --> Redis
    CartSvc --> Dynamo
    OrderSvc --> MySQL
    InvSvc --> MySQL
    PaySvc --> MySQL
    ReviewSvc --> Mongo
    RecSvc --> DataLake
    SellerSvc --> MySQL

    CatalogSvc --> S3
    ReviewSvc --> S3

5. Core Component Design

5.1 Product Catalog Service

Purpose: Authoritative source for product metadata, categories, and pricing.

Design:

  • Read path: API Gateway → Catalog Service → Redis Cache → MongoDB (if cache miss)
  • Write path: Seller Portal → Catalog Service → MongoDB → Kafka → Elasticsearch (async reindex)
  • Caching strategy:
    • Hot products (top 10% by views) cached in Redis with TTL = 5 min
    • Category tree cached with TTL = 30 min
    • Cache-aside pattern with dog-parking (lock-based) to prevent cache stampede
  • Consistency: Eventual consistency between MongoDB (source of truth) and Elasticsearch (search index) via Kafka change-data-capture pipeline
  • Partitioning: Products sharded by categoryId hash; each shard ~500K products

Key Challenge — Pricing:

  • Prices can change dynamically (promotions, seller updates, dynamic pricing)
  • Price evaluated at checkout time, not browse time
  • Browse shows “from” price; checkout computes final price via Pricing Engine

5.2 Search Service

Purpose: Full-text search with relevance ranking, faceting, and autocomplete.

Architecture:

User Query → API Gateway → Search Service → Elasticsearch Cluster
                                        → Result Cache (Redis)
                                        → Query Log → Kafka → ML Pipeline

Elasticsearch Design:

  • Index structure:
    • Primary index: products_v{version} with aliases for zero-downtime reindexing
    • Fields: title (text, analyzed), description (text), brand (keyword), category (keyword), price (float), rating (float), sellerId (keyword), inStock (boolean)
    • Custom analyzer with synonym dictionary, stop words, and n-gram tokenizer for fuzzy matching
  • Relevance ranking: BM25 + learning-to-rank (LTR) model incorporating:
    • Text relevance (BM25 score)
    • Popularity (recent sales velocity)
    • Rating and review count
    • Seller performance score
    • Personalization signals (user history)
  • Autocomplete: Separate suggest index with edge n-gram completion suggester; updated in near-real-time via Kafka
  • Sharding: 20 primary shards, 1 replica each; auto-scaling based on query load

Query Flow:

  1. Check Redis for exact query cache hit (TTL = 1 min for top queries)
  2. If miss, query Elasticsearch with faceted aggregation
  3. Apply LTR model scoring (pre-computed features joined at query time)
  4. Cache result and log query for offline model training

5.3 Shopping Cart Service

Purpose: Manage user shopping carts with persistence and cross-device sync.

Design Decisions:

  • Storage: Redis (primary, for speed) + DynamoDB (backup, for durability)
  • Guest carts: Stored with cookie-based session ID; merged on login
  • Cart item structure:
    {
      "cartId": "uuid",
      "userId": "user-123",
      "items": [
        {
          "sku": "B08N5WRWNW",
          "quantity": 2,
          "priceAtAdd": 29.99,
          "sellerId": "seller-456",
          "addedAt": "2026-08-28T10:30:00Z"
        }
      ],
      "expiresAt": "2026-09-28T10:30:00Z"
    }
  • TTL: 30 days; expired carts cleaned by background job
  • Price validation: At checkout, re-validate prices from Pricing Engine (prices may have changed since add-to-cart)
  • Concurrency: Redis Lua scripts for atomic add/update/remove operations

5.4 Order Service

Purpose: Orchestrate the checkout flow and manage order lifecycle.

Checkout Flow (saga pattern):

sequenceDiagram
    participant C as Client
    participant GW as API Gateway
    participant OS as Order Service
    participant IS as Inventory Service
    participant PS as Payment Service
    participant NS as Notification Service
    participant K as Kafka

    C->>GW: POST /orders (checkout)
    GW->>OS: CreateOrder
    OS->>OS: Create order (status=PENDING)
    OS->>IS: Reserve inventory
    alt Inventory insufficient
        IS-->>OS: ReserveFailed
        OS->>OS: Set status=FAILED
        OS-->>C: Error: Out of stock
    else Inventory reserved
        IS-->>OS: Reserved
        OS->>PS: Create payment intent
        PS-->>OS: PaymentIntentId
        OS-->>C: PaymentIntentId
        C->>PS: Confirm payment
        PS->>PS: Process payment
        alt Payment failed
            PS-->>OS: PaymentFailed
            OS->>IS: Release inventory
            OS->>OS: Set status=CANCELLED
            OS-->>C: Payment failed
        else Payment succeeded
            PS-->>OS: PaymentConfirmed
            OS->>IS: Commit reservation
            OS->>OS: Set status=CONFIRMED
            OS->>K: OrderCreated event
            K->>NS: Send confirmation
            K->>NS: Update seller
            OS-->>C: Order confirmed
        end
    end

Order States:

PENDING → (inventory reserved) → AWAITING_PAYMENT → (payment confirmed) → CONFIRMED
CONFIRMED → PROCESSING → SHIPPED → DELIVERED
Any state → CANCELLED (with conditions)
DELIVERED → RETURN_REQUESTED → RETURNED → REFUNDED

Idempotency: All order creation requests carry an idempotency-key header; duplicate requests return the original order.

5.5 Inventory Service

Purpose: Prevent overselling with accurate stock management.

Design:

  • Stock model:
    Available = Physical Stock - Reserved - Committed
  • Reservation lifecycle:
    1. Reserve: At checkout, reserve stock for 15 minutes (TTL-based)
    2. Commit: On payment confirmation, convert reservation to committed
    3. Release: On payment failure or timeout, release reservation back to available
  • Concurrency control:
    • Redis-based atomic decrement with Lua script for reservation
    • MySQL for persistent stock records with optimistic locking (version column)
    • Over-reservation tolerance: Allow 0.1% over-reserve as circuit breaker; handle via backorder
  • Multi-warehouse: Stock tracked per warehouse; allocation algorithm considers proximity to delivery address

Stock Table:

CREATE TABLE stock (
    sku VARCHAR(20) PRIMARY KEY,
    warehouse_id VARCHAR(10),
    physical_stock INT NOT NULL,
    reserved_stock INT NOT NULL DEFAULT 0,
    committed_stock INT NOT NULL DEFAULT 0,
    version BIGINT NOT NULL DEFAULT 1,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

5.6 Recommendation Service

Purpose: Personalized product recommendations using ML models.

Architecture:

Clickstream → Kafka → Flink (real-time features) → Feature Store
                                        → Data Lake (batch)
                                                    → Spark (offline training)
                                                        → Model Registry
                                                            → Model Serving (real-time inference)

Recommendation Strategies:

Strategy Use Case Latency Data
Collaborative Filtering “Customers who bought this also bought” < 100ms Historical purchases
Content-Based “Similar to items you viewed” < 50ms Product features
Real-Time Session Homepage, current session < 20ms Live clickstream
Trending “Best Sellers”, “New Arrivals” Pre-computed Sales velocity
Cross-Sell Post-purchase emails Batch Purchase patterns

Feature Store:

  • Real-time features: Recent clicks, cart additions, session duration (Flink → Redis)
  • Offline features: User lifetime value, category affinity, brand preference (Spark → MySQL)

5.7 Payment Service

Purpose: Secure payment processing with multiple payment methods.

Design:

  • PCI-DSS compliance: Payment data never touches application servers; tokenized via payment gateway
  • Supported methods: Credit/debit cards, digital wallets, bank transfer, BNPL
  • Payment flow:
    1. Create payment intent (returns client secret)
    2. Client SDK handles card input (hosted fields / redirect)
    3. Confirm payment with server-side verification
    4. Webhook for async payment status updates
  • Idempotency: Payment confirmations are idempotent via payment_intent_id
  • Reconciliation: Daily batch reconciliation with payment gateway; discrepancies flagged for manual review

5.8 Notification Service

Purpose: Multi-channel notifications (email, SMS, push, in-app).

Design:

  • Event-driven: Subscribes to Kafka topics (order.created, order.shipped, price.drop, etc.)
  • Template engine: Personalized templates with user locale support
  • Channel selection: User preference → fallback chain (push → email → SMS)
  • Rate limiting: Per-user throttling to prevent notification spam
  • Retry policy: Exponential backoff (1m, 5m, 15m, 1h, 6h) with dead-letter queue

6. Data Model

6.1 Relational Schema (MySQL — Orders, Users, Inventory)

-- Users
CREATE TABLE users (
    user_id VARCHAR(36) PRIMARY KEY,
    email VARCHAR(255) UNIQUE NOT NULL,
    password_hash VARCHAR(255) NOT NULL,
    display_name VARCHAR(100),
    phone VARCHAR(20),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    INDEX idx_email (email)
);

-- Addresses
CREATE TABLE addresses (
    address_id VARCHAR(36) PRIMARY KEY,
    user_id VARCHAR(36) NOT NULL,
    line1 VARCHAR(255) NOT NULL,
    line2 VARCHAR(255),
    city VARCHAR(100) NOT NULL,
    state VARCHAR(100),
    postal_code VARCHAR(20) NOT NULL,
    country VARCHAR(2) NOT NULL,
    is_default BOOLEAN DEFAULT FALSE,
    FOREIGN KEY (user_id) REFERENCES users(user_id)
);

-- Orders
CREATE TABLE orders (
    order_id VARCHAR(36) PRIMARY KEY,
    user_id VARCHAR(36) NOT NULL,
    status VARCHAR(20) NOT NULL DEFAULT 'PENDING',
    total_amount DECIMAL(12,2) NOT NULL,
    currency VARCHAR(3) NOT NULL DEFAULT 'USD',
    shipping_address_id VARCHAR(36),
    idempotency_key VARCHAR(64) UNIQUE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    FOREIGN KEY (user_id) REFERENCES users(user_id),
    FOREIGN KEY (shipping_address_id) REFERENCES addresses(address_id),
    INDEX idx_user_status (user_id, status),
    INDEX idx_created (created_at)
);

-- Order Items
CREATE TABLE order_items (
    item_id VARCHAR(36) PRIMARY KEY,
    order_id VARCHAR(36) NOT NULL,
    sku VARCHAR(20) NOT NULL,
    product_name VARCHAR(255) NOT NULL,
    quantity INT NOT NULL,
    unit_price DECIMAL(10,2) NOT NULL,
    seller_id VARCHAR(36) NOT NULL,
    FOREIGN KEY (order_id) REFERENCES orders(order_id),
    INDEX idx_order (order_id),
    INDEX idx_sku (sku)
);

-- Payments
CREATE TABLE payments (
    payment_id VARCHAR(36) PRIMARY KEY,
    order_id VARCHAR(36) NOT NULL UNIQUE,
    payment_intent_id VARCHAR(100) NOT NULL UNIQUE,
    amount DECIMAL(12,2) NOT NULL,
    currency VARCHAR(3) NOT NULL,
    status VARCHAR(20) NOT NULL,
    method VARCHAR(20) NOT NULL,
    gateway_response JSON,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (order_id) REFERENCES orders(order_id)
);

-- Inventory / Stock
CREATE TABLE stock (
    sku VARCHAR(20) NOT NULL,
    warehouse_id VARCHAR(10) NOT NULL,
    physical_stock INT NOT NULL DEFAULT 0,
    reserved_stock INT NOT NULL DEFAULT 0,
    committed_stock INT NOT NULL DEFAULT 0,
    version BIGINT NOT NULL DEFAULT 1,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (sku, warehouse_id)
);

-- Reservations
CREATE TABLE reservations (
    reservation_id VARCHAR(36) PRIMARY KEY,
    sku VARCHAR(20) NOT NULL,
    warehouse_id VARCHAR(10) NOT NULL,
    order_id VARCHAR(36) NOT NULL,
    quantity INT NOT NULL,
    status VARCHAR(20) NOT NULL DEFAULT 'ACTIVE',
    expires_at TIMESTAMP NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    INDEX idx_sku (sku, warehouse_id),
    INDEX idx_expires (expires_at),
    INDEX idx_order (order_id)
);

6.2 NoSQL Schema (MongoDB — Catalog, Reviews)

// Product Document
{
  "_id": "prod-001",
  "sku": "B08N5WRWNW",
  "title": "Wireless Noise-Cancelling Headphones",
  "description": "Premium over-ear headphones with ANC...",
  "categoryId": "cat-electronics-audio",
  "categoryPath": ["Electronics", "Audio", "Headphones"],
  "brand": "SoundMax",
  "sellerId": "seller-456",
  "pricing": {
    "listPrice": 299.99,
    "salePrice": 249.99,
    "currency": "USD",
    "effectiveFrom": ISODate("2026-08-01"),
    "effectiveTo": ISODate("2026-08-31")
  },
  "attributes": {
    "color": "Black",
    "weight": "250g",
    "batteryLife": "30 hours",
    "connectivity": "Bluetooth 5.2"
  },
  "images": [
    { "url": "https://cdn.example.com/prod-001/img1.jpg", "role": "primary" },
    { "url": "https://cdn.example.com/prod-001/img2.jpg", "role": "alternate" }
  ],
  "rating": { "average": 4.5, "count": 12847 },
  "status": "ACTIVE",
  "createdAt": ISODate("2026-01-15T10:00:00Z"),
  "updatedAt": ISODate("2026-08-28T08:00:00Z")
}

// Review Document
{
  "_id": "review-789",
  "productId": "prod-001",
  "userId": "user-123",
  "orderId": "order-456",
  "rating": 5,
  "title": "Best headphones I've ever owned",
  "body": "The noise cancellation is incredible...",
  "images": ["https://cdn.example.com/reviews/rev-789/img1.jpg"],
  "verifiedPurchase": true,
  "helpfulCount": 42,
  "createdAt": ISODate("2026-07-20T14:30:00Z")
}

6.3 Redis Data Structures

# Cart
HSET cart:{userId} items '[{"sku":"B08N","qty":2,"price":29.99}]'
EXPIRE cart:{userId} 2592000  # 30 days

# Product cache
SET product:{productId} '{json}' EX 300  # 5 min TTL

# Category cache
SET category:{categoryId} '{json}' EX 1800  # 30 min

# Search query cache
SET search:{md5(query)} '{results}' EX 60

# Rate limiting
INCR rate:{userId}:{endpoint}
EXPIRE rate:{userId}:{endpoint} 60

# Inventory reservation
SET reserve:{sku}:{orderId} '{qty}' EX 900  # 15 min TTL

7. Key Design Decisions

7.1 Trade-off Analysis

Decision Option A Option B Choice Rationale
Catalog Storage Relational DB Document DB (MongoDB) MongoDB Products have flexible, nested attributes that vary by category. Schema flexibility wins.
Cart Storage MySQL Redis + DynamoDB Redis + DynamoDB Cart is high-frequency read/write with short TTL. Redis gives sub-ms latency; DynamoDB provides durability.
Search Engine MySQL FULLTEXT Elasticsearch Elasticsearch Required for faceting, relevance ranking, fuzzy search, and autocomplete. MySQL FULLTEXT is insufficient.
Order Consistency Eventual (async) Strong (saga) Saga pattern Orders and payments require strong consistency. Saga provides distributed transactions without 2PC.
Inventory Deduction DB row lock Redis atomic + DB sync Redis atomic + DB sync Row locks don’t scale at 100K QPS peak. Redis Lua scripts provide atomic operations at scale.
Recommendations Real-time ML Pre-computed + real-time blend Hybrid Pure real-time is too slow; pure pre-computed lacks freshness. Blend gives best of both.
API Style GraphQL REST + gRPC REST (external) + gRPC (internal) REST for external clients (caching, simplicity); gRPC for internal (performance, strong typing).
Payment Processing Synchronous Async with webhooks Async with webhooks Payment gateways are async by nature. Webhooks handle delayed confirmations and retries.

7.2 Multi-Seller Marketplace Considerations

  • Seller isolation: Each seller has isolated inventory, pricing, and fulfillment
  • Commission model: Platform takes percentage per transaction; calculated at order confirmation
  • Seller ratings: Impact search ranking and buyer trust
  • Fulfillment options: FBA (Fulfilled by Amazon/platform) vs. FBM (Fulfilled by Merchant)

8. Scalability Strategy

8.1 Horizontal Scaling

Component Scaling Strategy Trigger
API Gateway Auto-scaling group (K8s HPA) CPU > 70%, RPS threshold
Stateless Services K8s HPA per service CPU > 60%, memory > 70%
Redis Cluster Add shards (resharding) Memory > 75%, hit rate < 95%
Elasticsearch Add data nodes Index size > 50 GB/shard, query latency > 200ms
MySQL Read replicas + sharding Read latency > 10ms, connection pool > 80%
Kafka Add partitions + brokers Consumer lag > 10K messages

8.2 Database Sharding Strategy

Orders: Shard by userId hash (128 shards)
  - Each shard ~4M users
  - Cross-shard queries via aggregation service

Inventory: Shard by SKU hash (64 shards)
  - Hot SKUs get dedicated shards
  - Cross-shard inventory checks via read replicas

Catalog: Shard by categoryId (32 shards)
  - Category-aligned access patterns
  - Elasticsearch handles cross-shard search

8.3 Caching Hierarchy

L1: In-process cache (Caffeine/Guava) — 10K entries, 10s TTL
L2: Redis Cluster — Hot data, carts, sessions
L3: CDN — Static assets, images, product pages (SSR cached)

8.4 Multi-Region Deployment

Primary Region (US-East):
  - Full read-write for US users
  - Active database primary

Secondary Regions (EU-West, AP-South):
  - Read replicas for local latency
  - Async replication from primary
  - Failover capable (RTO < 5 min, RPO < 1 min)

8.5 Peak Traffic Handling (Prime Day / Black Friday)

  • Pre-warming: Cache hot products 24h before event
  • Queue-based load leveling: Checkout requests queued in Kafka; processed at sustainable rate
  • Degradation modes:
    • Disable recommendations during extreme peak
    • Serve stale search results (extend cache TTL)
    • Queue non-critical writes (reviews, wishlist)
  • Auto-scaling pre-provisioning: Scale to 3x capacity 1h before event start

9. Curveball Scenarios

Curveball 1: “A flash sale causes 50x normal traffic in 30 seconds. How do you handle it?”

Analysis:

  • A flash sale on a popular product creates a thundering herd on a single SKU
  • Inventory for that SKU gets to zero in milliseconds
  • The bottleneck is not overall system capacity but hot-key contention

Response:

  1. Pre-sale preparation:

    • Pre-reserve inventory in Redis with atomic counter for the flash SKU
    • Pre-warm all caches for the product page
    • CDN pre-populate product pages
  2. During the sale:

    • Atomic stock counter in Redis: Single Redis key with Lua script for atomic decrement
      local stock = redis.call('GET', KEYS[1])
      if tonumber(stock) > 0 then
          redis.call('DECR', KEYS[1])
          return 1  -- Success
      end
      return 0  -- Sold out
    • Sold-out fast path: Once stock hits 0, serve cached “sold out” page directly from CDN (no backend call)
    • Queue-based checkout: Even if user “wins” the stock, checkout goes through a queue to prevent payment system overload
    • Rate limiting per user: Max 1 flash sale item per user; enforced at API Gateway
  3. Post-sale:

    • Async reconciliation: Compare Redis deductions with MySQL records
    • Handle any over-sales via automated refund + apology coupon

Key Insight: The goal is not to process all requests but to fail fast for the majority (sold out) while correctly handling the winners.

Curveball 2: “A seller accidentally lists a $2,000 laptop for $20. 10,000 orders are placed in 5 minutes. What do you do?”

Analysis:

  • This is a pricing error, not a system bug
  • Legal and business implications: Honoring vs. canceling orders
  • System must support rapid intervention

Response:

  1. Detection:

    • Anomaly detection: Real-time pricing monitor flags prices > 3σ from historical mean
    • Velocity alert: Unusual order velocity for a SKU triggers PagerDuty
    • Seller self-service: Seller can emergency-pull a listing
  2. Immediate containment:

    • Emergency listing freeze: Admin API to suspend a product listing within seconds
    • Inventory lock: Set available stock to 0 for the affected SKU
    • Checkout circuit breaker: Orders for frozen SKUs fail at checkout with “temporarily unavailable”
  3. Order handling:

    • Orders in PENDING / AWAITING_PAYMENT: Auto-cancel with notification
    • Orders in CONFIRMED: Business decision — honor or cancel with compensation
    • System supports bulk order status update for this scenario
  4. Prevention:

    • Price change guardrails: Sellers must confirm price changes > 50% from current price
    • Rate limiting on price updates: Max N price changes per hour per seller
    • Approval workflow: Large price changes require admin approval

Curveball 3: “The primary database region goes down. How do you maintain service?”

Analysis:

  • Regional outage affects all write operations
  • Read replicas in other regions may serve stale data
  • RTO and RPO requirements drive the design

Response:

  1. Detection (target: < 30 seconds):

    • Health checks on database primary (multi-probe from multiple AZs)
    • Cloud provider region health API integration
    • Automated failover trigger
  2. Failover process:

    • Phase 1 — Read-only mode (0-2 min):
      • Switch all reads to secondary region replicas
      • Writes queue in Kafka with persistent storage
      • Users can browse, search, view cart but cannot checkout
    • Phase 2 — Promote secondary (2-5 min):
      • Promote secondary region database to primary
      • Update DNS / service discovery
      • Resume write operations
    • Phase 3 — Drain queue (5-15 min):
      • Process queued writes from Kafka
      • Reconcile any conflicts
  3. Architecture enablers:

    • Multi-region active-active for stateless services (always running)
    • Active-passive for databases (async replication, automatic promotion)
    • Write-ahead log in Kafka ensures no write loss during failover
    • Circuit breakers prevent cascading failures
  4. RPO/RTO:

    • RPO: < 1 minute (async replication lag)
    • RTO: < 5 minutes (automated failover)

Curveball 4: “How do you prevent a bot from buying all available stock of a limited-release product?”

Response:

  1. Bot detection layers:

    • CAPTCHA / reCAPTCHA: Triggered on suspicious behavior patterns
    • Device fingerprinting: Browser/device hash to identify repeat offenders
    • Behavioral analysis: Mouse movement, typing patterns, navigation speed
    • Rate limiting: Per-IP, per-account, per-device limits on add-to-cart and checkout
  2. Queue-based fair access:

    • Virtual waiting room for high-demand products
    • Each user gets one fair shot via randomized queue position
    • Prevents refresh-based advantage
  3. Purchase limits:

    • Max 1 unit per user for limited-release items
    • Account age and purchase history requirements
    • Verified phone/email required for checkout
  4. Post-purchase analysis:

    • Flag accounts with bot-like patterns for review
    • Cancel suspicious orders before fulfillment

Curveball 5: “A third-party seller is fraudulently shipping counterfeit products. How does the system detect and respond?”

Response:

  1. Detection signals:

    • Spike in return rates for a seller’s products
    • Pattern of negative reviews mentioning “fake” / “counterfeit”
    • NLP analysis of review text for fraud indicators
    • Buyer reports via “Report counterfeit” button
  2. Automated response:

    • Seller score degradation: Automatic reduction in seller rating
    • Listing review: Flagged products go to manual review queue
    • Escalation: High-confidence fraud triggers seller suspension
  3. Prevention:

    • Brand registry program (brands verify authorized sellers)
    • Seller onboarding with identity verification
    • Transaction monitoring for unusual patterns

10. Failure Modes and Mitigations

10.1 Failure Mode Matrix

Component Failure Mode Impact Detection Mitigation RTO
API Gateway Process crash All requests fail Health check timeout Auto-restart, multi-AZ deployment < 30s
API Gateway Memory leak Gradual degradation Memory metric alert Rolling restart, memory limits < 2min
Catalog Service MongoDB primary down Product reads fail Replica lag alert Auto-failover to replica < 1min
Catalog Service Cache stampede DB overload, latency spike Latency P99 alert Dog-parking, cache warming Auto
Search Service ES cluster split-brain Inconsistent search results Cluster health RED Cluster rebootstrap, quorum config < 5min
Search Service Index corruption Wrong/missing results Search quality metrics Index rebuild from MongoDB < 30min
Cart Service Redis node failure Cart data loss for shard Node down alert Redis Cluster replication, rehash < 1min
Cart Service Redis memory full New carts fail Memory > 90% alert Maxmemory policy (allkeys-lru), scale Auto
Order Service MySQL primary down Cannot create orders Connection pool exhaustion Read replica promotion < 2min
Order Service Duplicate order creation Double charging Idempotency key collision Idempotency key enforcement N/A
Inventory Service Overselling Customer dissatisfaction Stock < 0 detection Atomic Redis decrement, tolerance buffer Auto
Inventory Service Reservation leak Stock permanently reserved Orphaned reservation scan TTL-based auto-release, cleanup job Auto
Payment Service Gateway timeout Payment hangs Timeout metric Retry with exponential backoff, webhook < 30s
Payment Service Double charge Financial loss Reconciliation mismatch Idempotent payment confirmations Manual
Kafka Broker failure Event loss / delay Consumer lag spike ISR replication, auto-rebalance < 1min
Kafka Topic partition imbalance Hot partition, slow consumers Partition lag metrics Reassign partitions, add consumers < 5min
CDN Cache poisoning Stale/wrong content served Content hash mismatch Cache invalidation API, short TTLs < 1min
Recommendation Model serving down No personalized recs Inference latency alert Fallback to trending/popular Auto

10.2 Cascading Failure Prevention

flowchart LR
    subgraph Patterns["Resilience Patterns"]
        CB[Circuit Breaker<br/>Open after N failures]
        B[Bulkhead<br/>Isolate failure domains]
        R[Retry with Backoff<br/>Exponential + jitter]
        T[Timeout<br/>Fast fail, don't hang]
        Q[Queue-Based Load Leveling<br/>Absorb spikes]
        D[Degradation<br/>Drop non-essentials]
    end

    CB -->|Prevents| CF[Cascading Failures]
    B -->|Prevents| CF
    R -->|Handles| TF[Transient Failures]
    T -->|Prevents| RH[Resource Exhaustion]
    Q -->|Handles| SP[Traffic Spikes]
    D -->|Preserves| CP[Critical Paths]

10.3 Disaster Recovery Plan

Scenario RPO RTO Procedure
AZ failure 0 < 1 min Traffic shifts to remaining AZs; auto-scaling compensates
Region failure < 1 min < 5 min DNS failover to secondary region; DB promotion
Data corruption Last backup < 1 hour Restore from backup; replay Kafka events for incremental recovery
Ransomware / security breach Last clean backup < 4 hours Isolate affected systems; restore from immutable backups; incident response

10.4 Monitoring and Alerting

Key Metrics (Golden Signals):

  • Latency: P50, P95, P99 per endpoint
  • Traffic: RPS per service, per endpoint
  • Errors: 4xx/5xx rate, error budget burn rate
  • Saturation: CPU, memory, disk I/O, connection pool usage

Business Metrics:

  • Checkout conversion rate
  • Cart abandonment rate
  • Search no-results rate
  • Order fulfillment time
  • Inventory accuracy rate

Alerting Strategy:

  • Page: P0 services down, error rate > 5%, latency P99 > 5s
  • Ticket: Error rate > 1%, cache hit rate < 90%, consumer lag > 10K
  • Dashboard: All metrics visible in real-time (Grafana / CloudWatch)

Appendix A: Technology Stack Summary

Layer Technology Rationale
Frontend React / Next.js (Web), React Native (Mobile) Component reuse, SSR for SEO
API Gateway Kong / AWS API Gateway Rate limiting, auth, routing, observability
Services Java (Spring Boot) / Go Type safety, ecosystem, performance
Message Queue Apache Kafka High throughput, durability, replayability
Relational DB MySQL (InnoDB) ACID, mature, widely supported
NoSQL DB MongoDB Flexible schema for catalog
Cache Redis Cluster Sub-ms latency, rich data structures
Search Elasticsearch Full-text, faceting, relevance ranking
Object Storage S3 / OSS Scalable, durable, CDN integration
ML Platform Spark + Flink + MLflow Batch + streaming, model lifecycle
Container Orchestration Kubernetes Auto-scaling, self-healing, multi-tenant
Service Mesh Istio mTLS, traffic management, observability
Monitoring Prometheus + Grafana + Jaeger Metrics, dashboards, distributed tracing

Appendix B: Capacity Planning Summary

Metric Steady State Peak (10x) Headroom Target
API Gateway RPS 83K 833K 1M
Search QPS 5K 50K 75K
Checkout TPS 55 550 1K
Redis Ops/sec 100K 1M 1.5M
Kafka throughput 50K msg/s 500K msg/s 750K msg/s
MySQL connections 2K 20K 30K