Cloud InfrastructureQuestion #4ADVANCED

AWS: Multi-Tenancy, VPC, EC2, S3 & IAM Architecture

Architectural deep dive into cloud hyperscaler foundations including hardware virtualization, software-defined networking VPCs, IAM policy evaluation, and fault isolation.

Category: Category 3: Cloud Infrastructure•Updated: 2026-08-30
#Cloud#Multi-tenancy#VPC#EC2#S3#IAM#Regions & AZs

Design a Cloud Solution Provider like AWS

Table of Contents

  1. Requirements Clarification
  2. Back-of-Envelope Calculations
  3. System Interface Design
  4. High-Level Architecture
  5. Core Component Design
  6. Data Model
  7. Key Design Decisions
  8. Scalability Strategy
  9. Curveball Scenarios
  10. Failure Modes and Mitigations

1. Requirements Clarification

1.1 Functional Requirements

Compute (EC2-like)

  • Launch, stop, start, terminate virtual machines of configurable types (CPU, memory, GPU)
  • Support multiple OS images (AMI) with user-defined custom images
  • Instance metadata service accessible from within instances
  • Auto-scaling groups based on metrics (CPU, memory, custom)
  • Elastic IP addresses and security groups (stateful firewalls)

Storage (S3-like)

  • Object storage with 99.999999999% (11 nines) durability SLA
  • Unlimited scale per bucket, billions of objects per account
  • Versioning, lifecycle policies, encryption (SSE-S3, SSE-KMS, SSE-C)
  • Pre-signed URLs for temporary access
  • Event notifications on object creation/deletion

Networking (VPC-like)

  • Virtual Private Cloud with configurable CIDR blocks
  • Subnets (public/private), route tables, internet gateways, NAT gateways
  • VPC peering and Transit Gateway for multi-VPC connectivity
  • Elastic Load Balancer (ALB, NLB, GWLB)
  • DNS resolution via Route 53-like service

Identity & Access Management (IAM)

  • Users, groups, roles with fine-grained policies (JSON-based)
  • Multi-factor authentication (MFA)
  • Cross-account role assumption (STS)
  • Policy evaluation with deny-overrides-allow semantics

Monitoring & Management

  • CloudWatch-like metrics, logs, and alarms
  • CloudTrail-like audit logging of all API calls
  • Cost explorer and budgeting

1.2 Non-Functional Requirements

Requirement Target Rationale
API Availability 99.99% (control plane) Control plane must be highly available
Data Durability 11 nines (S3) Customer data must never be lost
API Latency (p99) < 500ms for control plane Developer experience
Multi-tenancy Isolation Strong (hard isolation) No cross-tenant data leakage
Geographic Scale 6+ regions, 18+ AZs Global customer base
Throughput Millions of API calls/sec across all services Enterprise-scale usage
Compliance SOC 2, ISO 27001, HIPAA, PCI-DSS Enterprise trust

2. Back-of-Envelope Calculations

2.1 Assumption Sets

We consider three scales:

Parameter Startup (10K customers) Growth (100K customers) Scale (1M customers)
Active accounts 10,000 100,000 1,000,000
Avg instances/account 50 200 500
Total instances 500K 20M 500M
API calls/day 10M 100M 1B
S3 objects total 10B 100B 1T
S3 data volume 10 PB 100 PB 1 EB
Control plane QPS ~120 ~1,200 ~12,000
Data plane throughput 100 GB/s 1 TB/s 10 TB/s

2.2 Storage Calculations (S3)

Per-object metadata: ~1 KB (name, size, content-type, timestamps, owner, version ID, tags)

Scale Objects Metadata Storage Data Storage Total
Startup 10B 10 TB 10 PB ~10 PB
Growth 100B 100 TB 100 PB ~100 PB
Scale 1T 1 PB 1 EB ~1 EB

Replication factor: 3x across AZs → multiply all by 3 for physical storage.

Metadata index: Need a distributed key-value store for object → location mapping. At 1T objects × 1KB = 1PB of metadata index, partitioned across thousands of nodes.

2.3 Bandwidth Calculations

Control plane:

  • 1B API calls/day = ~11,500 QPS average
  • Peak = 3× average = ~35,000 QPS
  • Each API response ~2 KB → 70 MB/s average, 210 MB/s peak

Data plane (S3):

  • Assume 10% of data accessed daily: 100 PB/day at scale
  • 100 PB / 86,400s = ~1.16 TB/s average throughput
  • Peak = 3× = ~3.5 TB/s

2.4 Compute Infrastructure

Hypervisor layer:

  • 500M instances at scale, assume 10:1 overcommit ratio on average
  • Need ~50M physical CPU cores
  • At 64 cores/server = ~780K physical servers

Control plane servers:

  • 35,000 QPS peak, each server handles ~500 QPS
  • Need ~70 servers per service per region (with 3x redundancy = 210 servers)

3. System Interface Design

3.1 EC2 API

POST /v1/instances
Content-Type: application/json

{
  "imageId": "ami-0abcdef1234567890",
  "instanceType": "m5.xlarge",
  "minCount": 1,
  "maxCount": 10,
  "subnetId": "subnet-abc123",
  "securityGroupIds": ["sg-abc123"],
  "keyName": "my-key-pair",
  "tagSpecifications": [{
    "resourceType": "instance",
    "tags": [{"key": "Name", "value": "web-server"}]
  }]
}

Response 200:
{
  "instances": [
    {
      "instanceId": "i-0abc123def456789",
      "state": "pending",
      "publicIpAddress": "54.123.45.67",
      "privateIpAddress": "10.0.1.50"
    }
  ]
}
POST /v1/instances/{instanceId}/action
{ "action": "stop" | "start" | "terminate" }

GET /v1/instances?filter=tag:Name=web-server
DELETE /v1/instances?instanceIds=i-abc,i-def

3.2 S3 API

PUT /{bucket}/{key}
x-amz-acl: private
x-amz-storage-class: STANDARD
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream
Content-Length: 10485760

[object body]

Response 200:
x-amz-version-id: "v-abc123"
ETag: "\"d41d8cd98f00b204e9800998ecf8427e\""
GET /{bucket}/{key}?versionId=v-abc123
GET /{bucket}?list-type=2&prefix=logs/&max-keys=1000
DELETE /{bucket}/{key}?versionId=v-abc123
POST /{bucket}?delete  (multi-object delete)

3.3 VPC API

POST /v1/vpcs
{
  "cidrBlock": "10.0.0.0/16",
  "instanceTenancy": "default",
  "amazonProvidedIpv6CidrBlock": false
}

POST /v1/subnets
{
  "vpcId": "vpc-abc123",
  "cidrBlock": "10.0.1.0/24",
  "availabilityZone": "us-east-1a"
}

POST /v1/route-tables
{
  "vpcId": "vpc-abc123",
  "routes": [
    {"destinationCidrBlock": "0.0.0.0/0", "gatewayId": "igw-abc123"}
  ]
}

3.4 IAM API

POST /v1/roles
{
  "roleName": "EC2DeployRole",
  "assumeRolePolicyDocument": {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Service": "ec2.cloudprovider.com"},
      "Action": "sts:AssumeRole"
    }]
  }
}

POST /v1/policies
{
  "policyName": "S3ReadOnly",
  "policyDocument": {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"]
    }]
  }
}

POST /v1/roles/{roleName}/attach-policy
{ "policyArn": "arn:aws:iam::policy/S3ReadOnly" }

3.5 STS (Temporary Credentials)

POST /v1/sts/assume-role
{
  "roleArn": "arn:aws:iam::123456:role/EC2DeployRole",
  "roleSessionName": "deploy-session",
  "durationSeconds": 3600
}

Response 200:
{
  "credentials": {
    "accessKeyId": "ASIA...",
    "secretAccessKey": "wJalr...",
    "sessionToken": "FwoGZXIv...",
    "expiration": "2026-08-28T12:00:00Z"
  }
}

4. High-Level Architecture

graph TB
    subgraph "Customer"
        CLI[CLI / SDK]
        Console[Web Console]
    end

    subgraph "Edge Layer"
        DNS[Global DNS / Route 53]
        CDN[CDN / CloudFront]
        WAF[WAF / Shield]
    end

    subgraph "API Gateway Layer (per Region)"
        GW1[API Gateway - EC2]
        GW2[API Gateway - S3]
        GW3[API Gateway - VPC]
        GW4[API Gateway - IAM]
        GW5[API Gateway - ELB]
    end

    subgraph "Auth Layer"
        IAM_SVC[IAM Service]
        STS_SVC[STS Service]
        KMS_SVC[KMS Service]
    end

    subgraph "Control Plane Services"
        EC2_CP[EC2 Control Plane]
        S3_CP[S3 Control Plane]
        VPC_CP[VPC Control Plane]
        ELB_CP[ELB Control Plane]
        AS[Auto Scaling]
    end

    subgraph "Data Plane"
        subgraph "Region A"
            subgraph "AZ 1a"
                Hyp1[Hypervisor Pool]
                S3Node1[S3 Storage Nodes]
                VPC1a[VPC Overlay Network]
            end
            subgraph "AZ 1b"
                Hyp2[Hypervisor Pool]
                S3Node2[S3 Storage Nodes]
                VPC1b[VPC Overlay Network]
            end
            subgraph "AZ 1c"
                Hyp3[Hypervisor Pool]
                S3Node3[S3 Storage Nodes]
                VPC1c[VPC Overlay Network]
            end
        end

        subgraph "Region B"
            S3Node4[S3 Storage Nodes]
            Hyp4[Hypervisor Pool]
            VPC2[VPC Overlay Network]
        end
    end

    subgraph "Shared Services"
        CLOUDWATCH[CloudWatch Metrics]
        CLOUDTRAIL[CloudTrail Audit]
        CONFIG[Config Service]
        EVENTBRIDGE[Event Bridge]
    end

    subgraph "Metadata & State"
        DYNAMO[DynamoDB - Control State]
        PARAMETER[Parameter Store]
        SECRET[Secrets Manager]
    end

    CLI --> DNS
    Console --> DNS
    DNS --> CDN
    CDN --> WAF
    WAF --> GW1
    WAF --> GW2
    WAF --> GW3
    WAF --> GW4
    WAF --> GW5

    GW1 --> IAM_SVC
    GW2 --> IAM_SVC
    GW3 --> IAM_SVC
    GW4 --> IAM_SVC
    GW5 --> IAM_SVC

    IAM_SVC --> STS_SVC
    IAM_SVC --> KMS_SVC

    GW1 --> EC2_CP
    GW2 --> S3_CP
    GW3 --> VPC_CP
    GW5 --> ELB_CP

    EC2_CP --> AS
    EC2_CP --> Hyp1
    EC2_CP --> Hyp2
    EC2_CP --> Hyp3
    EC2_CP --> Hyp4

    S3_CP --> S3Node1
    S3_CP --> S3Node2
    S3_CP --> S3Node3
    S3_CP --> S3Node4

    VPC_CP --> VPC1a
    VPC_CP --> VPC1b
    VPC_CP --> VPC1c
    VPC_CP --> VPC2

    EC2_CP --> DYNAMO
    S3_CP --> DYNAMO
    VPC_CP --> DYNAMO

    Hyp1 --> CLOUDWATCH
    S3Node1 --> CLOUDWATCH
    GW1 --> CLOUDTRAIL
    GW2 --> CLOUDTRAIL
    EC2_CP --> EVENTBRIDGE
    S3_CP --> EVENTBRIDGE

5. Core Component Design

5.1 Multi-Tenancy Architecture

Tenant Isolation Model

graph LR
    subgraph "Tenant A"
        VPCA[Tenant A VPC]
        INSTA[Instances]
        S3A[S3 Buckets]
    end

    subgraph "Tenant B"
        VPCB[Tenant B VPC]
        INSTB[Instances]
        S3B[S3 Buckets]
    end

    subgraph "Shared Infrastructure"
        HW[Physical Hardware]
        NET[Network Fabric]
        HYPER[Hypervisor]
    end

    VPCA --> HYPER
    VPCB --> HYPER
    HYPER --> HW
    NET --> VPCA
    NET --> VPCB

Isolation Layers:

Layer Isolation Mechanism Guarantee
Network VXLAN overlay + security groups L2/L3 isolation, no cross-tenant traffic
Compute Hardware VMs (Nitro) with dedicated ENI Memory/CPU isolation via hardware
Storage Encrypted volumes with per-tenant keys Cryptographic isolation
Identity Separate IAM realms per account Policy-based access control
Metadata Partitioned by account ID in all DBs Query-level isolation

5.2 VPC Design

VXLAN Overlay Network:

graph TB
    subgraph "VPC: 10.0.0.0/16"
        subgraph "Subnet 10.0.1.0/24 (AZ-a)"
            ENI1[ENI: 10.0.1.10]
            ENI2[ENI: 10.0.1.11]
        end
        subgraph "Subnet 10.0.2.0/24 (AZ-b)"
            ENI3[ENI: 10.0.2.10]
            ENI4[ENI: 10.0.2.11]
        end
    end

    VTEP1[VTEP - AZ-a]
    VTEP2[VTEP - AZ-b]

    ENI1 --> VTEP1
    ENI2 --> VTEP1
    ENI3 --> VTEP2
    ENI4 --> VTEP2

    VTEP1 <-->|VXLAN VNI=1001| VTEP2

    IGW[Internet Gateway]
    NAT[NAT Gateway]
    RT[Route Table]

    VTEP1 --> RT
    VTEP2 --> RT
    RT --> IGW
    RT --> NAT

Key Design Points:

  • Each VPC assigned a unique VNI (VXLAN Network Identifier), 24-bit space = 16M VPCs
  • VTEPs (VXLAN Tunnel Endpoints) run on each hypervisor host
  • Security groups implemented as distributed stateful firewalls at the VTEP
  • Route tables evaluated per-packet at the VTEP before encapsulation
  • Cross-AZ traffic stays within the cloud fabric (no internet transit)

5.3 IAM Design

Policy Evaluation Engine:

graph TD
    REQ[API Request] --> EXTRACT[Extract Principal, Action, Resource]
    EXTRACT --> CACHE{Policy Cache Hit?}
    CACHE -->|Yes| EVAL[Evaluate Cached Policy]
    CACHE -->|No| FETCH[Fetch Policies from Store]
    FETCH --> COMPILE[Compile Policy Set]
    COMPILE --> CACHE2[Cache Policy Set]
    CACHE2 --> EVAL

    EVAL --> CHECK{Explicit Deny?}
    CHECK -->|Yes| DENY[Return DENY]
    CHECK -->|No| ALLOW{Explicit Allow?}
    ALLOW -->|Yes| COND{Conditions Met?}
    ALLOW -->|No| DENY2[Return DENY]
    COND -->|Yes| ALLOW2[Return ALLOW]
    COND -->|No| DENY3[Return DENY]

    DENY --> AUDIT[Audit Log]
    DENY2 --> AUDIT
    DENY3 --> AUDIT
    ALLOW2 --> AUDIT

Policy Store Design:

  • Policies stored in a distributed document store (JSON policies)
  • Index by: userId, groupId, roleId, policyId, resourceArn
  • Policy compilation: merge all attached policies → single decision tree
  • Cache compiled policies with TTL (5 min default, invalidated on policy change)
  • Explicit deny always wins (short-circuit evaluation)

5.4 EC2 / Compute Service

Instance Lifecycle:

stateDiagram-v2
    [*] --> Pending: RunInstances
    Pending --> Running: Hypervisor boots VM
    Pending --> Failed: Launch failure
    Running --> Stopping: StopInstances
    Stopping --> Stopped: VM halted
    Stopped --> Starting: StartInstances
    Starting --> Running: VM resumed
    Running --> ShuttingDown: TerminateInstances
    ShuttingDown --> Terminated: Resources released
    Terminated --> [*]
    Failed --> [*]

Placement Strategy:

  • Bin-packing algorithm for instance placement across hosts
  • Constraints: AZ affinity, host capacity, anti-affinity rules, dedicated hosts
  • RACK awareness for fault tolerance
  • Capacity reservation for critical workloads

Nitro Hypervisor Architecture:

  • Bare-metal servers with Nitro Cards handling networking, storage, and virtualization
  • Guest VMs run directly on host CPU (no hypervisor overhead for guest code path)
  • ENI (Elastic Network Interface) implemented in FPGA on Nitro Card
  • EBS volumes served via NVMe over Fabric from Nitro Card

5.5 S3 Object Storage

Storage Architecture:

graph TB
    subgraph "S3 Request Flow"
        CLIENT[Client] --> ZONE[Availability Zone Local Endpoint]
        ZONE --> INDEX[Metadata Index Service]
        INDEX --> ZONE_DATA{Data in this AZ?}
        ZONE_DATA -->|Yes| LOCAL[Local Data Node]
        ZONE_DATA -->|No| REMOTE[Cross-AZ Fetch]
        LOCAL --> CLIENT
        REMOTE --> CLIENT
    end

    subgraph "Data Replication"
        DN1[Data Node AZ-1a]
        DN2[Data Node AZ-1b]
        DN3[Data Node AZ-1c]
        DN1 <-->|Async Replication| DN2
        DN2 <-->|Async Replication| DN3
        DN3 <-->|Async Replication| DN1
    end

    subgraph "Metadata"
        META1[Metadata Shard 1]
        META2[Metadata Shard 2]
        META3[Metadata Shard N]
    end

    INDEX --> META1
    INDEX --> META2
    INDEX --> META3

Key Design Decisions:

  • Flat namespace: bucket/key → no directory structure in storage layer
  • Metadata separation: Object metadata in distributed KV store, data in erasure-coded storage
  • Erasure coding: 12+3 scheme (12 data blocks, 3 parity) for durability with 20% overhead vs 3x replication
  • Multi-AZ replication: Write to primary AZ, async replicate to 2 other AZs
  • Consistency model: Read-after-write consistent (eventual consistency for cross-region)
  • Large object handling: Multi-part upload for objects > 100 MB

Metadata Index:

  • Partitioned by hash(bucket + key) across thousands of shards
  • Each shard replicated 3x for high availability
  • Supports prefix scans via sorted key ordering within shards

5.6 Elastic Load Balancer

ALB (Application Load Balancer):

  • Layer 7 load balancer with path/host-based routing
  • WebSocket and HTTP/2 support
  • Integrated WAF and authentication

NLB (Network Load Balancer):

  • Layer 4 load balancer for TCP/UDP
  • Static IP assignment, ultra-low latency
  • Supports millions of RPS per endpoint

Health Checking:

  • Configurable health check endpoints per target group
  • Unhealthy targets removed from rotation automatically
  • Sticky sessions via cookies when needed

5.7 CloudWatch Monitoring

Metrics Pipeline:

Instances → Agent → Kinesis Firehose → Time-Series DB → Dashboard/Alarms
  • Custom metrics via PutMetricData API
  • Alarms evaluate metrics at configurable intervals (1 min minimum)
  • SNS integration for alarm notifications
  • Log ingestion via CloudWatch Logs agent

6. Data Model

6.1 IAM Schema

-- Users
CREATE TABLE iam_users (
    user_id         UUID PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    user_name       VARCHAR(128) NOT NULL,
    created_at      TIMESTAMP NOT NULL,
    password_hash   BYTEA,
    mfa_enabled     BOOLEAN DEFAULT FALSE,
    status          VARCHAR(16) DEFAULT 'Active'
);

-- Groups
CREATE TABLE iam_groups (
    group_id        UUID PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    group_name      VARCHAR(128) NOT NULL
);

-- Roles
CREATE TABLE iam_roles (
    role_id         UUID PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    role_name       VARCHAR(128) NOT NULL,
    assume_role_policy JSONB NOT NULL,
    arn             VARCHAR(512) GENERATED ALWAYS AS (
        'arn:aws:iam::' || account_id || ':role/' || role_name
    ) STORED
);

-- Policies
CREATE TABLE iam_policies (
    policy_id       UUID PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    policy_name     VARCHAR(128) NOT NULL,
    policy_document JSONB NOT NULL,
    is_system       BOOLEAN DEFAULT FALSE,
    version         INTEGER DEFAULT 1
);

-- User-Group Membership
CREATE TABLE iam_user_groups (
    user_id         UUID REFERENCES iam_users(user_id),
    group_id        UUID REFERENCES iam_groups(group_id),
    PRIMARY KEY (user_id, group_id)
);

-- Policy Attachments (Users, Groups, Roles)
CREATE TABLE iam_policy_attachments (
    attachment_id   UUID PRIMARY KEY,
    entity_type     VARCHAR(8) NOT NULL,  -- 'USER', 'GROUP', 'ROLE'
    entity_id       UUID NOT NULL,
    policy_id       UUID REFERENCES iam_policies(policy_id),
    attached_at     TIMESTAMP NOT NULL,
    UNIQUE (entity_type, entity_id, policy_id)
);

-- Access Keys
CREATE TABLE iam_access_keys (
    access_key_id   VARCHAR(20) PRIMARY KEY,
    user_id         UUID REFERENCES iam_users(user_id),
    secret_hash     BYTEA NOT NULL,
    status          VARCHAR(16) DEFAULT 'Active',
    created_at      TIMESTAMP NOT NULL,
    last_used_at    TIMESTAMP
);

-- Indexes
CREATE INDEX idx_users_account ON iam_users(account_id);
CREATE INDEX idx_roles_account ON iam_roles(account_id);
CREATE INDEX idx_policies_account ON iam_policies(account_id);
CREATE INDEX idx_attachments_entity ON iam_policy_attachments(entity_type, entity_id);

6.2 EC2 Schema

-- Instances
CREATE TABLE ec2_instances (
    instance_id     VARCHAR(17) PRIMARY KEY,  -- i-0abc123def456789
    account_id      VARCHAR(12) NOT NULL,
    region          VARCHAR(32) NOT NULL,
    availability_zone VARCHAR(32) NOT NULL,
    image_id        VARCHAR(17) NOT NULL,
    instance_type   VARCHAR(32) NOT NULL,
    state           VARCHAR(16) NOT NULL,     -- pending/running/stopped/terminated
    private_ip      INET,
    public_ip       INET,
    host_id         UUID,                     -- physical host
    vpc_id          VARCHAR(10),
    subnet_id       VARCHAR(17),
    launched_at     TIMESTAMP NOT NULL,
    terminated_at   TIMESTAMP,
    user_data       BYTEA,
    tags            JSONB
);

-- AMIs
CREATE TABLE ec2_amis (
    image_id        VARCHAR(17) PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    name            VARCHAR(256),
    description     TEXT,
    architecture    VARCHAR(16),              -- x86_64, arm64
    root_device_type VARCHAR(16),
    block_device_mappings JSONB,
    state           VARCHAR(16) DEFAULT 'available',
    is_public       BOOLEAN DEFAULT FALSE,
    created_at      TIMESTAMP NOT NULL
);

-- Security Groups
CREATE TABLE ec2_security_groups (
    group_id        VARCHAR(10) PRIMARY KEY,  -- sg-abc123
    account_id      VARCHAR(12) NOT NULL,
    vpc_id          VARCHAR(10),
    group_name      VARCHAR(256),
    description     VARCHAR(256),
    created_at      TIMESTAMP NOT NULL
);

CREATE TABLE ec2_security_group_rules (
    rule_id         UUID PRIMARY KEY,
    group_id        VARCHAR(10) REFERENCES ec2_security_groups(group_id),
    direction       VARCHAR(8) NOT NULL,      -- ingress/egress
    ip_protocol     VARCHAR(32),
    from_port       INTEGER,
    to_port         INTEGER,
    cidr_ipv4       INET,
    cidr_ipv6       CIDR,
    source_group_id VARCHAR(10),
    is_egress       BOOLEAN
);

-- Indexes
CREATE INDEX idx_instances_account ON ec2_instances(account_id);
CREATE INDEX idx_instances_state ON ec2_instances(account_id, state);
CREATE INDEX idx_instances_host ON ec2_instances(host_id);
CREATE INDEX idx_instances_vpc ON ec2_instances(vpc_id);

6.3 S3 Schema

-- Buckets
CREATE TABLE s3_buckets (
    bucket_name     VARCHAR(63) PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    region          VARCHAR(32) NOT NULL,
    created_at      TIMESTAMP NOT NULL,
    versioning      BOOLEAN DEFAULT FALSE,
    lifecycle_rules JSONB,
    encryption_default VARCHAR(16),
    tags            JSONB
);

-- Object Metadata (partitioned by hash of bucket+key)
CREATE TABLE s3_object_metadata (
    bucket_name     VARCHAR(63) NOT NULL,
    object_key      TEXT NOT NULL,
    version_id      VARCHAR(64),
    size            BIGINT NOT NULL,
    content_type  VARCHAR(256),
    content_md5     VARCHAR(32),
    storage_class   VARCHAR(16) DEFAULT 'STANDARD',
    owner_id        VARCHAR(12) NOT NULL,
    etag            VARCHAR(128),
    created_at      TIMESTAMP NOT NULL,
    last_modified   TIMESTAMP NOT NULL,
    is_latest       BOOLEAN DEFAULT TRUE,
    is_delete_marker BOOLEAN DEFAULT FALSE,
    user_metadata   JSONB,
    PRIMARY KEY (bucket_name, object_key, version_id)
);

-- Object Location (maps to physical storage)
CREATE TABLE s3_object_location (
    bucket_name     VARCHAR(63) NOT NULL,
    object_key      TEXT NOT NULL,
    version_id      VARCHAR(64) NOT NULL,
    partition_id    INTEGER NOT NULL,
    replica_az_1    VARCHAR(32) NOT NULL,
    replica_az_2    VARCHAR(32) NOT NULL,
    replica_az_3    VARCHAR(32) NOT NULL,
    PRIMARY KEY (bucket_name, object_key, version_id)
);

-- Indexes
CREATE INDEX idx_objects_bucket_key ON s3_object_metadata(bucket_name, object_key);
CREATE INDEX idx_objects_latest ON s3_object_metadata(bucket_name, object_key)
    WHERE is_latest = TRUE;
CREATE INDEX idx_objects_created ON s3_object_metadata(created_at);

6.4 VPC Schema

CREATE TABLE vpc_vpcs (
    vpc_id          VARCHAR(10) PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    cidr_block      CIDR NOT NULL,
    state           VARCHAR(16) DEFAULT 'available',
    dns_support     BOOLEAN DEFAULT TRUE,
    dns_hostnames   BOOLEAN DEFAULT TRUE,
    vxlan_vni       INTEGER UNIQUE NOT NULL,
    created_at      TIMESTAMP NOT NULL
);

CREATE TABLE vpc_subnets (
    subnet_id       VARCHAR(17) PRIMARY KEY,
    vpc_id          VARCHAR(10) REFERENCES vpc_vpcs(vpc_id),
    cidr_block      CIDR NOT NULL,
    availability_zone VARCHAR(32) NOT NULL,
    map_public_ip   BOOLEAN DEFAULT FALSE,
    available_ips   INTEGER GENERATED ALWAYS AS (
        HOSTMASK(cidr_block) - 4
    ) STORED,
    created_at      TIMESTAMP NOT NULL
);

CREATE TABLE vpc_route_tables (
    route_table_id VARCHAR(17) PRIMARY KEY,
    vpc_id           VARCHAR(10) REFERENCES vpc_vpcs(vpc_id),
    is_main          BOOLEAN DEFAULT FALSE,
    created_at       TIMESTAMP NOT NULL
);

CREATE TABLE vpc_routes (
    route_id        UUID PRIMARY KEY,
    route_table_id  VARCHAR(17) REFERENCES vpc_route_tables(route_table_id),
    destination_cidr CIDR NOT NULL,
    target_type     VARCHAR(16) NOT NULL,  -- igw, nat, vpc, instance
    target_id       VARCHAR(20) NOT NULL,
    state           VARCHAR(16) DEFAULT 'active'
);

CREATE TABLE vpc_internet_gateways (
    igw_id          VARCHAR(10) PRIMARY KEY,
    account_id      VARCHAR(12) NOT NULL,
    vpc_id          VARCHAR(10) REFERENCES vpc_vpcs(vpc_id),
    state           VARCHAR(16) DEFAULT 'available',
    created_at      TIMESTAMP NOT NULL
);

7. Key Design Decisions

7.1 Region vs Availability Zone

Aspect Region Availability Zone
Definition Geographic area (e.g., us-east-1) Isolated datacenter within a region
Latency between units 50-200ms+ < 2ms
Failure independence Independent power, network Independent power, cooling, network
Data replication Cross-region (async, user-configured) Cross-AZ (automatic, synchronous for control plane)
Use case Disaster recovery, latency optimization High availability within a region

Decision: 3 AZs per region minimum. Each AZ has independent:

  • Power feeds from separate substations
  • Cooling systems
  • Network core switches
  • Physical security

7.2 Consistency Model Trade-offs

Service Consistency Rationale
IAM Strong Security-critical, low write volume
EC2 Control Strong Instance state must be consistent
S3 Data Read-after-write Developer expectation, achievable with modern distributed systems
S3 Cross-Region Eventual Latency vs consistency trade-off
VPC State Strong Network configuration must be consistent

7.3 Multi-Tenancy Isolation: Hard vs Soft

Decision: Hard isolation (dedicated hardware paths where possible)

Approach Pros Cons Our Choice
Soft (containers) High density, fast startup Weaker isolation, noisy neighbor Not for customer VMs
Hard (VMs with Nitro) Strong isolation, hardware-backed Lower density, higher cost Chosen for EC2
Hybrid Best of both Complex Containers for our own services

7.4 S3 Durability: Erasure Coding vs Replication

Decision: Erasure coding (12+3) for standard storage, 3x replication for metadata

Approach Overhead Survives Complexity
3x Replication 200% 2 of 3 failures Low
EC 12+3 25% 3 block failures Medium
EC 8+2 25% 2 block failures Medium

Erasure coding chosen for data (87.5% space efficiency vs 33% for replication). Metadata uses 3x replication for faster reads and simpler implementation.

7.5 API Gateway: Per-Service vs Unified

Decision: Per-service API gateways with shared auth middleware

  • Each service owns its gateway → independent scaling, deployment
  • Shared IAM validation middleware → consistent auth, single source of truth
  • API versioning per service → independent evolution

8. Scalability Strategy

8.1 Horizontal Scaling Dimensions

graph LR
    subgraph "Scale Axis 1: Regions"
        R1[Region US-East]
        R2[Region EU-West]
        R3[Region AP-South]
    end

    subgraph "Scale Axis 2: AZs per Region"
        R1 --> AZ1a[AZ 1a]
        R1 --> AZ1b[AZ 1b]
        R1 --> AZ1c[AZ 1c]
    end

    subgraph "Scale Axis 3: Shards per Service"
        AZ1a --> S1[Shard 1]
        AZ1a --> S2[Shard 2]
        AZ1a --> SN[Shard N]
    end

    subgraph "Scale Axis 4: Instances per Shard"
        S1 --> I1[Instance 1]
        S1 --> I2[Instance 2]
    end

8.2 Scaling Strategies by Component

Component Scaling Strategy Trigger Max Scale
API Gateways Horizontal pod autoscaling CPU > 70%, queue depth 1000+ per region
Control Plane Shard by account_id Shard size > 10K accounts Unlimited
S3 Metadata Consistent hashing shards Shard > 100M objects 10,000+ shards
S3 Data Partition by object hash Partition > 100 TB Unlimited
Hypervisors Cluster-based placement Cluster utilization > 80% 100K+ hosts
VPC VTEPs Per-host deployment New host added 1:1 with hosts

8.3 Database Sharding Strategy

Control Plane State (DynamoDB-like):

  • Primary shard key: account_id (hash)
  • Secondary index: service_type + resource_id
  • Hot account handling: Sub-sharding by resource type for high-volume accounts

S3 Metadata:

  • Shard key: hash(bucket_name + object_key)
  • 16-bit partition number → 65,536 partitions
  • Each partition fits on a single storage node
  • Rebalancing via consistent hashing with virtual nodes

8.4 Capacity Planning

Growth Projections:

Year Accounts Instances S3 Data Regions AZs
Year 1 10K 500K 10 PB 3 9
Year 3 100K 20M 100 PB 6 18
Year 5 1M 500M 1 EB 12 36

Infrastructure Scaling:

  • Add AZs before adding regions (deeper before wider)
  • Pre-provision capacity 20% above forecast
  • Auto-scale data plane independently of control plane

9. Curveball Scenarios

9.1 Curveball 1: A Major Customer Launches a Viral App Overnight

Scenario: A customer’s application goes viral, generating 100× normal traffic in 2 hours. Their S3 bucket serves 50,000 GET requests/sec, and their EC2 instances need to scale from 100 to 10,000.

Impact Analysis:

  • S3 GET rate: 50K RPS from one account (potential hot partition)
  • EC2 launch: 9,900 new instances in minutes
  • Network bandwidth: Potential egress spike to 100 Gbps from one tenant

Response Strategy:

Layer Action Timeline
S3 Consistent hashing distributes across partitions; no single hot partition if key distribution is uniform. If hot key detected, enable read replicas automatically. Immediate
EC2 Auto-scaling group triggers scale-out. Pre-warmed instance pool provides immediate capacity. Spot instances supplement on-demand. 1-5 min
Network Egress rate limiting per account (configurable). Customer notified if approaching limits. Immediate
Billing Real-time cost alerting to customer. Budget breach notifications via SNS. Immediate
Infrastructure Regional capacity monitoring triggers pre-provisioning if regional utilization > 85%. 5-15 min

Prevention:

  • Customer education on auto-scaling best practices
  • Service quotas with easy increase process
  • Predictive scaling based on historical patterns

9.2 Curveball 2: An AZ Goes Down Due to Power Failure

Scenario: AZ-1a in us-east-1 loses power due to a substation failure. 50,000 instances and critical S3 data affected.

Impact Analysis:

  • All instances in AZ-1a become unreachable
  • S3 objects with primary replica in AZ-1a: reads served from AZ-1b/1c replicas
  • VPC subnets in AZ-1a: routing disrupted
  • Estimated recovery time: 2-4 hours for power restoration

Response Strategy:

Phase Action Timeline
Detection Health monitors detect AZ unreachability within 30 seconds. Auto-classify as AZ-down event. 0-30 sec
Failover S3: Reads automatically served from remaining AZ replicas. Writes queued and redirected to healthy AZs. 30-60 sec
Compute EC2: Instances marked as impaired. Auto-scaling groups launch replacement instances in healthy AZs. 1-5 min
Network VPC: Route tables updated to remove AZ-1a routes. Cross-AZ traffic rerouted. 1-2 min
Notification CloudWatch alarms fire. SNS notifications to affected customers. Status page updated. 1-3 min
Recovery Power restored → instances in AZ-1a recover. Data consistency verified. Customer instances restarted if needed. 2-4 hrs

Data Safety:

  • S3: No data loss (3 AZ replication, quorum-based writes)
  • EBS: Volumes in AZ-1a may need recovery from cross-AZ snapshots
  • RDS: Multi-AZ failover to standby in healthy AZ

9.3 Curveball 3: A Bug in the IAM Policy Engine Causes Mass Denial

Scenario: A deployment of the IAM policy evaluation service introduces a bug that denies all requests for accounts with more than 100 policies attached.

Impact Analysis:

  • Affected: Large enterprise accounts with complex IAM setups
  • Symptom: All API calls return AccessDenied
  • Blast radius: Potentially 10-20% of enterprise customers

Response Strategy:

Phase Action Timeline
Detection Error rate monitoring detects spike in AccessDenied responses. Anomaly detection flags the pattern. 1-3 min
Triage On-call engineer identifies correlation with recent IAM deployment. 3-5 min
Mitigation Emergency rollback of IAM service to previous version. Circuit breaker activates for affected accounts, bypassing to cached policies. 5-10 min
Verification Canary validation against test accounts. Gradual traffic shift back. 10-15 min
Root Cause Post-incident analysis. Bug: integer overflow in policy compilation for >100 policies. Post-incident

Prevention:

  • Canary deployments with automatic rollback on error rate increase
  • Policy evaluation unit tests with edge cases (100+ policies)
  • Chaos engineering:定期 inject IAM failures in staging
  • Policy compilation result caching as fallback

9.4 Curveball 4: Cross-Region Data Sovereignty Regulation Change

Scenario: A new regulation requires that data for customers in Region EU-West must never leave the EU geographic boundary, including metadata and audit logs.

Impact Analysis:

  • Current state: CloudTrail audit logs replicated to us-east-1 for central analysis
  • Current state: S3 cross-region replication configured by some EU customers to US regions
  • Current state: IAM policy cache may serve from any region

Response Strategy:

Component Change Effort
CloudTrail Add data residency tags. EU customers’ logs stay in EU regions. Regional analysis pipelines. Medium
S3 Enforce cross-region replication restrictions for EU buckets. Add compliance validation. Low
IAM Regional policy cache isolation. EU accounts evaluate policies within EU. Medium
Billing Regional billing data isolation. Low
Support Regional support ticket routing. Low

Architecture Change:

graph LR
    subgraph "EU Region"
        EU_API[EU API Gateway]
        EU_IAM[EU IAM Service]
        EU_TRAIL[EU CloudTrail]
        EU_ANALYTICS[EU Analytics]
    end

    subgraph "US Region"
        US_API[US API Gateway]
        US_IAM[US IAM Service]
        US_TRAIL[US CloudTrail]
        US_ANALYTICS[US Analytics]
    end

    EU_API --> EU_IAM
    EU_API --> EU_TRAIL
    EU_TRAIL --> EU_ANALYTICS

    US_API --> US_IAM
    US_API --> US_TRAIL
    US_TRAIL --> US_ANALYTICS

    EU_ANALYTICS -.->|No cross-region data| US_ANALYTICS

9.5 Curveball 5: A Customer Discovers They Can Access Another Customer’s Data

Scenario: A security researcher finds that by crafting specific S3 requests with manipulated VNI headers, they can read objects from another tenant’s bucket.

Impact Analysis:

  • Severity: CRITICAL - Cross-tenant data leakage
  • Root cause: VTEP not validating VNI against tenant’s VPC membership
  • Blast radius: All customers sharing VTEP infrastructure

Response Strategy:

Phase Action Timeline
Detection Bug bounty report received. Security team validates the exploit. T+0
Containment Emergency patch: VTEP validates VNI against tenant authorization before processing. Deploy to all hosts. T+1 hr
Assessment Scan CloudTrail logs for suspicious cross-tenant access patterns. Identify affected customers. T+2 hrs
Notification Notify affected customers per breach notification policy. Engage legal and PR. T+24 hrs
Remediation Full security audit of VTEP code. Additional validation layers. Enhanced monitoring. T+1 week

Prevention:

  • Defense in depth: VNI validation at multiple layers (VTEP, hypervisor, network fabric)
  • Regular penetration testing by third parties
  • Bug bounty program with attractive rewards
  • Zero-trust network architecture: never trust tenant-supplied network parameters

10. Failure Modes and Mitigations

10.1 Failure Mode Matrix

Failure Mode Probability Impact Detection Mitigation RTO RPO
Single host failure High Low (1-2 instances) Health monitor Auto-replace, live migration 5 min 0
AZ power loss Medium High Multi-probe monitoring Cross-AZ replication, failover 30 min 0 (S3)
Region outage Low Critical Regional health dashboard Multi-region active-active 1 hr 1 hr
Control plane DB corruption Low Critical Checksum validation Multi-AZ sync replication 15 min 0
Network partition Medium Medium BFD, ECMP monitoring Automatic rerouting 1 min 0
IAM service outage Low Critical API error rate monitoring Cached credentials, STS token validity N/A N/A
S3 metadata corruption Low High CRC validation on reads 3x replication, automatic repair 1 hr 0
Hypervisor compromise Very Low Critical Integrity monitoring Nitro hardware isolation, attestation 4 hrs N/A
DDoS attack Medium Medium Traffic anomaly detection WAF, rate limiting, scrubbing 5 min N/A
Data center flood/fire Very Low High Physical sensors Geographic distribution 4 hrs 0

10.2 Control Plane Resilience

graph TB
    subgraph "Control Plane HA"
        subgraph "AZ-1a"
            CP1[Control Plane Primary]
            DB1[State DB Primary]
        end
        subgraph "AZ-1b"
            CP2[Control Plane Standby]
            DB2[State DB Replica]
        end
        subgraph "AZ-1c"
            CP3[Control Plane Standby]
            DB3[State DB Replica]
        end
    end

    LB[Load Balancer] --> CP1
    LB --> CP2
    LB --> CP3

    CP1 --> DB1
    CP2 --> DB2
    CP3 --> DB3

    DB1 <-->|Sync Replication| DB2
    DB2 <-->|Sync Replication| DB3

    HEALTH[Health Checker] --> CP1
    HEALTH --> CP2
    HEALTH --> CP3
    HEALTH --> LB

Key Principles:

  • Quorum-based writes: Control plane writes require 2-of-3 AZ acknowledgment
  • Automatic leader election: If primary fails, standby takes over within 30 seconds
  • Read-your-writes: Client sessions pinned to consistent replica during write operations
  • Circuit breakers: If a service dependency fails, fallback to cached state or graceful degradation

10.3 Data Plane Resilience

S3 Self-Healing:

  1. Background scanner runs every hour checking object integrity (checksums)
  2. Corrupted or missing replicas detected → automatic regeneration from healthy replicas
  3. If all replicas corrupted → alert operations team (extremely rare with EC + 3 AZ)
  4. Storage node failure → data redistributed automatically within 15 minutes

EC2 Instance Recovery:

  1. Host failure detected → instances marked as impaired
  2. Auto-scaling groups launch replacements in healthy hosts
  3. EBS volumes remain intact (stored separately from compute)
  4. Instance recovery: start new instance attached to existing EBS volume

10.4 Disaster Recovery Plan

Scenario Strategy RTO RPO
AZ failure Cross-AZ failover (automatic) 5 min 0
Region failure Multi-region active-active (automatic for S3, manual for EC2) 30 min 0 (S3), variable (EC2)
Global failure Not designed for (requires geographic distribution at customer level) N/A N/A
Data corruption Versioning + cross-region replication 1 hr Last replicated version
Ransomware Immutable backups + versioning 4 hrs Last clean version

Appendix: Glossary

Term Definition
VNI VXLAN Network Identifier - 24-bit ID for overlay network segmentation
VTEP VXLAN Tunnel Endpoint - encapsulates/decapsulates VXLAN packets
ENI Elastic Network Interface - virtual network card for instances
AMI Amazon Machine Image - template for launching instances
STS Security Token Service - issues temporary credentials
Nitro Hardware-based virtualization architecture using dedicated cards
EBS Elastic Block Store - persistent block storage for instances
RTO Recovery Time Objective - max acceptable downtime
RPO Recovery Point Objective - max acceptable data loss