Domain 3Exploit Dev & AV EvasionCORE

Endianness Converter Reference

Complete reference for understanding and converting between big-endian and little-endian byte ordering. Covers x86 memory layout, conversion techniques, common mistakes, and practical tools for exploit development.

Domain: Domain 3: Exploit Mechanics & AV Evasion•Updated: 2026-09-01
#endianness#little-endian#byte-order#memory-layout#oscp

Appendix B: Endianness Converter Reference

Why this matters: Endianness is one of the most fundamental concepts in exploit development. Every time you write a return address, a JMP ESP address, or any multi-byte value into a buffer, you must get the byte order right — or your exploit silently fails. This appendix is your definitive reference.


Table of Contents

  1. What Is Endianness?
  2. Big-Endian vs Little-Endian: Visual Comparison
  3. Why x86 Uses Little-Endian
  4. Visual Memory Diagrams
  5. Conversion Techniques
  6. Common Address Conversions Table
  7. Null Byte Detection
  8. Why Null Bytes Matter in Exploits
  9. 64-Bit Endianness
  10. Common Mistakes & Debugging
  11. Practice Exercises
  12. Quick-Reference Cheat Sheet

1. What Is Endianness?

Endianness (or byte order) defines the sequence in which bytes of a multi-byte value are stored in computer memory.

A 32-bit (4-byte) value like 0x41424344 consists of four individual bytes:

Byte Position Big-Endian Name Hex Value
Most Significant MSB (Most Significant Byte) 0x41
2nd — 0x42
3rd — 0x43
Least Significant LSB (Least Significant Byte) 0x44

The question is: which byte goes at the lowest memory address?

The Two Conventions

Big-Endian (Network Order):       Little-Endian (x86 Order):
┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐  ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐
│0x41 │ │0x42 │ │0x43 │ │0x44 │  │0x44 │ │0x43 │ │0x42 │ │0x41 │
└─────┘ └─────┘ └─────┘ └─────┘  └─────┘ └─────┘ └─────┘ └─────┘
 ^         ^         ^         ^    ^         ^         ^         ^
Low Addr   |         |    High Addr  Low Addr   |         |    High Addr
MSB first  |         |    LSB last   LSB first  |         |    MSB last
  • Big-Endian: Most significant byte at the lowest address (like we write numbers: left-to-right)
  • Little-Endian: Least significant byte at the lowest address (reversed)

Etymology

The terms come from Jonathan Swift’s Gulliver’s Travels (1726):

  • Big-Endian — the empire that cracks eggs at the big end
  • Little-Endian — the empire that cracks eggs at the little end

In computing, this playful naming stuck to describe the two competing byte-order conventions.


2. Big-Endian vs Little-Endian: Visual Comparison

The Same Value, Two Different Layouts

Consider the 32-bit value 0xDEADBEEF:

Value: 0xDEADBEEF
Bytes: DE | AD | BE | EF
       MSB              LSB

Big-Endian Memory Layout

Memory Address:  0xFFE4  0xFFE5  0xFFE6  0xFFE7
                  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
Stored Bytes:     │ 0xDE │ │ 0xAD │ │ 0xBE │ │ 0xEF │
                  └──────┘ └──────┘ └──────┘ └──────┘
                    MSB                            LSB
                  (Low Address)                (High Address)

Reading left-to-right:  DEAD BEEF  ← matches human-readable order

Little-Endian Memory Layout

Memory Address:  0xFFE4  0xFFE5  0xFFE6  0xFFE7
                  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐
Stored Bytes:     │ 0xEF │ │ 0xBE │ │ 0xAD │ │ 0xDE │
                  └──────┘ └──────┘ └──────┘ └──────┘
                    LSB                            MSB
                  (Low Address)                (High Address)

Reading left-to-right:  EF BE AD DE  ← REVERSED from human-readable order

Mermaid Diagram: Byte Ordering Comparison

graph LR
    subgraph "Big-Endian"
        A1["Addr+0: 0xDE (MSB)"] --> A2["Addr+1: 0xAD"]
        A2 --> A3["Addr+2: 0xBE"]
        A3 --> A4["Addr+3: 0xEF (LSB)"]
    end

    subgraph "Little-Endian (x86)"
        B1["Addr+0: 0xEF (LSB)"] --> B2["Addr+1: 0xBE"]
        B2 --> B3["Addr+2: 0xAD"]
        B3 --> B4["Addr+3: 0xDE (MSB)"]
    end

    style A1 fill:#ff9999
    style A4 fill:#99ccff
    style B1 fill:#99ccff
    style B4 fill:#ff9999

Key Mnemonic

“Little-Endian = Little End First” — the least significant byte (the “little end”) comes first in memory.


3. Why x86 Uses Little-Endian

Historical Context

The x86 architecture (Intel 8086, 1978) chose little-endian for a practical reason:

Efficient arithmetic on variable-length integers.

When performing addition, the CPU starts with the least significant byte. In little-endian, the LSB is at the lowest address, so the CPU can:

  1. Start reading from the base address immediately
  2. Process bytes in memory order (no need to skip ahead)
  3. Handle numbers of any size without knowing the size in advance

Big-Endian Counter-Argument

Big-endian has its own advantages:

  • Sign bit always at the same position regardless of integer size (easier for comparisons)
  • Matches human-readable notation (what you write is what you see in memory)
  • Used by network protocols (TCP/IP — “network byte order”)

Who Uses What?

Architecture Endianness Notes
x86 / x86_64 Little-Endian Intel, AMD — what you’ll exploit in OSCP
ARM Bi-Endian Can be configured either way (most default to little-endian)
PowerPC Big-Endian (traditionally) Some PowerPC variants; newer ones bi-endian
MIPS Bi-Endian Depends on configuration
SPARC Big-Endian Oracle SPARC
Network Protocols Big-Endian TCP/IP, HTTP headers, etc.

Why This Matters for OSCP

Nearly every OSCP lab machine runs x86 or x86_64 Linux/Windows. This means:

  • Every return address you inject must be in little-endian order
  • Every address you find with msf-pattern_create, objdump, or gdb must be reversed before placing in your payload
  • If you get this wrong, your exploit crashes with a segmentation fault — and debugging can be frustrating

4. Visual Memory Diagrams

4.1 A 32-bit Address in Memory

Let’s trace how the address 0x00401234 is stored in x86 (little-endian) memory:

Human-readable address:  0x00401234

                        ┌─────────────────────────────────────────┐
                        │           Memory Map (x86 LE)           │
                        └─────────────────────────────────────────┘

Memory Address:         0xBFFFF0    0xBFFFF1    0xBFFFF2    0xBFFFF3
                         ┌────────┐   ┌────────┐   ┌────────┐   ┌────────┐
Stored Byte:             │ 0x34   │   │ 0x12   │   │ 0x40   │   │ 0x00   │
                         └────────┘   └────────┘   └────────┘   └────────┘
                                  ↑                    ↑
                              Least Sig            Most Sig
                              Byte (LSB)           Byte (MSB)

In a hex dump, you would see:  34 12 40 00

4.2 Buffer Overflow Context: Return Address Overwrite

This is the most critical diagram for exploit development:

┌──────────────────────────────────────────────────────────────────┐
│              Stack Memory (growing upward for clarity)           │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Buffer (vulnerable variable)                                    │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │ 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A' 'A' 'A' 'A' 'A'    │   │
│  │ 0x41 0x41 0x41 0x41 ... 0x41 0x41 0x41 0x41 0x41 0x41   │   │
│  └──────────────────────────────────────────────────────────┘   │
│                              ↑                                   │
│                        Buffer boundary                           │
│                                                                  │
│  Saved EBP (4 bytes)                                            │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │ 'A' 'A' 'A' 'A'                                          │   │
│  │ 0x41 0x41 0x41 0x41                                      │   │
│  └──────────────────────────────────────────────────────────┘   │
│                              ↑                                   │
│                        Overwritten EBP                           │
│                                                                  │
│  Return Address (4 bytes) ← YOU CONTROL THIS                    │
│  ┌──────┬──────┬──────┬──────┐                                  │
│  │ \x34 │ \x12 │ \x40 │ \x00 │   ← Little-endian for           │
│  │ LSB  │      │      │ MSB  │     0x00401234                  │
│  └──────┴──────┴──────┴──────┘                                  │
│                              ↑                                   │
│                  When function returns, EIP ← 0x00401234        │
└──────────────────────────────────────────────────────────────────┘

4.3 What Happens When You Get It Wrong

┌──────────────────────────────────────────────────────────────────┐
│              CORRECT (Little-Endian)                             │
├──────────────────────────────────────────────────────────────────┤
│  Payload bytes:  \x34\x12\x40\x00                                │
│  Memory reads as:  0x00401234  ✓ Valid address                   │
│  EIP jumps to:     0x00401234  ✓ Shellcode executes              │
└──────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────┐
│              WRONG (Big-Endian by mistake)                        │
├──────────────────────────────────────────────────────────────────┤
│  Payload bytes:  \x00\x40\x12\x34                                │
│  Memory reads as:  0x34124000  ✗ Wrong address                   │
│  EIP jumps to:     0x34124000  ✗ Segmentation fault!             │
└──────────────────────────────────────────────────────────────────┘

4.4 Mermaid: Exploit Flow with Endianness

sequenceDiagram
    participant Attacker
    participant Payload
    participant Buffer
    participant Stack
    participant CPU

    Attacker->>Payload: Find JMP ESP at 0x78011234
    Attacker->>Payload: Convert to LE: \x34\x12\x01\x78
    Attacker->>Buffer: Send overflow payload
    Buffer->>Stack: Overwrite return address
    Note over Stack: Bytes stored: 34 12 01 78
    CPU->>Stack: Pop EIP from stack
    Note over CPU: CPU reads LE → 0x78011234
    CPU->>CPU: EIP = 0x78011234 (JMP ESP)
    CPU->>CPU: Execute shellcode ✓

5. Conversion Techniques

5.1 Manual Byte Reversal

The simplest technique: read the hex address in pairs from right to left.

Step-by-step for 0x7E482390:

Step 1: Write the address in hex pairs
        7E 48 23 90

Step 2: Reverse the order of the pairs
        90 23 48 7E

Step 3: Convert to escape sequences
        \x90\x23\x48\x7e

Result: In your exploit payload, write \x90\x23\x48\x7e
        The CPU will read this as 0x7E482390

5.2 Python: struct.pack()

Python’s struct module is the gold standard for endianness conversion in exploit scripts.

Format Characters for Integers

Format Size Endianness Description
<I 4 bytes Little-Endian Unsigned int (x86)
>I 4 bytes Big-Endian Unsigned int (network)
=I 4 bytes Native System default
!I 4 bytes Big-Endian Network byte order (same as >I)
<L 4 bytes Little-Endian Unsigned long
<Q 8 bytes Little-Endian Unsigned long long (x64)

Python 3 Examples

import struct

# ── Basic conversion ──────────────────────────────────────────────
address = 0x7E482390

# Little-endian (what x86 needs)
le_bytes = struct.pack('<I', address)
print(le_bytes)
# Output: b'\x90#\x48~'

# Big-endian (network order)
be_bytes = struct.pack('>I', address)
print(be_bytes)
# Output: b'~H#\x90'

# ── Verify the round-trip ─────────────────────────────────────────
# Pack then unpack
packed = struct.pack('<I', 0x00401234)
unpacked = struct.unpack('<I', packed)[0]
print(f"Original: 0x{0x00401234:08x}")
print(f"Unpacked: 0x{unpacked:08x}")
# Both print: 0x00401234

# ── Generate payload-friendly string ──────────────────────────────
def to_little_endian_hex(address):
    """Convert address to little-endian hex escape sequence."""
    return struct.pack('<I', address).hex()

print(to_little_endian_hex(0x7E482390))
# Output: 9023487e

# ── Build a complete return address payload ────────────────────────
ret_addr = 0x78011234
payload = b'A' * 1024           # NOP sled / filler
payload += struct.pack('<I', ret_addr)  # Return address (LE)
payload += b'\x90' * 16         # NOP sled
payload += shellcode            # Your shellcode

Python One-Liners (for quick use in terminal)

# Quick conversion in Python one-liner
python3 -c "import struct; print(struct.pack('<I', 0x7E482390).hex())"
# Output: 9023487e

# With escape sequences
python3 -c "import struct; print(repr(struct.pack('<I', 0x7E482390)))"
# Output: b'\x90#\x48~'

# Multiple addresses at once
python3 -c "
import struct
addrs = [0x7E482390, 0x00401234, 0x7FFD4A20]
for a in addrs:
    print(f'0x{a:08x} -> {struct.pack(\"<I\", a).hex()}')
"
# Output:
# 0x7e482390 -> 9023487e
# 0x00401234 -> 34124000
# 0x7ffd4a20 -> 204affd7

5.3 Ruby: pack()

Ruby’s pack method is equally powerful and commonly used in Metasploit-style exploits.

Format Characters

Format Endianness Size Description
V Little-Endian 4 bytes 32-bit unsigned (x86)
N Big-Endian 4 bytes 32-bit unsigned (network)
v Little-Endian 2 bytes 16-bit unsigned
n Big-Endian 2 bytes 16-bit unsigned
Q< Little-Endian 8 bytes 64-bit unsigned
Q> Big-Endian 8 bytes 64-bit unsigned

Ruby Examples

# ── Basic conversion ──────────────────────────────────────────────
address = 0x7E482390

# Little-endian (V = 32-bit unsigned, little-endian)
le = [address].pack('V')
puts le.inspect
# Output: "\x90#\x48~"

# Big-endian (N = 32-bit unsigned, big-endian)
be = [address].pack('N')
puts be.inspect
# Output: "~H#\x90"

# ── Hex representation ────────────────────────────────────────────
puts [address].pack('V').unpack('H*').first
# Output: 9023487e

# ── Build a payload ───────────────────────────────────────────────
ret_addr = 0x78011234
payload = 'A' * 1024
payload << [ret_addr].pack('V')   # Return address in LE
payload << "\x90" * 16            # NOP sled
payload << shellcode

# ── Multiple addresses ────────────────────────────────────────────
addresses = [0x7E482390, 0x00401234, 0xDEADBEEF]
addresses.each do |addr|
  puts "0x%08x -> %s" % [addr, [addr].pack('V').unpack('H*').first]
end
# Output:
# 0x7e482390 -> 9023487e
# 0x00401234 -> 34124000
# 0xdeadbeef -> efbeadde

Ruby One-Liners

# Quick conversion
ruby -e "puts [0x7E482390].pack('V').unpack('H*').first"
# Output: 9023487e

# With escape sequences
ruby -e "puts [0x7E482390].pack('V').inspect"
# Output: "\x90#\x48~"

5.4 Using printf in Bash

# Using printf to convert (GNU printf with %s and $'...')
printf '\\x%s' $(echo "7E482390" | sed 's/../\\x& /g' | rev | sed 's/\\x//; s/ //g' | sed 's/\(.\{2\}\)/\\x\1 /g')

# Simpler approach: use xxd
echo -n "7E482390" | xxd -r -p | rev | xxd -p
# Output: 9023487e

5.5 Online Converters and Calculator Methods

Windows Calculator (Programmer Mode)

Method:
1. Open calc.exe
2. Switch to Programmer mode
3. Enter the address in HEX: 7E482390
4. The bytes are displayed — mentally reverse them
5. Result: 90 23 48 7E
Tool URL Pattern Notes
Endianness Converter Various Search “endianness converter online”
CyberChef gchq.github.io/CyberChef “From Hex” → “Reverse” recipe
Python/Terminal (local) Most reliable — no internet needed

CyberChef Recipe for Endianness

From Hex("Space")
Reverse(32)
To Hex("Space", "Lower")

5.6 Comparison Table: All Methods Side by Side

Converting 0x7E482390 to little-endian bytes:

Method Command / Code Output
Manual Reverse pairs: 7E 48 23 90 → 90 23 48 7E \x90\x23\x48\x7e
Python struct.pack('<I', 0x7E482390) b'\x90#\x48~'
Ruby [0x7E482390].pack('V') "\x90#\x48~"
Perl pack('V', 0x7E482390) \x90#\x48~
Bash printf '\x90\x23\x48\x7e' (raw bytes)
xxd echo -n "7E482390" | xxd -r -p | rev | xxd -p 9023487e

6. Common Address Conversions Table

Below is a pre-computed reference table of addresses you’ll commonly encounter in OSCP labs.

6.1 Common JMP/CALL ESP Addresses (Little-Endian)

Original Address Description Little-Endian Bytes Hex String
0x78011234 Example JMP ESP (MSVCRT) \x34\x12\x01\x78 34120178
0x7E482390 Example POP POP RET \x90\x23\x48\x7E 9023487e
0x7C801E00 Kernel32 base (XP) \x00\x1E\x80\x7C 001e807c
0x77D20000 Kernel32 base (7) \x00\x00\xD2\x77 0000d277
0x7FFD0000 TLS callback area \x00\x00\xFD\x7F 0000fd7f
0x00401000 Typical EXE base \x00\x10\x40\x00 00104000
0x08048000 Typical ELF base \x00\x80\x04\x08 00800408
0xBFFF0000 Stack top area \x00\x00\xFF\xBF 0000ffbf

6.2 MSVCRT.dll Common Gadgets (Windows XP SP3)

Address Instruction Little-Endian Notes
0x77C301AE JMP ESP \xAE\x01\xC3\x77 Classic target
0x77C3336E CALL ESP \x6E\x33\xC3\x77 Alternative
0x77C2B89A POP EDI; RET \x9A\xB8\xC2\x77 ROP gadget
0x77C34417 XCHG EAX, ESP \x17\x44\xC3\x77 Pivot gadget

Note: Actual addresses vary by OS version, service pack, and patch level. Always verify with mona or manual inspection on your target.

6.3 Linux ELF Common Addresses

Address Description Little-Endian Notes
0x08048000 Typical .text base \x00\x80\x04\x08 32-bit ELF
0x08049000 Typical .data base \x00\x90\x04\x08 Writable segment
0xBFFF0000 Stack area \x00\x00\xFF\xBF Approximate
0xF7E00000 libc base area \x00\x00\xE0\xF7 Approximate

6.4 Quick-Reference: Common Hex Values in LE

Value LE Bytes Use Case
0x00000000 \x00\x00\x00\x00 NULL pointer (avoid!)
0xFFFFFFFF \xFF\xFF\xFF\xFF -1, often used as marker
0x41414141 \x41\x41\x41\x41 ‘AAAA’ — symmetric, no reversal needed
0x90909090 \x90\x90\x90\x90 NOP sled — symmetric
0xDEADBEEF \xEF\xBE\xAD\xDE Debug marker
0xC0DEFACE \xCE\xFA\xDC\xC0 Debug marker
0x13371337 \x37\x13\x37\x13 “LEET” marker

7. Null Byte Detection

7.1 What Is a Null Byte?

A null byte is the byte 0x00. In C strings, null bytes serve as string terminators.

7.2 How to Spot Addresses with Null Bytes

An address contains null bytes if any of its hex byte pairs is 00:

Address: 0x00401234
Bytes:   00 | 40 | 12 | 34
         ↑
       NULL BYTE! ← This address contains a null byte

Address: 0x7E482390
Bytes:   7E | 48 | 23 | 90
         No null bytes ✓

Address: 0x00007FFE
Bytes:   00 | 00 | 7F | FE
         ↑    ↑
       TWO null bytes! ← Very problematic

7.3 Quick Null Byte Check Algorithm

def has_null_bytes(address):
    """Check if an address contains any null bytes."""
    return b'\x00' in struct.pack('<I', address)

# Examples
print(has_null_bytes(0x00401234))  # True  — has null byte
print(has_null_bytes(0x7E482390))  # False — clean
print(has_null_bytes(0x7FFD4A20))  # False — clean
print(has_null_bytes(0x00007FFE))  # True  — has null bytes

7.4 Null Byte Detection by Address Range

Address Range Null Bytes? Safe for strcpy?
0x00000000 — 0x000000FF 3 null bytes NO
0x00000100 — 0x0000FFFF 2 null bytes NO
0x00010000 — 0x00FFFFFF 1 null byte NO
0x01000000 — 0xFFFFFFFF 0 null bytes YES

Rule of thumb: Any address below 0x01000000 contains at least one null byte.

7.5 Visual Null Byte Map

Address Space:

0xFFFFFFFF  ────────────────────────────────────────────────────
            │                                                   │
            │   SAFE ZONE (no null bytes)                       │
            │   0x01000000 - 0xFFFFFFFF                         │
            │   ✓ Safe for string-based functions               │
            │                                                   │
0x01000000  ├───────────────────────────────────────────────────┤
            │                                                   │
            │   DANGER ZONE (contains null bytes)               │
            │   0x00000000 - 0x00FFFFFF                         │
            │   ✗ Truncated by strcpy, strlen, etc.             │
            │                                                   │
0x00000000  ────────────────────────────────────────────────────

8. Why Null Bytes Matter in Exploits

8.1 String Termination: The Core Problem

C string functions treat 0x00 as the end of a string:

// These functions STOP at the first null byte:
strcpy(dest, src);      // Copies until null byte
strcat(dest, src);      // Appends until null byte
strlen(str);            // Counts until null byte
sprintf(buf, "%s", str); // Formats until null byte
printf("%s", str);      // Prints until null byte

8.2 Exploit Impact Scenarios

Scenario 1: strcpy-Based Overflow

Your payload:  AAAA[padding]\x00\x12\x40\x00[shellcode]
                        ↑
                  Null byte here!

What strcpy does:
  Copies: AAAA[padding]
  Stops at: \x00  ← NULL BYTE TERMINATES THE COPY

Result: Your return address \x00\x12\x40\x00 is NEVER written to memory.
        Exploit FAILS.

Scenario 2: sprintf-Based Overflow

char buf[256];
sprintf(buf, "User: %s", user_input);

Same problem — sprintf with %s stops at null bytes.

Scenario 3: memcpy — Safe from Null Bytes

// memcpy does NOT stop at null bytes — it copies exactly N bytes:
memcpy(dest, src, count);

If the vulnerable function uses memcpy, memmove, or reads with a fixed count, null bytes are NOT a problem.

8.3 Function Safety Matrix

Function Stops at Null Byte? Safe for Addresses with 0x00?
strcpy YES NO
strcat YES NO
sprintf (%s) YES NO
scanf (%s) YES NO
gets NO (reads until newline) YES
memcpy NO YES
memmove NO YES
read() NO YES
recv() NO YES
fgets NO (reads until newline) YES

8.4 Finding Null-Byte-Free Addresses

When you need an address without null bytes:

Strategy 1: Use addresses >= 0x01000000
  ✓ Most library addresses (MSVCRT, kernel32, libc) are in this range
  ✓ Stack addresses on 32-bit are usually 0xBFxxxxxx — safe

Strategy 2: Use Mona to find gadgets
  !mona find -s "\\xff\\xff" -m msvcrt.dll   # Find specific byte sequences
  !mona jmp -r esp                             # Find JMP ESP (usually safe)

Strategy 3: Use a different gadget
  If your target address has null bytes, find an alternative
  JMP ESP / CALL ESP in the same library

9. 64-Bit Endianness

9.1 x64 Address Layout

In 64-bit (x86_64), addresses are 8 bytes (64 bits) instead of 4 bytes (32 bits).

32-bit address:  0x00401234  (4 bytes)
Little-endian:   \x34\x12\x40\x00

64-bit address:  0x00007FFD4A201000  (8 bytes)
Little-endian:   \x00\x10\x20\x4A\xFD\x7F\x00\x00

9.2 64-Bit Conversion Examples

Address (64-bit) Little-Endian Bytes
0x00007FFD4A201000 \x00\x10\x20\x4A\xFD\x7F\x00\x00
0x0000000040000000 \x00\x00\x00\x40\x00\x00\x00\x00
0x00007FFEEDBADBEF \xEF\xBD\xAD\xEE\xFE\x7F\x00\x00
0xFFFFFFFFFFFFFFFF \xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF

9.3 Python for 64-Bit

import struct

# 64-bit little-endian
addr64 = 0x00007FFD4A201000
packed = struct.pack('<Q', addr64)
print(packed)
# Output: b'\x00\x10 \x4A\xfd\x7f\x00\x00'

# Unpack to verify
unpacked = struct.unpack('<Q', packed)[0]
print(f"0x{unpacked:016x}")
# Output: 0x00007ffd4a201000

9.4 Ruby for 64-Bit

# 64-bit little-endian
addr64 = 0x00007FFD4A201000
packed = [addr64].pack('Q<')
puts packed.inspect
# Output: "\x00\x10 \x4A\xFD\x7F\x00\x00"

# Hex string
puts [addr64].pack('Q<').unpack('H*').first
# Output: 0010204afd7f0000

9.5 64-Bit Null Byte Challenges

64-bit addresses have a much higher chance of containing null bytes because user-space addresses typically use only the lower 48 bits:

User-space 64-bit addresses typically look like:
  0x0000XXXX_YYYYYYYY

The upper 16 bits are almost always zero → at least 2 null bytes!

This is why 64-bit exploitation often requires:
  1. Using binary-safe functions (memcpy, read, recv)
  2. Or finding addresses in the high range (rare)
  3. Or using ROP to write addresses piece by piece

9.6 x64 Register Changes

In x64, the return address is still on the stack, but:

32-bit stack at return:
  [Return Address: 4 bytes]
  ↑ EIP pops this

64-bit stack at return:
  [Return Address: 8 bytes]
  ↑ RIP pops this

The RET instruction in 64-bit mode pops 8 bytes, not 4.

10. Common Mistakes & Debugging

10.1 Mistake #1: Forgetting to Reverse Bytes Entirely

WRONG:  Address 0x7E482390 → \x7E\x48\x23\x90  (big-endian)
RIGHT:  Address 0x7E482390 → \x90\x23\x48\x7E  (little-endian)

Symptom: Segmentation fault. The CPU interprets your address as 0x9023487E instead of 0x7E482390, jumping to invalid memory.

10.2 Mistake #2: Partial Reversal (Reversing Characters Instead of Bytes)

WRONG:  0x7E482390 → \x09\x32\x84\xE7  (reversed individual hex digits!)
RIGHT:  0x7E482390 → \x90\x23\x48\x7E  (reversed byte PAIRS)

Key rule: Reverse in pairs of hex digits (bytes), never individual digits.

10.3 Mistake #3: Using Wrong Python Format

# WRONG: Big-endian format
struct.pack('>I', 0x7E482390)  # Produces \x7E\x48\x23\x90

# RIGHT: Little-endian format
struct.pack('<I', 0x7E482390)  # Produces \x90\x23\x48\x7E

10.4 Mistake #4: Not Accounting for Null Bytes

Address found: 0x00401234
Payload: \x34\x12\x40\x00  ← Contains null byte!

If the vulnerable function is strcpy → payload truncated → exploit fails

Fix: Find an alternative address without null bytes, or verify the vulnerable function is binary-safe.

10.5 Mistake #5: Confusing Hex and Decimal

# WRONG: Treating hex as decimal
struct.pack('<I', 7E482390)  # SyntaxError!

# RIGHT: Use 0x prefix for hex
struct.pack('<I', 0x7E482390)

# Also wrong: forgetting 0x in Python
addr = 7E482390  # Error
addr = 0x7E482390  # Correct

10.6 Mistake #6: Wrong Size for the Architecture

# 32-bit exploit — use 4 bytes
struct.pack('<I', addr)    # Correct for x86

# 64-bit exploit — use 8 bytes
struct.pack('<Q', addr)    # Correct for x64

# WRONG: Using 4 bytes for a 64-bit address
struct.pack('<I', 0x00007FFD4A201000)  # Truncates to lower 32 bits!

10.7 Debugging Checklist

When your exploit crashes, work through this checklist:

┌─────────────────────────────────────────────────────────────────┐
│              ENDIANNESS DEBUGGING CHECKLIST                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  □ 1. Verify the address is correct (check with gdb/mona)       │
│                                                                  │
│  □ 2. Confirm you reversed the bytes (LE, not BE)               │
│     → struct.pack('<I', addr) not struct.pack('>I', addr)       │
│                                                                  │
│  □ 3. Check for null bytes in the address                       │
│     → Is addr >= 0x01000000?                                    │
│                                                                  │
│  □ 4. Verify the vulnerable function is binary-safe             │
│     → strcpy/strcat = NOT safe                                   │
│     → memcpy/read/recv = safe                                    │
│                                                                  │
│  □ 5. Confirm correct size (4 bytes for x86, 8 for x64)        │
│                                                                  │
│  □ 6. In GDB, verify what's actually in memory:                 │
│     x/4xb $esp   → Check raw bytes                              │
│     x/wx $esp    → Check as 32-bit word                         │
│                                                                  │
│  □ 7. Check ASLR — is the address still valid?                  │
│     → Run the target again and re-verify                         │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

10.8 GDB Verification Commands

# After your payload is in memory, verify the return address:

# Show raw bytes at the return address location
(gdb) x/4xb 0xBFFFF7F4
0xbffff7f4: 0x90  0x23  0x48  0x7e

# Show as a 32-bit word (CPU interpretation)
(gdb) x/wx 0xBFFFF7F4
0xbffff7f4: 0x7e482390    ← Should match your target!

# Show what EIP will be after RET
(gdb) info registers eip
eip            0x7e482390  0x7e482390

# Single-step through the RET instruction
(gdb) stepi

11. Practice Exercises

Exercise 1: Basic Conversion

Convert the following addresses to little-endian byte sequences:

# Address Your Answer (LE bytes)
1 0x7E482390
2 0x00401234
3 0xDEADBEEF
4 0x7FFD4A20
5 0xC0DEFACE
Click to reveal answers
# Address Little-Endian
1 0x7E482390 \x90\x23\x48\x7E
2 0x00401234 \x34\x12\x40\x00
3 0xDEADBEEF \xEF\xBE\xAD\xDE
4 0x7FFD4A20 \x20\x4A\xFD\x7F
5 0xC0DEFACE \xCE\xFA\xDC\xC0

Exercise 2: Null Byte Detection

For each address, determine:

  1. Does it contain null bytes? (Yes/No)
  2. How many null bytes?
  3. Is it safe for strcpy-based exploitation?
# Address Has Null? Count Safe for strcpy?
1 0x7E482390
2 0x00401234
3 0x7FFD4A20
4 0x00007FFE
5 0x08048000
6 0x12345678
Click to reveal answers
# Address Has Null? Count Safe for strcpy?
1 0x7E482390 No 0 YES
2 0x00401234 Yes 1 NO
3 0x7FFD4A20 No 0 YES
4 0x00007FFE Yes 2 NO
5 0x08048000 Yes 1 NO
6 0x12345678 No 0 YES

Exercise 3: Python Conversion

Write Python one-liners to convert these addresses:

# Convert these to little-endian bytes:
addresses = [
    0x78011234,
    0x7C801E00,
    0x00401000,
    0xBFFF0000,
    0xDEADBEEF
]

# Your code here:
for addr in addresses:
    # TODO: Convert and print
    pass
Click to reveal solution
import struct

addresses = [
    0x78011234,
    0x7C801E00,
    0x00401000,
    0xBFFF0000,
    0xDEADBEEF
]

for addr in addresses:
    le = struct.pack('<I', addr)
    print(f"0x{addr:08x} → {le.hex()} → {repr(le)}")

Output:

0x78011234 → 34120178 → b'4\x12\x01x'
0x7c801e00 → 001e807c → b'\x00\x1e\x80|'
0x00401000 → 00104000 → b'\x00\x00@\x00'
0xbfff0000 → 0000ffbf → b'\x00\x00\xff\xbf'
0xdeadbeef → efbeadde → b'\xef\xbe\xad\xde'

Exercise 4: 64-Bit Conversion

Convert these 64-bit addresses to little-endian:

# Address Your Answer (LE bytes)
1 0x00007FFD4A201000
2 0x0000000040000000
3 0x00007FFEEDBADBEF
Click to reveal answers
# Address Little-Endian
1 0x00007FFD4A201000 \x00\x10\x20\x4A\xFD\x7F\x00\x00
2 0x0000000040000000 \x00\x00\x00\x40\x00\x00\x00\x00
3 0x00007FFEEDBADBEF \xEF\xBD\xAD\xEE\xFE\x7F\x00\x00

Exercise 5: Build a Complete Payload

Given:

  • Offset to return address: 524 bytes
  • JMP ESP address: 0x7E411234
  • Shellcode: 45 bytes of \xCC (INT3 breakpoints for testing)
  • NOP sled: 32 bytes of \x90

Build the complete exploit payload in Python:

import struct

# Given values
offset = 524
jmp_esp = 0x7E411234
nop_sled_size = 32
shellcode = b'\xcc' * 45

# Build the payload
payload = b''
# TODO: Complete the payload

print(f"Total payload size: {len(payload)} bytes")
print(f"Return address in LE: {repr(struct.pack('<I', jmp_esp))}")
Click to reveal solution
import struct

# Given values
offset = 524
jmp_esp = 0x7E411234
nop_sled_size = 32
shellcode = b'\xcc' * 45

# Build the payload
payload = b''
payload += b'A' * offset                          # Fill buffer to return address
payload += struct.pack('<I', jmp_esp)              # Return address (LE)
payload += b'\x90' * nop_sled_size                 # NOP sled
payload += shellcode                               # Shellcode

print(f"Total payload size: {len(payload)} bytes")
print(f"Return address in LE: {repr(struct.pack('<I', jmp_esp))}")
print(f"Hex of return address: {struct.pack('<I', jmp_esp).hex()}")

Output:

Total payload size: 605 bytes
Return address in LE: b'4\x12\x01~'
Hex of return address: 3412417e

Exercise 6: Debugging Challenge

You wrote an exploit with return address 0x7E482390. Your payload contains:

payload += struct.pack('>I', 0x7E482390)

The exploit crashes with a segmentation fault. What is wrong? How do you fix it?

Click to reveal answer

Problem: You used '>I' (big-endian) instead of '<I' (little-endian).

The bytes written to memory are: \x7E\x48\x23\x90

The CPU reads this as the address: 0x9023487E (not 0x7E482390)

This address is invalid → segmentation fault.

Fix:

# WRONG
payload += struct.pack('>I', 0x7E482390)

# RIGHT
payload += struct.pack('<I', 0x7E482390)

12. Quick-Reference Cheat Sheet

12.1 Conversion Quick Reference

┌─────────────────────────────────────────────────────────────────────┐
│                    ENDIANNESS CHEAT SHEET                           │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ADDRESS → LITTLE-ENDIAN (x86)                                     │
│  ─────────────────────────────────────────                         │
│                                                                     │
│  0x7E482390  →  \x90\x23\x48\x7E                                   │
│  0x00401234  →  \x34\x12\x40\x00  (has null byte!)                 │
│  0xDEADBEEF  →  \xEF\xBE\xAD\xDE                                   │
│                                                                     │
│  PYTHON:                                                           │
│  ───────                                                           │
│  struct.pack('<I', 0x7E482390)   # 32-bit LE                       │
│  struct.pack('<Q', 0xADDR)      # 64-bit LE                        │
│  struct.pack('>I', 0x7E482390)   # 32-bit BE (network)             │
│                                                                     │
│  RUBY:                                                             │
│  ──────                                                            │
│  [0x7E482390].pack('V')       # 32-bit LE                          │
│  [0x7E482390].pack('N')       # 32-bit BE                          │
│  [addr64].pack('Q<')          # 64-bit LE                          │
│                                                                     │
│  NULL BYTE RULES:                                                  │
│  ────────────────                                                  │
│  Address < 0x01000000 → Contains null bytes                        │
│  strcpy/strcat → STOPS at null byte → Use addr >= 0x01000000       │
│  memcpy/read/recv → Binary safe → Any address OK                   │
│                                                                     │
│  REVERSAL RULE:                                                    │
│  ────────────                                                      │
│  Reverse in BYTE PAIRS, not individual digits:                     │
│  7E 48 23 90  →  90 23 48 7E  ✓                                    │
│  7E 48 23 90  →  09 32 84 E7  ✗ (wrong!)                           │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

12.2 Format Character Quick Reference

┌──────────┬──────────┬──────────┬──────────────────────┐
│  Python  │   Ruby   │  Size    │     Description      │
├──────────┼──────────┼──────────┼──────────────────────┤
│  '<I'    │  'V'     │  4 bytes │  32-bit LE (x86)     │
│  '>I'    │  'N'     │  4 bytes │  32-bit BE (network) │
│  '<Q'    │  'Q<'    │  8 bytes │  64-bit LE (x64)     │
│  '>Q'    │  'Q>'    │  8 bytes │  64-bit BE           │
│  '<H'    │  'v'     │  2 bytes │  16-bit LE           │
│  '>H'    │  'n'     │  2 bytes │  16-bit BE           │
└──────────┴──────────┴──────────┴──────────────────────┘

12.3 Memory Layout Summary

┌──────────────────────────────────────────────────────────────────┐
│              x86 MEMORY LAYOUT (Little-Endian)                   │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Value: 0x7E482390                                               │
│                                                                  │
│  ┌───────────┬───────────┬───────────┬───────────┐              │
│  │ Addr+0    │ Addr+1    │ Addr+2    │ Addr+3    │              │
│  ├───────────┼───────────┼───────────┼───────────┤              │
│  │ 0x90      │ 0x23      │ 0x48      │ 0x7E      │              │
│  │ (LSB)     │           │           │ (MSB)     │              │
│  └───────────┴───────────┴───────────┴───────────┘              │
│                                                                  │
│  In payload:  \x90\x23\x48\x7E                                   │
│  CPU reads:   0x7E482390  ✓                                      │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

Appendix: Additional Resources

Tools Used in This Guide

Tool Purpose
struct (Python) Pack/unpack binary data with endianness control
pack (Ruby) Same as Python struct
mona.py (Mona) Find addresses and gadgets in memory
gdb Verify memory layout and debug exploits
xxd Hex dump and conversion

Common OSCP Scenarios Requiring Endianness

  1. Buffer overflow with return address overwrite — Every return address must be LE
  2. Format string exploits — Writing addresses to memory requires LE
  3. ROP chains — Each gadget address must be LE
  4. SEH overwrites — Next SEH and SEH handler addresses must be LE
  5. Heap exploits — Chunk metadata and function pointers must be LE

Final Note: Endianness is simple in concept but critical in practice. One wrong byte order and your exploit fails silently. Make it a habit to always verify your byte order before testing. Use the tools above, build muscle memory with the exercises, and soon it will become second nature.


This study guide is part of the OSCP Module 10: Fixing Exploits series. Last updated: 2026-09-01