Domain 1Recon & ReportingCORE

OS Fingerprinting

Master active and passive OS fingerprinting techniques to identify target operating systems, understand TCP/IP stack differences, and leverage OS-specific attack vectors in OSCP engagements.

Domain: Domain 1: Reconnaissance & Reporting•Updated: 2026-09-01
#os-fingerprinting#nmap-os#ttl#tcp-stack

OS Fingerprinting

Attack Chain Position

graph LR
    A[Reconnaissance] --> B[Port Scanning]
    B --> C[Service Detection]
    C --> D[OS Fingerprinting]
    D --> E[Vulnerability Research]
    E --> F[Exploitation]
    
    style D fill:#f9d56e,stroke:#c79632,stroke-width:3px

Where we are: After identifying open ports and service versions, OS fingerprinting determines the target’s operating system. This information guides exploit selection, privilege escalation strategies, and post-exploitation techniques.


Why OS Fingerprinting Matters

Strategic Value

Knowing the target OS helps you:

Decision Point Linux Knowledge Windows Knowledge
Exploit Selection Linux kernel exploits, Samba, Apache MS17-010, EternalBlue, SMB exploits
Privilege Escalation Linux PE scripts, SUID binaries, cron jobs Windows PE, AlwaysInstallElevated, services
Post-Exploitation Linux enumeration scripts, /etc/passwd Windows enumeration, SAM database, registry
Lateral Movement SSH keys, sudo abuse, Linux pivoting Pass-the-hash, WMI, PsExec, Windows pivoting
Payload Generation Linux reverse shells, ELF binaries Windows reverse shells, EXE/PS1 payloads

OSCP Lab OS Distribution

Based on common OSCP lab targets:

OS Type Approximate Frequency Common Versions
Linux (Ubuntu/Debian) ~40% Ubuntu 14.04, 16.04, 18.04, 20.04
Linux (CentOS/RHEL) ~15% CentOS 6, 7, 8
Windows Server ~25% 2008 R2, 2012 R2, 2016, 2019
Windows (Desktop) ~10% Windows 7, 10
Other (BSD, etc.) ~10% FreeBSD, OpenBSD

Active OS Fingerprinting with Nmap

The -O Flag

Nmap’s active OS detection sends a series of TCP and UDP packets to the target and analyzes the responses to determine the operating system.

# Basic OS detection
nmap -O <target>

# OS detection with version detection
nmap -O -sV <target>

# Aggressive OS detection with guessing
nmap -O --osscan-guess <target>

# OS detection with timing template
nmap -O -T4 <target>

# OS detection with output
nmap -O -oN nmap_os_scan.txt <target>

# Full scan with OS detection
nmap -A -O -sV -p- <target>

How Nmap OS Detection Works

Nmap’s OS detection engine uses a sophisticated fingerprinting database:

  1. TCP/IP Stack Probing: Sends specially crafted TCP and UDP packets
  2. Response Analysis: Analyzes how the target responds to each probe
  3. Fingerprint Matching: Compares responses against a database of known OS fingerprints (nmap-os-db)
  4. Confidence Rating: Assigns a confidence percentage to each match

Nmap OS Detection Output

Remote OS details: Linux 4.15 - 5.6
OS CPE: cpe:/o:linux:linux_kernel:4.15

Network Distance: 1 hop
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Output Interpretation

Output Field Meaning
Remote OS details Most likely OS and version range
OS CPE Common Platform Enumeration identifier
Network Distance Number of hops to target
Service Info OS hints from service banners
Aggressive OS guesses Additional possible OS matches

--osscan-guess Flag

When Nmap is not confident enough to make a definitive determination, --osscan-guess tells it to make its best guess anyway.

# Without guessing (conservative)
nmap -O <target>

# With guessing (more aggressive)
nmap -O --osscan-guess <target>

When to use --osscan-guess:

  • Nmap returns “OS detection could not be completed”
  • You need any OS hint, even if uncertain
  • Combined with other fingerprinting methods for validation

Nmap OS Detection Requirements

For accurate OS detection, Nmap needs:

Requirement Description
Root/Admin Privileges Raw packet crafting requires elevated privileges
At Least One Open Port Nmap needs an open port for TCP stack analysis
At Least One Closed Port Nmap needs a closed port (>1000) for comparison
No Firewall Interference Firewalls can alter or block probe packets
Direct Network Path NAT and proxies can interfere with detection

Troubleshooting OS Detection

# Common issues and solutions

# Issue: "OS detection could not be completed"
# Solution: Ensure you have root privileges and open/closed ports
sudo nmap -O --osscan-guess <target>

# Issue: "Too many fingerprints match this host"
# Solution: Use --osscan-guess for best match
nmap -O --osscan-guess <target>

# Issue: "Remote OS version detection" shows multiple possibilities
# Solution: Combine with service banners and manual techniques
nmap -O -sV -A <target>

Passive OS Fingerprinting

Passive OS fingerprinting analyzes network traffic and service responses without sending active probes. This is stealthier and can be performed during normal reconnaissance.

TCP/IP Stack Analysis

Every OS implements the TCP/IP stack slightly differently. These differences create unique fingerprints.

Key TCP/IP Stack Characteristics

Characteristic Description Variation
Initial TTL Starting TTL value for outgoing packets Windows: 128, Linux: 64, macOS: 64, Cisco: 255
Window Size Initial TCP window size Varies by OS and configuration
DF Flag Don’t Fragment flag behavior Some OS always set, some never
TCP Options Order and presence of TCP options OS-specific ordering
MSS Value Maximum Segment Size Varies by network interface
ICMP Responses How OS responds to ICMP probes Different error codes and formats

HTTP Headers Analysis

Web server headers often reveal the underlying OS:

# Check HTTP headers for OS clues
curl -I http://<target>:80

# Look for OS-specific headers
curl -I http://<target>:80 | grep -iE "server|x-powered-by|x-aspnet"

OS-Indicative HTTP Headers:

Header Value Likely OS
Server Apache/2.4.49 (Ubuntu) Linux (Ubuntu)
Server Apache/2.4.49 (Win64) Windows
Server Microsoft-IIS/10.0 Windows Server 2016+
Server Microsoft-IIS/8.5 Windows Server 2012 R2
Server Microsoft-IIS/7.5 Windows Server 2008 R2
X-Powered-By ASP.NET Windows
X-AspNet-Version 4.0.30319 Windows
Server nginx/1.18.0 (Ubuntu) Linux (Ubuntu)
Server lighttpd/1.4.53 Linux (commonly)

SMB Signatures

SMB responses contain OS-specific information:

# Using crackmapexec
crackmapexec smb <target>

# Using rpcclient
rpcclient -U "" <target>

# Using nmap
nmap -p 445 --script "smb-os-discovery,smb-protocols" <target>

# Using impacket
impacket-smbclient ''@<target>

SMB OS Signatures:

Server String Likely OS
Windows Server 2019 Windows Server 2019
Windows Server 2016 Windows Server 2016
Windows 8 Windows 8/8.1
Windows 7 Windows 7
Samba 4.11.6-Ubuntu Linux (Ubuntu)
Samba 3.6.6 Linux (older distro)

SSH Banner Analysis

SSH banners often contain OS-specific information:

# Grab SSH banner
nc -v <target> 22

# Using nmap
nmap -p 22 --script "ssh2-protocol-version" <target>

SSH Banner Patterns:

Banner Pattern Likely OS
OpenSSH_7.2p2 Ubuntu-4ubuntu2.10 Linux (Ubuntu 16.04)
OpenSSH_8.2p1 Ubuntu-4ubuntu0.5 Linux (Ubuntu 20.04)
OpenSSH_7.4p1 Debian-10+deb9u7 Linux (Debian 9)
OpenSSH_8.0p1 Ubuntu-6ubuntu0.6 Linux (Ubuntu 20.04)
SSHDD-1.28.0.48 Windows (Bitvise SSH Server)
WinSSHD-3.2.0 Windows (FreeSSHd)

SNMP Analysis

SNMP can reveal detailed OS information:

# Using snmpwalk
snmpwalk -v 2c -c public <target> 1.3.6.1.2.1.1

# Check system description
snmpwalk -v 2c -c public <target> 1.3.6.1.2.1.1.1

# Check system OID
snmpwalk -v 2c -c public <target> 1.3.6.1.2.1.1.2

# Using nmap
nmap -p 161 --script "snmp-netstat,snmp-processes,snmp-sysdescr" <target>

SNMP System Description Examples:

System Description OS
Linux ubuntu 5.4.0-42-generic Linux (Ubuntu)
Windows Server 2019 Standard Windows Server 2019
Cisco IOS Software, Version 15.2 Cisco IOS

ICMP Response Analysis

Different OSes respond differently to ICMP probes:

# Basic ping
ping <target>

# Ping with different packet sizes
ping -s 64 <target>
ping -s 128 <target>
ping -s 512 <target>

# Ping with different TTL
ping -t 1 <target>
ping -t 64 <target>
ping -t 128 <target>

# Using hping3 for detailed ICMP analysis
hping3 -1 <target>
hping3 -1 -c 10 <target>

TTL-Based OS Detection

Understanding TTL

Time To Live (TTL) is a value in IP packets that decreases by 1 at each router hop. The initial TTL is set by the source OS.

Initial TTL Values by OS

OS Family Initial TTL Common Values Seen
Linux 64 64, 63, 62, 61, 60…
Windows 128 128, 127, 126, 125, 124…
macOS / BSD 64 64, 63, 62, 61, 60…
Cisco IOS 255 255, 254, 253, 252, 251…
Juniper 255 255, 254, 253, 252, 251…
Solaris 255 255, 254, 253, 252, 251…

Calculating Hops from TTL

Initial TTL - Received TTL = Number of Hops

Examples:
- Received TTL = 62, Initial = 64 → 2 hops (Linux)
- Received TTL = 126, Initial = 128 → 2 hops (Windows)
- Received TTL = 124, Initial = 128 → 4 hops (Windows)
- Received TTL = 253, Initial = 255 → 2 hops (Cisco)

TTL Detection Methods

# Using ping
ping <target>
# Look for "ttl=62" or "ttl=126" in output

# Using nmap
nmap -sn <target>
# Check the TTL column in output

# Using hping3
hping3 <target>
# Look for "ttl=62" in response

# Using tcpdump
tcpdump -i eth0 ip and host <target>
# Check TTL field in captured packets

TTL-Based OS Detection Table

Received TTL Likely Initial Likely OS Confidence
60-64 64 Linux / macOS / BSD High
124-128 128 Windows High
251-255 255 Cisco / Juniper / Solaris High
50-59 64 Linux (many hops) Medium
114-123 128 Windows (many hops) Medium
241-250 255 Network device (many hops) Medium

TTL Limitations

Limitation Description
Firewall Modification Some firewalls modify TTL values
NAT Devices NAT can alter TTL values
Load Balancers May use different backend OS
TTL Randomization Some OS randomize initial TTL
Multiple Hops Many hops reduce distinguishing value

OS Fingerprinting Indicators Table

Comprehensive Indicator Reference

Indicator Linux Windows macOS BSD
Initial TTL 64 128 64 64
TCP Window Size 64240, 29200 8192, 65535 16384, 65535 16384, 65535
DF Flag Usually set Usually set Usually set Usually set
TCP Options Order MSS, WScale, SACK, Timestamps MSS, WScale, SACK, Timestamps MSS, SACK, WScale, Timestamps MSS, SACK, WScale
SMB Server String Samba x.x.x Windows Server/Client x.x N/A N/A
HTTP Server Header Apache/nginx + distro IIS, Apache Win32/Win64 Apache + Darwin Apache + FreeBSD
SSH Banner OpenSSH + distro WinSSHD, SSHDD OpenSSH + Darwin OpenSSH + FreeBSD
ICMP Timestamp Unix epoch Windows FILETIME Unix epoch Unix epoch
DNS Response Lowercase Mixed case Lowercase Lowercase
Common Services SSH, Apache, MySQL SMB, RDP, IIS SSH, Apache SSH, Apache

TCP/IP Stack Differences

TCP Options Ordering

The order of TCP options in the initial SYN packet is OS-specific:

Linux:   MSS | Window Scale | SACK Permitted | NOP | Timestamps
Windows: MSS | Window Scale | SACK Permitted | NOP | Timestamps
macOS:   MSS | SACK Permitted | Window Scale | NOP | Timestamps
FreeBSD: MSS | SACK Permitted | Window Scale | NOP

Window Size Analysis

OS Common Window Sizes Notes
Linux 64240, 29200, 5840 Auto-tuned, varies by kernel
Windows 8192, 65535, 14600 Fixed values, version-dependent
macOS 16384, 65535 Similar to BSD
FreeBSD 16384, 65535 Similar to macOS

DF (Don’t Fragment) Flag Analysis

OS DF Flag Behavior Notes
Linux Usually set Path MTU Discovery enabled
Windows Usually set Path MTU Discovery enabled
macOS Usually set Path MTU Discovery enabled
FreeBSD Usually set Path MTU Discovery enabled
Older Windows Sometimes not set Windows 9x/2000

Combining Multiple Indicators for Accuracy

The Multi-Indicator Approach

No single indicator is 100% reliable. Combine multiple sources for accurate identification:

flowchart TD
    A[Start OS Fingerprinting] --> B[TTL Analysis]
    A --> C[Service Banners]
    A --> D[HTTP Headers]
    A --> E[SMB Signatures]
    A --> F[Nmap OS Detection]
    A --> G[SSH Banner]
    
    B --> H[Correlate Indicators]
    C --> H
    D --> H
    E --> H
    F --> H
    G --> H
    
    H --> I{Consensus?}
    I -->|Yes| J[High Confidence OS ID]
    I -->|No| K[Additional Research]
    
    K --> L[Check SNMP]
    K --> M[Analyze DNS]
    K --> N[Review ICMP]
    
    L --> H
    M --> H
    N --> H

Confidence Scoring

Number of Matching Indicators Confidence Level Action
5+ indicators agree Very High (95%+) Proceed with OS-specific attacks
3-4 indicators agree High (80-95%) Proceed, verify during exploitation
2 indicators agree Medium (60-80%) Proceed with caution, have backup plan
1 indicator Low (<60%) Gather more data before proceeding
Conflicting indicators Uncertain Use multiple attack paths

Practical Example

Target: 10.10.10.100

Indicator 1 - TTL: Received TTL = 62 → Initial TTL = 64 → Linux
Indicator 2 - SSH Banner: OpenSSH_7.2p2 Ubuntu-4ubuntu2.10 → Linux (Ubuntu 16.04)
Indicator 3 - HTTP Header: Apache/2.4.18 (Ubuntu) → Linux (Ubuntu)
Indicator 4 - Nmap OS: Linux 4.15 - 5.6 → Linux
Indicator 5 - SMB: Samba 4.3.11-Ubuntu → Linux (Ubuntu)

Conclusion: Linux (Ubuntu 16.04) - Very High Confidence

OS Fingerprinting in OSCP Context

OSCP-Specific Strategies

1. Quick OS Identification

# Quick OS check
nmap -O -sV --top-ports 100 <target>

# If that fails, try with guessing
nmap -O --osscan-guess -sV <target>

# Check TTL with ping
ping -c 4 <target>

# Check SSH banner
nc -v <target> 22

# Check HTTP headers
curl -I http://<target>:80

2. Linux-Specific Follow-Up

If OS is Linux:

# Check for common Linux services
nmap -sV -p 22,80,443,3306,5432,8080,8443 <target>

# Check for Samba
nmap -sV -p 139,445 <target>

# Check for common Linux vulnerabilities
searchsploit linux <kernel_version>

# Prepare Linux enumeration scripts
# linpeas, linux-smart-enumeration, etc.

3. Windows-Specific Follow-Up

If OS is Windows:

# Check for common Windows services
nmap -sV -p 135,139,445,3389,5985,80,443 <target>

# Check for SMB vulnerabilities
crackmapexec smb <target>

# Check for common Windows vulnerabilities
searchsploit windows <version>

# Prepare Windows enumeration scripts
# winPEAS, PowerView, etc.

Common OSCP Lab OS Signatures

Lab Target OS Key Indicators
DC-1 Linux (Ubuntu) SSH banner, HTTP headers
DC-2 Linux (Ubuntu) SSH banner, HTTP headers
DC-3 Linux (Ubuntu) SSH banner, HTTP headers
DC-4 Linux (Ubuntu) SSH banner, HTTP headers
DC-5 Linux (Ubuntu) SSH banner, HTTP headers
DC-6 Linux (Ubuntu) SSH banner, HTTP headers
DC-7 Linux (Ubuntu) SSH banner, HTTP headers
DC-8 Linux (Ubuntu) SSH banner, HTTP headers
HackTheBox - Lame Linux (Kali) Samba 3.0.20, SSH banner
HackTheBox - Blunder Linux (Ubuntu) SSH banner, HTTP headers
HackTheBox - Legacy Windows Server 2003 SMB signature, TTL
HackTheBox - Scooby Windows Server 2008 SMB signature, TTL

Limitations and False Positives

Common Limitations

Limitation Description Impact
Firewalls Can block or modify probe packets Inaccurate or failed detection
NAT/Proxies Intermediate devices alter packets Misidentification
Virtualization VMs may have different stack behavior False positives
Custom Kernels Modified TCP/IP stack Unrecognized fingerprints
OS Updates New versions not in database Outdated matches
Service Wrappers Services behind proxies Misleading banners
Containerization Docker containers share host kernel Host OS detected, not container

Handling False Positives

  1. Cross-verify with multiple methods: Don’t rely on a single indicator
  2. Check service banners: Often more reliable than TCP stack analysis
  3. Use manual techniques: Banner grabbing can confirm or refute Nmap findings
  4. Consider the context: Lab environments may have unusual configurations
  5. Update Nmap databases: Ensure nmap-os-db is current

When OS Detection Fails

# If Nmap OS detection fails, try these alternatives:

# 1. Check service banners manually
nc -v <target> 22
nc -v <target> 80
nc -v <target> 445

# 2. Use SNMP if available
snmpwalk -v 2c -c public <target> 1.3.6.1.2.1.1.1

# 3. Check HTTP headers
curl -I http://<target>:80

# 4. Use crackmapexec for SMB info
crackmapexec smb <target>

# 5. Try with different timing
nmap -O -T4 --osscan-guess <target>

# 6. Use multiple probes
nmap -O --max-os-tries 5 <target>

OSCP Exam Tips

OS Fingerprinting Tips

  1. Always include -O in your Nmap scans: OS information guides your entire attack strategy.

  2. Use --osscan-guess when needed: In the exam, any OS hint is valuable.

  3. Cross-verify with service banners: SSH and HTTP banners often reveal the OS directly.

  4. Check TTL values: A quick ping can give you immediate OS hints.

  5. Document OS findings: Record the OS in your notes for exploitation planning.

  6. Prepare both Linux and Windows tools: Have enumeration scripts ready for both OS types.

  7. Don’t over-rely on Nmap OS detection: It can be wrong. Use multiple indicators.

  8. SMB signatures are gold for Windows: crackmapexec smb often reveals the exact Windows version.

  9. SSH banners reveal Linux distros: The banner often contains the exact Ubuntu/Debian version.

  10. HTTP headers can reveal everything: Server and X-Powered-By headers are very informative.

Time Management

Activity Recommended Time
Nmap OS detection scan 2-5 minutes
Manual banner verification 3-5 minutes
TTL analysis 1-2 minutes
SMB/SSH banner checking 2-3 minutes
Total 8-15 minutes

Common Pitfalls

  • Trusting Nmap blindly: Always verify with manual techniques
  • Ignoring service banners: They often contain more accurate OS info
  • Not checking SMB: On Windows targets, SMB is the most reliable indicator
  • Assuming all Linux is the same: Ubuntu vs CentOS requires different approaches
  • Forgetting about virtualization: VMs may have unusual TCP stack behavior

Summary

OS fingerprinting is a critical reconnaissance step that informs your entire attack strategy. By combining:

  1. Active techniques (Nmap -O, TCP stack analysis)
  2. Passive techniques (service banners, HTTP headers, SMB signatures)
  3. TTL analysis (initial TTL values, hop calculation)
  4. Multi-indicator correlation (combining all sources)

You can accurately identify the target OS and select appropriate exploitation and post-exploitation techniques. In OSCP labs, Linux (Ubuntu) and Windows Server are the most common targets, so be prepared for both.


References