Domain 1Recon & ReportingCORE

Certificate Transparency Logs

Deep dive into Certificate Transparency (CT) logs as a powerful passive reconnaissance source for subdomain discovery, historical data extraction, and infrastructure mapping. Covers crt.sh, alternative CT sources, certificate parsing, wildcard handling, and building CT-driven attack workflows for OSCP.

Domain: Domain 1: Reconnaissance & Reporting•Updated: 2026-09-01
#certificate-transparency#crt-sh#ssl#subdomain-discovery

Certificate Transparency Logs

Attack Chain Position

graph LR
    A[Reconnaissance] --> B[CT Log Analysis]
    B --> C[Subdomain Discovery]
    C --> D[Port Scanning]
    D --> E[Service Enumeration]
    E --> F[Vulnerability Identification]
    F --> G[Exploitation]
    
    style B fill:#f9d56e,stroke:#333,stroke-width:2px

Position: Passive Reconnaissance Phase — CT logs are one of the most underutilized yet powerful sources for discovering subdomains, internal hostnames, and infrastructure details without generating any traffic to the target.


Table of Contents

  1. What Are Certificate Transparency Logs
  2. Why CT Logs Matter for Recon
  3. crt.sh Deep Dive
  4. Other CT Log Sources
  5. Certificate Parsing and Analysis
  6. Historical Certificate Analysis
  7. Wildcard Certificate Detection
  8. CT Log Monitoring Setup
  9. Correlating CT Data with Other Sources
  10. CT Logs in OSCP Context
  11. Building CT-Driven Subdomain Lists
  12. OSCP Exam Tips

What Are Certificate Transparency Logs

History and Background

Certificate Transparency (CT) was proposed by Google in 2013 as a response to several high-profile incidents where Certificate Authorities (CAs) issued certificates without proper authorization:

  • 2011 DigiNotar Incident: A Dutch CA was compromised, leading to the issuance of a fraudulent Google certificate that enabled man-in-the-middle attacks against Gmail users in Iran.
  • 2011 Comodo Breach: Hackers obtained a code-signing certificate from Comodo by compromising a reseller, leading to the issuance of unauthorized certificates for Google, Yahoo, and Microsoft.
  • 2012 GlobalSign Incident: A misconfigured system at GlobalSign allowed anyone to obtain a valid certificate for any domain.

These incidents highlighted a fundamental problem: there was no public, auditable record of which certificates had been issued by CAs. Domain owners had no way to detect if a rogue or mistakenly-issued certificate existed for their domain.

The CT Framework

Certificate Transparency is an open-framework designed to monitor and detect misissued or malicious TLS certificates. It was standardized as RFC 6962 (published in 2013) and has since become a mandatory requirement for all publicly-trusted CAs.

Key Components of the CT Framework:

Component Description
Certificate Authorities (CAs) Entities that issue TLS certificates. Must submit all publicly-trusted certificates to CT logs.
CT Logs Publicly-accessible, append-only registries of TLS certificates. Operated by various organizations (Google, Facebook, DigiCert, etc.).
CT Monitors Services that monitor CT logs for certificates related to specific domains. Used by domain owners to detect unauthorized issuance.
CT Auditors Verify the integrity and consistency of CT log data, ensuring logs cannot be tampered with.

How CT Works

sequenceDiagram
    participant CA as Certificate Authority
    participant CTL as CT Log Server
    participant M as CT Monitor
    participant O as Observer (Pentester)
    
    CA->>CTL: Submit certificate for inclusion
    CTL->>CTL: Assign SCT (Signed Certificate Timestamp)
    CTL-->>CA: Return SCT
    CA->>CA: Embed SCT in issued certificate
    Note over M,O: Anyone can query CT Logs
    M->>CTL: Monitor for domain certificates
    O->>CTL: Query for subdomain discovery
    CTL-->>O: Return all certificates for domain

The CT Log Submission Process:

  1. Certificate Issuance: When a CA issues a publicly-trusted TLS certificate, it must submit the certificate to multiple CT log servers.
  2. SCT Generation: Each CT log server generates a Signed Certificate Timestamp (SCT) — a cryptographic proof that the certificate was included in the log at a specific time.
  3. Certificate Delivery: The CA includes the SCT(s) in the issued certificate (via TLS extensions or OCSP stapling).
  4. Public Availability: The certificate becomes publicly queryable in the CT log, typically within minutes to hours of issuance.

Key Properties of CT Logs:

  • Append-only: Once a certificate is logged, it cannot be removed or modified.
  • Publicly accessible: Anyone can query CT logs without authentication.
  • Mandatory: All publicly-trusted CAs must submit certificates to CT logs (enforced by browser vendors).
  • Merkle Tree Structure: CT logs use Merkle trees to ensure data integrity and enable efficient auditing.

Types of Certificates in CT Logs

Certificate Type Description CT Logged?
Domain Validated (DV) Basic TLS certificates validating domain ownership Yes
Organization Validated (OV) Certificates with organization verification Yes
Extended Validation (EV) Highest level of identity verification Yes
Wildcard Certificates Cover *.example.com Yes
Self-Signed Certificates Not issued by a public CA No
Internal/Private CA Certs Issued by private/internal CAs No
Let’s Encrypt Certs Automated, free certificates Yes

Why CT Logs Matter for Recon

Subdomain Discovery

CT logs are arguably the single best passive source for subdomain discovery because:

  1. Every TLS-enabled service leaves a trace: Any subdomain that has ever had an SSL/TLS certificate issued for it will appear in CT logs.
  2. Historical data: Even if a subdomain is no longer active, its certificate remains in the logs indefinitely.
  3. No rate limiting: Unlike DNS enumeration tools, CT log queries don’t generate traffic to the target.
  4. High accuracy: Certificates are issued by real CAs for real services — low false positive rate compared to wordlist-based enumeration.

What You Can Discover

Data Type Example Value for Recon
Subdomains dev.example.com, staging.example.com Direct targets for further enumeration
Internal Hostnames internal.corp.example.com May reveal internal infrastructure patterns
Development Environments dev01.example.com, test.example.com Often less secured, easier entry points
Legacy Services old-portal.example.com May run outdated software with known vulnerabilities
API Endpoints api.example.com, api-v2.example.com Potential attack surface for web application testing
Admin Interfaces admin.example.com, manage.example.com High-value targets for authentication bypass
Cloud Infrastructure example.azurewebsites.net Reveals cloud provider usage
CDN/Proxy Services example.cloudfront.net Reveals infrastructure architecture

Historical Data Value

CT logs preserve a timeline of a target’s infrastructure:

2023-01-15: cert issued for dev.example.com
2023-03-22: cert issued for staging.example.com
2023-06-10: cert issued for api-v2.example.com
2023-09-05: cert issued for internal.corp.example.com
2024-01-18: cert issued for test-newapp.example.com
2024-04-30: cert issued for legacy.example.com

This timeline can reveal:

  • Infrastructure growth patterns — new services being deployed
  • Decommissioned services — subdomains that existed but may no longer be actively monitored
  • Development cycles — patterns in how the organization names and deploys environments
  • M&A activity — sudden appearance of new domain patterns

Internal Hostnames and Dev Environments

One of the most valuable aspects of CT logs is the discovery of non-production environments:

Production:    www.example.com, app.example.com
Staging:       staging.example.com, stage.example.com
Development:   dev.example.com, dev01.example.com, dev02.example.com
Testing:       test.example.com, qa.example.com, uat.example.com
Internal:      internal.example.com, intranet.example.com
Legacy:        old.example.com, legacy.example.com, v1.example.com

Why these matter:

  • Development and staging environments often have debug features enabled
  • They may run older software versions not yet patched
  • Authentication may be weaker or disabled entirely
  • They may expose internal APIs not available on production
  • They often share infrastructure or credentials with production

crt.sh Deep Dive

Overview

crt.sh (https://crt.sh) is the most popular and accessible CT log monitoring service. It aggregates data from multiple CT log servers and provides both a web interface and an API for querying certificate data.

Key Features:

  • Free and open-source (created by Farah Al Massri)
  • Aggregates data from all major CT log providers
  • Web interface with search and filtering
  • REST API for programmatic access
  • JSON output support
  • SAML and certificate details extraction

Web Interface Usage

  1. Navigate to https://crt.sh
  2. Enter the target domain in the search box
  3. Results display in a table format showing:
    • Not Before / Not After (validity period)
    • Certificate serial number
    • Certificate issuer
    • Domain Name (including SANs)
    • Certificate entry (downloadable)

Filtering Options

Filter Description Example
Include Minimized Show certificates with minimized/shortened entries Toggle on/off
Exclude Minimized Hide minimized entries Toggle on/off
Show Certificate View full certificate details Click “cert” link
Show CRL View Certificate Revocation List Click “crl” link
Show Revoked Include revoked certificates Toggle on/off

API Queries

Basic API Endpoint

https://crt.sh/?q=example.com&output=json

API Parameters

Parameter Description Example
q Query string (domain name) q=example.com
output Output format json, jsonl, csv
include_subdomains Include subdomain matches 1 or 0
ignore_wildcards Exclude wildcard certificates 1 or 0
pretty Pretty-print JSON output 1 or 0
delay Delay between queries (seconds) 5

Query Examples

Basic domain query (JSON output):

# Get all certificates for example.com and subdomains
curl -s "https://crt.sh/?q=example.com&output=json&include_subdomains=1" | \
  python3 -m json.tool

Query with pretty printing:

curl -s "https://crt.sh/?q=example.com&output=json&pretty=1" | \
  python3 -m json.tool

CSV output for spreadsheet analysis:

curl -s "https://crt.sh/?q=example.com&output=csv" > example_com_certs.csv

JSONL output (one JSON object per line, ideal for processing):

curl -s "https://crt.sh/?q=example.com&output=jsonl" > example_com_certs.jsonl

Extracting Unique Subdomains

Method 1: Using curl and grep

# Extract all unique domain names from crt.sh results
curl -s "https://crt.sh/?q=example.com&output=json&include_subdomains=1" | \
  grep -oP '"name_value"\s*:\s*"\K[^"]+' | \
  sort -u > subdomains_crtsh.txt

Method 2: Using Python

import json
import requests
import re

def extract_subdomains_from_crtsh(domain):
    """Extract unique subdomains from crt.sh for a given domain."""
    url = f"https://crt.sh/?q={domain}&output=json&include_subdomains=1"
    
    try:
        response = requests.get(url, timeout=30)
        response.raise_for_status()
        data = response.json()
        
        subdomains = set()
        for cert in data:
            name_value = cert.get('name_value', '')
            if name_value:
                # Handle wildcard certificates
                clean_name = name_value.lstrip('*.')
                subdomains.add(clean_name)
        
        return sorted(subdomains)
    
    except requests.exceptions.RequestException as e:
        print(f"Error querying crt.sh: {e}")
        return []

# Usage
domain = "example.com"
subdomains = extract_subdomains_from_crtsh(domain)
for sub in subdomains:
    print(sub)

Method 3: One-liner with jq

# Using jq to extract and deduplicate subdomains
curl -s "https://crt.sh/?q=example.com&output=json&include_subdomains=1" | \
  jq -r '.[].name_value' | \
  sed 's/\*\.//' | \
  sort -u > subdomains_crtsh.txt

Method 4: Comprehensive Extraction Script

#!/bin/bash
# crtsh_extractor.sh - Comprehensive crt.sh subdomain extractor

DOMAIN="${1:-example.com}"
OUTPUT_FILE="subdomains_${DOMAIN}.txt"

echo "[*] Querying crt.sh for ${DOMAIN}..."

# Query crt.sh and extract all name_values
curl -s "https://crt.sh/?q=${DOMAIN}&output=json&include_subdomains=1" | \
  python3 -c "
import sys, json

data = json.load(sys.stdin)
subdomains = set()

for cert in data:
    name = cert.get('name_value', '')
    if name:
        # Remove wildcard prefix
        name = name.lstrip('*.')
        # Only keep domains matching our target
        if name.endswith('${DOMAIN}'):
            subdomains.add(name)

for sub in sorted(subdomains):
    print(sub)
" > "$OUTPUT_FILE"

COUNT=$(wc -l < "$OUTPUT_FILE")
echo "[+] Found ${COUNT} unique subdomains"
echo "[+] Results saved to ${OUTPUT_FILE}"

Understanding crt.sh JSON Output

A typical crt.sh JSON response contains the following fields per certificate:

{
  "id": 1234567890,
  "serial_number": "03:AB:CD:EF:12:34:56:78",
  "score": 2,
  "modulus_publication_type": "",
  "leaf_certificates_in_chain": 1,
  "not_before": "2024-01-15T00:00:00.0000000Z",
  "not_after": "2024-04-15T23:59:59.0000000Z",
  "revoked": 0,
  "revocation_reason": "",
  "revocation_date": "",
  "crl_reason": "",
  "issuer_id": 98765,
  "issuer": {
    "C": "US",
    "O": "Let's Encrypt",
    "CN": "R10"
  },
  "entry_timestamp": "2024-01-15T12:30:45.0000000Z",
  "exact_timestamp": "2024-01-15T12:30:45.1234567Z",
  "parsed": {
    "subject": {
      "CN": "example.com"
    },
    "tbs": {
      "extensions": {
        "subject_alt_name": [
          "DNS:example.com",
          "DNS:www.example.com",
          "DNS:api.example.com"
        ]
      }
    }
  },
  "name_value": "api.example.com",
  "domain_name": "api.example.com",
  "common_name": "example.com",
  "cert_link": "https://crt.sh/?d=ABC123...",
  "crl_link": "",
  "ct_log": "Google Argon 2024",
  "log_id": "F4:25:3B:1D:6E:8A:9C:0D"
}

Key Fields for Recon:

Field Description Recon Value
name_value The specific domain this entry matches Direct subdomain discovery
not_before Certificate validity start date Timeline analysis
not_after Certificate validity end date Identify active vs expired services
issuer Certificate Authority information Identify CA patterns, Let’s Encrypt usage
parsed.tbs.extensions.subject_alt_name All SANs in the certificate Multiple subdomains per certificate
ct_log Which CT log recorded this entry Cross-reference with other sources
revoked Whether the certificate was revoked Identify decommissioned services

Wildcard Certificate Handling

Wildcard certificates (*.example.com) can both help and hinder subdomain discovery:

Challenges:

  • A single wildcard certificate can cover thousands of subdomains
  • crt.sh will show *.example.com as a single entry
  • You cannot determine which specific subdomains are actually in use

Strategies:

# 1. Separate wildcard from non-wildcard entries
curl -s "https://crt.sh/?q=example.com&output=json&include_subdomains=1" | \
  python3 -c "
import sys, json

data = json.load(sys.stdin)
wildcards = set()
specific = set()

for cert in data:
    name = cert.get('name_value', '')
    if name.startswith('*.'):
        wildcards.add(name)
    elif name:
        specific.add(name)

print('=== Wildcard Certificates ===')
for w in sorted(wildcards):
    print(w)

print('\n=== Specific Subdomains ===')
for s in sorted(specific):
    print(s)

print(f'\n[*] Wildcards: {len(wildcards)}, Specific: {len(specific)}')
"
# 2. For wildcard domains, combine with active subdomain enumeration
# First get the wildcard base domains
WILDCARD_BASES=$(curl -s "https://crt.sh/?q=example.com&output=json" | \
  grep -oP '"name_value"\s*:\s*"\*\.\K[^"]+' | sort -u)

# Then enumerate actual subdomains using other methods
for base in $WILDCARD_BASES; do
  echo "[*] Enumerating subdomains for: $base"
  # Use subfinder, amass, or gobuster DNS here
  subfinder -d "$base" >> all_subdomains.txt
done

Other CT Log Sources

Censys CT

Censys (https://censys.io) provides CT log data as part of its broader internet scanning platform.

Features:

  • CT log data combined with active internet scanning
  • Certificate details with associated IP addresses
  • Historical certificate data
  • API access (free tier available)

Querying Censys CT:

# Web interface: https://censys.io/certificates
# Search query: "names: *.example.com"

# API query (requires authentication)
curl -s -u "API_ID:API_SECRET" \
  "https://search.censys.io/api/v2/certificates/search?q=names:example.com&per_page=100" | \
  python3 -m json.tool

Censys Python API:

from censys.search import CensysCertSearch

# Initialize with API credentials
cert_search = CensysCertSearch(api_id="YOUR_ID", api_secret="YOUR_SECRET")

# Search for certificates
results = cert_search.search("names: example.com", fields=[
    "names",
    "parsed.subject_cn",
    "parsed.validity.not_before",
    "parsed.validity.not_after",
    "cert_integrity"
], per_page=100)

for cert in results:
    print(cert.get('names', []))

Google CT Transparency Report

Google maintains the CT Transparency Report (https://transparencyreport.google.com/ssl-certificate):

Features:

  • Visual dashboard of CT log coverage
  • Statistics on certificate issuance
  • Browser enforcement policies
  • CT log operator information

Usage for Recon:

  • Verify CT log coverage for a domain
  • Understand which CT logs are being used
  • Check for CT compliance issues

SecurityTrails

SecurityTrails (https://securitytrails.com) aggregates CT log data with DNS history and other sources.

Features:

  • Historical DNS records
  • Certificate data
  • WHOIS history
  • API access (paid)

API Query:

curl -s -H "Accept: application/json" \
  -H "APIKEY: YOUR_API_KEY" \
  "https://api.securitytrails.com/v1/domain/example.com/certificates" | \
  python3 -m json.tool

CertSpotter

CertSpotter (https://certspotter.com) by SSLMate provides a clean interface for CT log queries.

Features:

  • Simple web interface
  • Domain-based search
  • Certificate details view
  • API access

API Query:

curl -s "https://api.certspotter.com/v0/issuances?domain=example.com&include_subdomains=true&expand=dns_names" | \
  python3 -m json.tool

CT Log Source Comparison

Source Free Tier API Access Historical Data Ease of Use Best For
crt.sh Full Yes (no auth) Yes Medium Quick queries, scripting
Censys Limited Yes (auth) Yes Medium Combined cert + IP data
SecurityTrails Limited Yes (paid) Yes High Comprehensive DNS + cert data
CertSpotter Full Yes (no auth) Yes High Simple queries
Google CT Report Full No Yes High Statistics, compliance
Google Transparency Full No Yes High Visual dashboard

Certificate Parsing and Analysis

Extracting SANs (Subject Alternative Names)

SANs are the most valuable field in a certificate for subdomain discovery. A single certificate can contain multiple SANs:

# Extract SANs from a certificate file
openssl x509 -in certificate.pem -noout -text | \
  grep -A 1 "Subject Alternative Name"

Example SAN output:

X509v3 Subject Alternative Name:
    DNS:example.com, DNS:www.example.com, DNS:api.example.com,
    DNS:mail.example.com, DNS:vpn.example.com

Extracting Issuer Information

# Get issuer details
openssl x509 -in certificate.pem -noout -issuer

# Output example:
# issuer=C = US, O = Let's Encrypt, CN = R10

Common Issuers and Their Significance:

Issuer Significance
Let’s Encrypt Automated certificate management, likely web services
DigiCert Enterprise-grade, likely critical infrastructure
GlobalSign Enterprise-grade, common for corporate sites
Comodo/Sectigo Wide range of services, common for web hosting
Amazon RSA AWS infrastructure, likely cloud-hosted services
Self-Signed Internal services, potentially less secure

Validity Period Analysis

# Extract validity dates
openssl x509 -in certificate.pem -noout -dates

# Output:
# notBefore=Jan 15 00:00:00 2024 GMT
# notAfter=Apr 15 23:59:59 2024 GMT

Validity Period Insights:

Validity Duration Likely Scenario
90 days Let’s Encrypt (automated renewal)
1 year Standard commercial certificate
2 years Older commercial certificate
Very short (<30 days) Development/testing environment
Expired Decommissioned service or neglected renewal

Full Certificate Parsing Script

#!/usr/bin/env python3
"""
certificate_parser.py - Parse certificate data from crt.sh for recon analysis
"""

import json
import requests
import re
from datetime import datetime
from collections import defaultdict

def parse_crtsh_certificates(domain):
    """Parse and analyze certificates from crt.sh."""
    url = f"https://crt.sh/?q={domain}&output=json&include_subdomains=1"
    
    try:
        response = requests.get(url, timeout=30)
        response.raise_for_status()
        data = response.json()
        
        results = {
            'total_certificates': len(data),
            'unique_subdomains': set(),
            'issuers': defaultdict(int),
            'wildcards': set(),
            'expired': [],
            'active': [],
            'timeline': [],
            'sans_per_cert': [],
        }
        
        for cert in data:
            name = cert.get('name_value', '')
            if not name:
                continue
            
            # Extract subdomain
            clean_name = name.lstrip('*.')
            results['unique_subdomains'].add(clean_name)
            
            # Track wildcards
            if name.startswith('*.'):
                results['wildcards'].add(name)
            
            # Track issuers
            issuer = cert.get('issuer', {})
            issuer_cn = issuer.get('CN', 'Unknown')
            results['issuers'][issuer_cn] += 1
            
            # Validity analysis
            not_before = cert.get('not_before', '')
            not_after = cert.get('not_after', '')
            
            if not_after:
                try:
                    expiry = datetime.fromisoformat(not_after.replace('Z', '+00:00'))
                    now = datetime.now(expiry.tzinfo)
                    
                    if expiry < now:
                        results['expired'].append({
                            'domain': clean_name,
                            'expired': not_after,
                        })
                    else:
                        results['active'].append({
                            'domain': clean_name,
                            'expires': not_after,
                        })
                    
                    results['timeline'].append({
                        'domain': clean_name,
                        'issued': not_before,
                        'expires': not_after,
                    })
                except ValueError:
                    pass
            
            # Extract SANs from parsed data
            parsed = cert.get('parsed', {})
            tbs = parsed.get('tbs', {})
            extensions = tbs.get('extensions', {})
            sans = extensions.get('subject_alt_name', [])
            if sans:
                results['sans_per_cert'].append({
                    'common_name': cert.get('common_name', ''),
                    'sans': sans,
                })
        
        # Convert sets to sorted lists for output
        results['unique_subdomains'] = sorted(results['unique_subdomains'])
        results['wildcards'] = sorted(results['wildcards'])
        
        return results
    
    except requests.exceptions.RequestException as e:
        print(f"Error: {e}")
        return None

def print_analysis(domain, results):
    """Print formatted analysis results."""
    if not results:
        return
    
    print(f"\n{'='*60}")
    print(f"Certificate Transparency Analysis: {domain}")
    print(f"{'='*60}")
    
    print(f"\n[*] Total Certificates Found: {results['total_certificates']}")
    print(f"[*] Unique Subdomains: {len(results['unique_subdomains'])}")
    print(f"[*] Wildcard Certificates: {len(results['wildcards'])}")
    print(f"[*] Active Certificates: {len(results['active'])}")
    print(f"[*] Expired Certificates: {len(results['expired'])}")
    
    print(f"\n--- Issuers ---")
    for issuer, count in sorted(results['issuers'].items(), key=lambda x: x[1], reverse=True):
        print(f"  {issuer}: {count}")
    
    print(f"\n--- Wildcard Certificates ---")
    for wc in results['wildcards']:
        print(f"  {wc}")
    
    print(f"\n--- Unique Subdomains ---")
    for sub in results['unique_subdomains']:
        print(f"  {sub}")
    
    print(f"\n--- Recent Certificates (last 30 days) ---")
    now = datetime.now()
    for entry in results['timeline'][-10:]:
        print(f"  {entry['domain']} | Issued: {entry['issued'][:10]} | Expires: {entry['expires'][:10]}")

if __name__ == "__main__":
    import sys
    domain = sys.argv[1] if len(sys.argv) > 1 else "example.com"
    results = parse_crtsh_certificates(domain)
    print_analysis(domain, results)

Historical Certificate Analysis

Why Historical Analysis Matters

Historical certificate data reveals the evolution of a target’s infrastructure:

  1. Decommissioned Services: Subdomains that had certificates but no longer do may still be accessible or may have been forgotten.
  2. Infrastructure Changes: Patterns in certificate issuance can reveal migration patterns, cloud adoption, or rebranding.
  3. Security Posture Changes: Shifts in CA usage, validity periods, or certificate types can indicate changes in security practices.
  4. M&A Indicators: Sudden appearance of certificates for new domain patterns may indicate acquisitions.

Timeline Analysis

# Create a timeline of certificate issuances
curl -s "https://crt.sh/?q=example.com&output=json&include_subdomains=1" | \
  python3 -c "
import sys, json
from datetime import datetime

data = json.load(sys.stdin)
timeline = []

for cert in data:
    name = cert.get('name_value', '').lstrip('*.')
    issued = cert.get('not_before', '')
    if issued and name:
        timeline.append((issued[:10], name))

# Sort by date
timeline.sort()

# Group by month
from collections import defaultdict
by_month = defaultdict(list)
for date, name in timeline:
    month = date[:7]  # YYYY-MM
    by_month[month].append(name)

for month in sorted(by_month.keys()):
    subs = sorted(set(by_month[month]))
    print(f'{month}: {len(subs)} unique subdomains')
    for sub in subs[:5]:  # Show first 5
        print(f'  - {sub}')
    if len(subs) > 5:
        print(f'  ... and {len(subs) - 5} more')
"

Identifying Decommissioned Services

def find_decommissioned_services(domain):
    """Find subdomains with expired certificates that may still be accessible."""
    url = f"https://crt.sh/?q={domain}&output=json&include_subdomains=1"
    response = requests.get(url, timeout=30)
    data = response.json()
    
    now = datetime.now()
    decommissioned = []
    
    for cert in data:
        name = cert.get('name_value', '').lstrip('*.')
        not_after = cert.get('not_after', '')
        
        if not_after:
            try:
                expiry = datetime.fromisoformat(not_after.replace('Z', '+00:00'))
                # Certificate expired more than 30 days ago
                if (now - expiry).days > 30:
                    decommissioned.append({
                        'subdomain': name,
                        'expired': not_after[:10],
                        'days_ago': (now - expiry).days,
                    })
            except ValueError:
                pass
    
    # Sort by most recently expired
    decommissioned.sort(key=lambda x: x['days_ago'])
    return decommissioned

Wildcard Certificate Detection

Identifying Wildcard Certificates

# Find all wildcard certificates for a domain
curl -s "https://crt.sh/?q=example.com&output=json" | \
  python3 -c "
import sys, json

data = json.load(sys.stdin)
wildcards = []

for cert in data:
    name = cert.get('name_value', '')
    if name.startswith('*.'):
        wildcards.append({
            'domain': name,
            'issuer': cert.get('issuer', {}).get('CN', 'Unknown'),
            'issued': cert.get('not_before', '')[:10],
            'expires': cert.get('not_after', '')[:10],
        })

for wc in wildcards:
    print(f\"{wc['domain']} | {wc['issuer']} | {wc['issued']} - {wc['expires']}\")
"

Wildcard Impact Assessment

Scenario Impact on Recon Mitigation
Single wildcard (*.example.com) All subdomains covered by one cert Combine with active enumeration
Multiple wildcards (*.dev.example.com, *.staging.example.com) Reveals environment structure Enumerate each wildcard base
No wildcards Each subdomain has individual cert Maximum subdomain discovery from CT
Wildcard + specific Mixed coverage Filter wildcards, focus on specific entries

Strategy for Wildcard Domains

# Step 1: Identify wildcard bases
WILDCARD_BASES=$(curl -s "https://crt.sh/?q=example.com&output=json" | \
  grep -oP '"name_value"\s*:\s*"\*\.\K[^"]+' | sort -u)

# Step 2: For each wildcard base, enumerate actual subdomains
for base in $WILDCARD_BASES; do
  echo "[*] Enumerating subdomains for wildcard base: $base"
  
  # Use subfinder for passive enumeration
  subfinder -d "$base" -silent >> "${base}_subdomains.txt"
  
  # Use amass for deeper enumeration
  amass enum -passive -d "$base" -o "${base}_amass.txt"
  
  # Combine and deduplicate
  cat "${base}_subdomains.txt" "${base}_amass.txt" | sort -u > "${base}_all.txt"
done

CT Log Monitoring Setup

Automated Monitoring Script

#!/usr/bin/env python3
"""
ct_monitor.py - Monitor CT logs for new certificates on target domains
"""

import json
import requests
import time
import sqlite3
import argparse
from datetime import datetime

class CTMonitor:
    def __init__(self, db_path="ct_monitor.db"):
        self.db_path = db_path
        self.init_db()
    
    def init_db(self):
        """Initialize SQLite database for tracking certificates."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        cursor.execute('''
            CREATE TABLE IF NOT EXISTS certificates (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                domain TEXT,
                name_value TEXT,
                not_before TEXT,
                not_after TEXT,
                issuer_cn TEXT,
                first_seen TEXT,
                ct_log TEXT
            )
        ''')
        conn.commit()
        conn.close()
    
    def query_crtsh(self, domain):
        """Query crt.sh for certificates."""
        url = f"https://crt.sh/?q={domain}&output=json&include_subdomains=1"
        response = requests.get(url, timeout=30)
        return response.json()
    
    def get_known_certs(self, domain):
        """Get known certificates from database."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        cursor.execute(
            "SELECT name_value FROM certificates WHERE domain = ?",
            (domain,)
        )
        return {row[0] for row in cursor.fetchall()}
    
    def save_cert(self, domain, cert):
        """Save certificate to database."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        cursor.execute('''
            INSERT INTO certificates 
            (domain, name_value, not_before, not_after, issuer_cn, first_seen, ct_log)
            VALUES (?, ?, ?, ?, ?, ?, ?)
        ''', (
            domain,
            cert.get('name_value', ''),
            cert.get('not_before', ''),
            cert.get('not_after', ''),
            cert.get('issuer', {}).get('CN', ''),
            datetime.now().isoformat(),
            cert.get('ct_log', '')
        ))
        conn.commit()
        conn.close()
    
    def check_new_certs(self, domain):
        """Check for new certificates since last check."""
        print(f"[*] Checking CT logs for {domain}...")
        
        known = self.get_known_certs(domain)
        certs = self.query_crtsh(domain)
        
        new_certs = []
        for cert in certs:
            name = cert.get('name_value', '')
            if name and name not in known:
                new_certs.append(cert)
                self.save_cert(domain, cert)
                print(f"[!] NEW CERTIFICATE: {name}")
                print(f"    Issuer: {cert.get('issuer', {}).get('CN', 'Unknown')}")
                print(f"    Issued: {cert.get('not_before', '')[:10]}")
                print(f"    Expires: {cert.get('not_after', '')[:10]}")
        
        if not new_certs:
            print("[+] No new certificates found")
        
        return new_certs

def main():
    parser = argparse.ArgumentParser(description="CT Log Monitor")
    parser.add_argument("domain", help="Domain to monitor")
    parser.add_argument("--interval", type=int, default=3600,
                       help="Check interval in seconds (default: 3600)")
    parser.add_argument("--once", action="store_true",
                       help="Run once and exit")
    
    args = parser.parse_args()
    monitor = CTMonitor()
    
    if args.once:
        monitor.check_new_certs(args.domain)
    else:
        print(f"[*] Monitoring {args.domain} every {args.interval} seconds...")
        while True:
            monitor.check_new_certs(args.domain)
            time.sleep(args.interval)

if __name__ == "__main__":
    main()

Cron-Based Monitoring

# Add to crontab for daily monitoring
0 8 * * * /usr/bin/python3 /opt/tools/ct_monitor.py example.com --once >> /var/log/ct_monitor.log 2>&1

# For multiple domains
0 8 * * * for domain in example.com target.org another.net; do
  /usr/bin/python3 /opt/tools/ct_monitor.py "$domain" --once
done >> /var/log/ct_monitor.log 2>&1

Correlating CT Data with Other Sources

Cross-Reference Strategy

graph TD
    A[CT Logs] --> D[Correlation Engine]
    B[DNS Enumeration] --> D
    C[Search Engine Dorking] --> D
    E[Shodan/Censys] --> D
    D --> F[Unified Subdomain List]
    F --> G[Priority Scoring]
    G --> H[Active Targets]
    
    style D fill:#f9d56e,stroke:#333,stroke-width:2px

Correlation Script

def correlate_sources(domain):
    """Correlate CT data with other recon sources."""
    results = {
        'ct_only': set(),
        'dns_only': set(),
        'both': set(),
        'ct_with_live_dns': set(),
    }
    
    # Get CT subdomains
    ct_subs = set(extract_subdomains_from_crtsh(domain))
    
    # Get DNS subdomains (from subfinder, amass, etc.)
    dns_subs = set()
    with open(f"{domain}_dns_subs.txt", 'r') as f:
        dns_subs = {line.strip() for line in f if line.strip()}
    
    # Correlate
    results['ct_only'] = ct_subs - dns_subs
    results['dns_only'] = dns_subs - ct_subs
    results['both'] = ct_subs & dns_subs
    
    # Check which CT-only subdomains have live DNS
    import subprocess
    for sub in results['ct_only']:
        try:
            result = subprocess.run(
                ['host', sub],
                capture_output=True, text=True, timeout=5
            )
            if result.returncode == 0 and 'has address' in result.stdout:
                results['ct_with_live_dns'].add(sub)
        except (subprocess.TimeoutExpired, Exception):
            pass
    
    return results

Priority Scoring

Source Match Priority Reasoning
CT + DNS + Live Critical Active service with certificate
CT + DNS High Registered subdomain, may be active
CT Only Medium Historical, may still be accessible
DNS Only Medium Active DNS, no cert (HTTP service?)
CT + Shodan High Active service with banner data

CT Logs in OSCP Context

OSCP Relevance Assessment

OSCP Scenario CT Log Value Notes
External Pentest High Primary subdomain discovery source
Internal Pentest Low-Medium Internal CAs not in public CT logs
Web App Focus High Discover web-facing subdomains
Infrastructure Focus Medium Supplemental to DNS enumeration
Time-Constrained High Fast, passive, no rate limiting

OSCP-Specific Workflow

# OSCP CT Log Quick Workflow

# Step 1: Quick subdomain extraction
DOMAIN="target.oscp.com"
curl -s "https://crt.sh/?q=${DOMAIN}&output=json&include_subdomains=1" | \
  grep -oP '"name_value"\s*:\s*"\K[^"]+' | \
  sed 's/\*\.//' | \
  sort -u > ct_subdomains.txt

# Step 2: Filter for target domain only
grep "${DOMAIN}$" ct_subdomains.txt > ct_target_subdomains.txt

# Step 3: Resolve DNS for live targets
while read sub; do
  host "$sub" 2>/dev/null | grep "has address" | awk '{print $4}'
done < ct_target_subdomains.txt > ct_resolved_ips.txt

# Step 4: Quick port scan on resolved IPs
nmap -sV -p 80,443,8080,8443 -iL ct_resolved_ips.txt -oA ct_nmap_quick

Limitations in OSCP

  1. Internal CAs: OSCP labs often use internal/private CAs that don’t submit to public CT logs.
  2. Self-Signed Certs: Many lab services use self-signed certificates.
  3. No Internet Access: Some lab environments don’t have external internet access.
  4. Time Factor: CT logs are most valuable for external assessments with real-world targets.

Bottom Line: CT logs are a valuable supplemental technique for OSCP. Use them when available, but don’t rely on them as your primary subdomain source for lab exams.


Building CT-Driven Subdomain Lists

Complete Workflow Script

#!/bin/bash
# ct_recon_pipeline.sh - Complete CT-driven reconnaissance pipeline

DOMAIN="${1:-example.com}"
OUTPUT_DIR="ct_recon_${DOMAIN}"
mkdir -p "$OUTPUT_DIR"

echo "[*] Starting CT-driven reconnaissance for ${DOMAIN}"
echo "[*] Output directory: ${OUTPUT_DIR}"

# Step 1: Query crt.sh
echo "[1/6] Querying crt.sh..."
curl -s "https://crt.sh/?q=${DOMAIN}&output=json&include_subdomains=1" > \
  "${OUTPUT_DIR}/raw_certs.json"

# Step 2: Extract subdomains
echo "[2/6] Extracting subdomains..."
python3 -c "
import json, sys

with open('${OUTPUT_DIR}/raw_certs.json', 'r') as f:
    data = json.load(f)

subdomains = set()
for cert in data:
    name = cert.get('name_value', '')
    if name:
        name = name.lstrip('*.')
        if name.endswith('${DOMAIN}'):
            subdomains.add(name)

with open('${OUTPUT_DIR}/subdomains.txt', 'w') as f:
    for sub in sorted(subdomains):
        f.write(sub + '\n')

print(f'[*] Found {len(subdomains)} unique subdomains')
"

# Step 3: Separate wildcards
echo "[3/6] Identifying wildcard certificates..."
python3 -c "
import json

with open('${OUTPUT_DIR}/raw_certs.json', 'r') as f:
    data = json.load(f)

wildcards = set()
for cert in data:
    name = cert.get('name_value', '')
    if name.startswith('*.'):
        wildcards.add(name)

with open('${OUTPUT_DIR}/wildcards.txt', 'w') as f:
    for wc in sorted(wildcards):
        f.write(wc + '\n')

print(f'[*] Found {len(wildcards)} wildcard certificates')
"

# Step 4: DNS resolution
echo "[4/6] Resolving DNS for subdomains..."
while IFS= read -r sub; do
  ip=$(host "$sub" 2>/dev/null | grep "has address" | awk '{print $4}' | head -1)
  if [ -n "$ip" ]; then
    echo "${sub} ${ip}" >> "${OUTPUT_DIR}/resolved.txt"
  fi
done < "${OUTPUT_DIR}/subdomains.txt"

# Step 5: Quick HTTP check
echo "[5/6] Checking HTTP services..."
if [ -f "${OUTPUT_DIR}/resolved.txt" ]; then
  while IFS= read -r line; do
    sub=$(echo "$line" | awk '{print $1}')
    ip=$(echo "$line" | awk '{print $2}')
    
    # Quick HTTP check
    status=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 3 \
      "http://${sub}" 2>/dev/null)
    
    if [ "$status" != "000" ]; then
      echo "${sub} ${ip} HTTP:${status}" >> "${OUTPUT_DIR}/http_services.txt"
    fi
    
    # HTTPS check
    status_ssl=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 3 -k \
      "https://${sub}" 2>/dev/null)
    
    if [ "$status_ssl" != "000" ]; then
      echo "${sub} ${ip} HTTPS:${status_ssl}" >> "${OUTPUT_DIR}/https_services.txt"
    fi
  done < "${OUTPUT_DIR}/resolved.txt"
fi

# Step 6: Summary report
echo "[6/6] Generating summary report..."
cat > "${OUTPUT_DIR}/SUMMARY.md" << EOF
# CT Reconnaissance Report: ${DOMAIN}

## Summary
- **Total Subdomains Found:** $(wc -l < "${OUTPUT_DIR}/subdomains.txt")
- **Wildcard Certificates:** $(wc -l < "${OUTPUT_DIR}/wildcards.txt" 2>/dev/null || echo 0)
- **Resolved IPs:** $(wc -l < "${OUTPUT_DIR}/resolved.txt" 2>/dev/null || echo 0)
- **HTTP Services:** $(wc -l < "${OUTPUT_DIR}/http_services.txt" 2>/dev/null || echo 0)
- **HTTPS Services:** $(wc -l < "${OUTPUT_DIR}/https_services.txt" 2>/dev/null || echo 0)

## Top-Level Subdomains
$(head -20 "${OUTPUT_DIR}/subdomains.txt")

## Wildcard Certificates
$(cat "${OUTPUT_DIR}/wildcards.txt" 2>/dev/null || echo "None found")

## Active HTTP Services
$(cat "${OUTPUT_DIR}/http_services.txt" 2>/dev/null || echo "None found")

## Active HTTPS Services
$(cat "${OUTPUT_DIR}/https_services.txt" 2>/dev/null || echo "None found")
EOF

echo "[+] Complete! Results in ${OUTPUT_DIR}/"
echo "[+] See ${OUTPUT_DIR}/SUMMARY.md for overview"

Integrating with Other Tools

# Combine CT results with subfinder and amass
cat ct_recon_*.txt subfinder_*.txt amass_*.txt | sort -u > all_subdomains_combined.txt

# Use the combined list for further enumeration
gobuster dns -d example.com -w all_subdomains_combined.txt -o gobuster_results.txt

# Or feed into nmap
nmap -sV -sC -iL all_subdomains_combined.txt -oA nmap_full_scan

OSCP Exam Tips

Key Takeaways for OSCP

┌─────────────────────────────────────────────────────────────────┐
│                    OSCP CT Log Tips                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  1. CT logs are PASSIVE - no traffic to target                  │
│     Use them early in recon to build subdomain lists            │
│                                                                 │
│  2. crt.sh is the easiest source - no API key needed            │
│     Bookmark: https://crt.sh                                    │
│                                                                 │
│  3. Focus on the "name_value" field for subdomain extraction    │
│     This contains the actual domain/SAN entries                 │
│                                                                 │
│  4. Wildcard certs limit discovery value                        │
│     If you see *.target.com, combine with active enumeration    │
│                                                                 │
│  5. CT logs may not work in all OSCP labs                       │
│     Internal CAs and self-signed certs won't appear             │
│     Always have backup subdomain discovery methods              │
│                                                                 │
│  6. Quick one-liner for exam day:                               │
│     curl -s "https://crt.sh/?q=TARGET&output=json" |           │
│       grep -oP '"name_value"\s*:\s*"\K[^"]+' | sort -u         │
│                                                                 │
│  7. Cross-reference CT results with DNS enumeration             │
│     Subdomains in CT + live DNS = high priority targets         │
│                                                                 │
│  8. Don't spend too much time on CT logs                        │
│     5-10 minutes max per target domain                          │
│     Move on to active scanning after initial discovery          │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Common Mistakes to Avoid

Mistake Consequence Fix
Ignoring wildcard certificates Missing entire subdomain trees Always check for *.domain entries
Not filtering by target domain Too much noise in results Filter results to match target domain
Relying solely on CT logs Missing non-TLS services Combine with DNS enumeration and search engines
Not checking SANs Missing subdomains in multi-domain certs Parse subject_alt_name field
Skipping historical data Missing decommissioned services Check expired certificates too

Quick Reference Commands

# Quick subdomain extraction (exam-ready)
curl -s "https://crt.sh/?q=TARGET&output=json&include_subdomains=1" | \
  grep -oP '"name_value"\s*:\s*"\K[^"]+' | \
  sed 's/\*\.//' | sort -u

# Quick wildcard check
curl -s "https://crt.sh/?q=TARGET&output=json" | \
  grep -oP '"name_value"\s*:\s*"\*\.[^"]+'

# Quick issuer analysis
curl -s "https://crt.sh/?q=TARGET&output=json" | \
  grep -oP '"CN"\s*:\s*"\K[^"]+' | sort | uniq -c | sort -rn

Summary

Certificate Transparency logs represent one of the most powerful passive reconnaissance techniques available. By leveraging crt.sh and other CT sources, you can:

  1. Discover subdomains that have ever had TLS certificates issued for them
  2. Map infrastructure through historical certificate analysis
  3. Identify development and staging environments that may be less secure
  4. Build comprehensive subdomain lists without generating any traffic to the target
  5. Correlate with other sources to prioritize active targets

While CT logs may not always be applicable in OSCP lab environments (due to internal CAs and self-signed certificates), they are an essential technique for real-world engagements and external assessments. Master this technique as a supplement to your primary subdomain discovery methods.

Next Steps: After gathering subdomains from CT logs, proceed to:

  • DNS resolution and validation
  • Port scanning on discovered hosts
  • Service enumeration on live services
  • Web application reconnaissance on HTTP/HTTPS services