Domain 9Security & PKIINTRO

(Moyogalpa) Troubleshooting Golang Web App HTTPS, PKI Trust, POSIX Permissions, and AppArmor MAC Restrictions

A comprehensive post-mortem guide detailing how to troubleshoot process startup failures, TLS certificate validation errors, POSIX directory permissions, and AppArmor Mandatory Access Control policies on a Golang HTTPS web application.

Domain: Domain 9: Security & PKI•Updated: 2026-11-02
#golang#https#tls#pki#apparmor#posix-permissions#systemd#linux-troubleshooting#openssl#curl

Overview

In modern Linux infrastructure, deploying secure web services requires multi-layered coordination across system daemons, Public Key Infrastructure (PKI), Discretionary Access Control (DAC), and kernel Mandatory Access Control (MAC). When security hardening goes wrong, services often fail silently or emit opaque HTTP status codes like 403 Forbidden or TCP connection refusals.

This technical post-mortem analyzes a hands-on troubleshooting exercise involving a Golang web application hosted on Ubuntu Linux. After security measures were introduced, the application stopped serving client requests. This article documents the step-by-step diagnostic journey required to fix the application, restore HTTPS communication over port 7000, establish proper PKI trust without ignoring SSL checks, adjust POSIX file permissions, and reconfigure an restricting AppArmor profile.

Scope & Core Tooling Summary

  • Domain: Linux Systems Administration & Application Security
  • Services Involved: Custom Golang Web Application (webapp.service), AppArmor MAC Module (apparmor), systemd (systemd-journald)
  • Primary Utilities: journalctl, openssl, apparmor_parser, update-ca-certificates, chmod, chown, curl, ps, find
  • Core Objective: Enable clients to fetch static web content securely over HTTPS (curl https://webapp:7000/users.html) without using insecure flags (e.g., -k / --insecure).

Problem Description

Developers Mary and John were tasked with implementing HTTPS security and static file serving for a Golang web application running on port 7000. Following their changes, the application broke completely and stopped responding to incoming requests.

Client Request: curl https://webapp:7000/users.html
                    │
                    ▼
     [ Target Server: webapp:7000 ]
                    │
      ┌─────────────┴─────────────┐
      ▼                           ▼
Initial State:              Subsequent State:
Connection Refused          HTTP 403 Forbidden

Initial Constraints & Verification Requirements

  • Target Endpoint: https://webapp:7000/users.html
  • TLS Security Requirement: Clients must perform full TLS validation without bypassing SSL checks (i.e., curl must execute without -k or --insecure).
  • Content Requirement: The application must correctly serve static files stored on the filesystem.
  • Administrative Privileges: Root (sudo) access is available.
  • Success Criteria: Executing curl https://webapp:7000/users.html returns HTTP 200 OK with the HTML content of /home/webapp/static-files/users.html, and /home/admin/agent/check.sh returns OK.

Initial Investigation and Troubleshooting Strategy

A structured troubleshooting methodology is essential when diagnosing broken web services. When an application fails after security updates, issues typically span four distinct system layers:

┌────────────────────────────────────────────────────────────────────────┐
│                      Troubleshooting Layer Stack                       │
├────────────────────────────────────────────────────────────────────────┤
│ 1. Process & Daemon Layer │ Is the service active & listening on TCP?  │
├───────────────────────────┼────────────────────────────────────────────┤
│ 2. PKI & TLS Layer        │ Are certs valid, trusted, and matching SAN?│
├───────────────────────────┼────────────────────────────────────────────┤
│ 3. POSIX Permissions (DAC)│ Can service user read files & traverse dirs│
├───────────────────────────┼────────────────────────────────────────────┤
│ 4. Kernel Security (MAC)  │ Is AppArmor or SELinux blocking syscalls?  │
└───────────────────────────┴────────────────────────────────────────────┘

Investigation Sequence

  1. Process State Verification: Inspect whether the binary process is running and bound to TCP port 7000.
  2. System Log Inspection: Review systemd journal logs (journalctl) to catch runtime exceptions, file access errors, or panic traces.
  3. PKI & Certificate Validation: Examine the server’s TLS certificate chain using OpenSSL tools, ensure the local system trusts the issuing Certificate Authority (CA), and confirm hostname resolution (/etc/hosts).
  4. POSIX Permission Analysis: Verify file ownership (chown) and access modes (chmod) for the service account across all binary, PKI, and static document paths.
  5. Mandatory Access Control Inspection: Check kernel security modules (AppArmor) for rule violations restricting process capabilities or file access paths.

Step-by-Step Troubleshooting

Step 1: Initial System Discovery and Process Inspection

Objective

Discover running processes, locate the web application source/static directory, locate TLS certificates, and test initial network connectivity.

Commands Executed

ps aux | grep -i go
find / -name ".go" 2>/dev/null
find / -name "users.html" 2>/dev/null
find / -name "*.crt" -o -name "*.pem" -o -name "*.key" 2>/dev/null
grep -q "webapp" /etc/hosts || echo "127.0.0.1 webapp" | sudo tee -a /etc/hosts
curl https://webapp:7000/users.html

Command Breakdown

  • ps aux | grep -i go: Lists all active system processes and filters case-insensitively for go.
  • find / -name ".go" 2>/dev/null: Searches the root filesystem for Go source files while suppressing permission errors.
  • find / -name "users.html" 2>/dev/null: Locates the static HTML target file across the filesystem.
  • find / -name "*.crt" -o -name "*.pem" -o -name "*.key" 2>/dev/null: Searches for certificate and key files.
  • curl https://webapp:7000/users.html: Attempts an HTTPS request to the web application endpoint.

Observed Results

  1. ps aux | grep -i go returned only /usr/local/gotty and grep. No Go web application binary was visibly running in the process table.
  2. find / -name ".go" returned no results. (Note: The search string .go was typed instead of *.go).
  3. find / -name "users.html" identified the static file path: /home/webapp/static-files/users.html.
  4. find / -name "*.crt" ... listed standard system root certificates under /etc/ssl/certs/ and /usr/share/ca-certificates/, but did not list custom certificates under /home/webapp/pki/ because the user lacked permission to enter the directory.
  5. curl https://webapp:7000/users.html failed with:
curl: (7) Failed to connect to webapp port 7000: Connection refused

Interpretation

The Connection refused error confirmed that no process was bound to listening port 7000. The service was either crashing on startup or failing to launch.


Step 2: Diagnosing Process Startup Failures via Journalctl

Objective

Examine the systemd daemon log for webapp.service to determine why the service failed to start.

Command Executed

sudo journalctl -u webapp -ef

Command Breakdown

  • journalctl: Queries the systemd journal database.
  • -u webapp: Restricts output to the webapp.service systemd unit.
  • -e: Jumps directly to the end of the journal log.
  • -f: Follows live log entries as they are generated.

Observed Result

Apr 10 01:15:58 ip-10-1-13-82 systemd[1]: Started Webapp.
Apr 10 01:15:58 ip-10-1-13-82 webapp[3423]: 2024/04/10 01:15:58 open /home/webapp/pki/server.crt: permission denied
Apr 10 01:15:58 ip-10-1-13-82 webapp[3423]: 2024/04/10 01:15:58 open /home/webapp/pki/server.pem: permission denied
Apr 10 01:15:58 ip-10-1-13-82 webapp[3423]: 2024/04/10 01:15:58 can not access certificate/key file. sleeping for 10s and will retry

Interpretation

The systemd service webapp.service started, but the underlying application daemon was stuck in an infinite error loop. It was failing to open /home/webapp/pki/server.crt and /home/webapp/pki/server.pem due to POSIX Permission denied errors. Because it could not read its TLS certificate and private key, it was sleeping 10 seconds and retrying without opening listening port 7000.


Step 3: Resolving PKI Permission Denied Errors

Objective

Inspect ownership and permission bits on /home/webapp/pki/, fix ownership and permissions for the service user, update system CA trust, and restart webapp.service.

Commands Executed

ls -la /home/webapp/pki/
sudo ls -la /home/webapp/pki/
systemctl show -p User webapp
sudo chown -R webapp:webapp /home/webapp/pki/
sudo chmod 755 /home/webapp/pki/
sudo chmod 644 /home/webapp/pki/server.crt
sudo chmod 640 /home/webapp/pki/server.pem
sudo cp /home/webapp/pki/server.crt /usr/local/share/ca-certificates/webapp.crt
sudo update-ca-certificates
sudo systemctl restart webapp
sudo journalctl -u webapp -n 10 --no-pager

Command Breakdown

  • systemctl show -p User webapp: Queries systemd for the execution user assigned to webapp.service (User=webapp).
  • sudo chown -R webapp:webapp /home/webapp/pki/: Recursively sets the owner and group of the PKI directory to webapp.
  • sudo chmod 755 /home/webapp/pki/: Sets directory permissions to rwxr-xr-x (allowing owner full access, and others read/traverse access).
  • sudo chmod 644 /home/webapp/pki/server.crt: Sets server certificate permissions to rw-r--r--.
  • sudo chmod 640 /home/webapp/pki/server.pem: Sets TLS private key permissions to rw-r-----.
  • sudo cp ... /usr/local/share/ca-certificates/: Copies the certificate into the system local CA directory.
  • sudo update-ca-certificates: Rebuilds /etc/ssl/certs/ca-certificates.crt to register new CA files.

Observed Results

  • ls -la /home/webapp/pki/ failed initially with Permission denied.
  • sudo ls -la /home/webapp/pki/ revealed:
drwx------ 2 root  root  4096 Apr 10  2024 .
-rw-r----- 1 admin admin 1870 Apr 10  2024 CA.crt
-rw-r----- 1 admin admin 1927 Apr 10  2024 server.crt
-rw-r----- 1 admin admin 3247 Apr 10  2024 server.pem

The directory was locked down to root:root with 0700 (drwx------), and files were owned by admin:admin. Service user webapp had zero access.

  • After applying chown and chmod, sudo systemctl restart webapp was executed.
  • journalctl confirmed successful startup:
Nov 01 15:01:40 ip-10-1-13-82 webapp[2532]: 2026/11/01 15:01:40 Starting webserver, listening on port 7000

Interpretation

Fixing the PKI permissions allowed the webapp binary to read its TLS assets. The Go application successfully bound to TCP port 7000.


Step 4: Investigating HTTP 403 Forbidden Errors (POSIX Permissions)

Objective

Test the HTTP endpoint and fix permissions on static directory files when curl returns 403 Forbidden.

Commands Executed

curl https://webapp:7000/users.html
ls -lah /home/webapp/
ls -lah /home/webapp/static-files/
sudo chown -Rv webapp.webapp /home/webapp/static-files
sudo chmod -Rv 0640 /home/webapp/static-files/*
sudo chmod 0755 /home/webapp
sudo chmod 0755 /home/webapp/static-files
sudo systemctl restart webapp
curl https://webapp:7000/users.html

Observed Results

  • curl https://webapp:7000/users.html returned Forbidden.
  • ls -lah /home/webapp/static-files/ showed:
drwxr-xr-x 2 admin admin 4.0K Apr 10  2024 .
-rwxr-x--- 1 admin admin   78 Apr 10  2024 users.html

The static files were owned by admin:admin with -rwxr-x---. User webapp was not in the admin group and could not read users.html.

  • Even after setting ownership to webapp:webapp, files to 0640, and parent directories to 0755, curl https://webapp:7000/users.html **still returned Forbidden**.

Interpretation

POSIX permissions on /home/webapp/static-files/users.html were now fully accessible by user webapp. However, the 403 Forbidden error persisted, indicating that another security subsystem (outside standard POSIX permissions) was blocking access.


Step 5: Validating Private CA Trust and SAN Hostname Resolution

Objective

Ensure the client (curl) trusts the application’s root CA certificate, verify the Subject Alternative Name (SAN) of the certificate, and update local DNS resolution in /etc/hosts.

Commands Executed

sudo cp -v /home/webapp/pki/CA.crt /usr/local/share/ca-certificates/CA.crt
sudo chmod 0644 /usr/local/share/ca-certificates/CA.crt
sudo update-ca-certificates
echo QUIT | openssl s_client -connect localhost:7000 -showcerts 2>/dev/null | openssl x509 -noout -subject -ext subjectAltName
sudo vim /etc/hosts

Command Breakdown

  • openssl s_client -connect localhost:7000: Initiates an SSL/TLS connection to inspect handshakes and certificates.
  • openssl x509 -noout -subject -ext subjectAltName: Extracts the certificate subject and Subject Alternative Name extensions without printing encoded data.

Observed Results

  • update-ca-certificates output: 1 added, 0 removed; done.
  • OpenSSL output:
subject=CN = webapp
X509v3 Subject Alternative Name: critical
    DNS:webapp
  • Attempted shell redirection to /etc/hosts:
echo '127.0.10.1 webapp' >> /etc/hosts        # Failed: Permission denied
sudo echo '127.0.10.1 webapp' >> /etc/hosts   # Failed: Permission denied

(Note: Standard shell redirection >> runs under the unprivileged user shell before sudo executes. The file was successfully updated using sudo vim /etc/hosts).

Interpretation

The certificate required clients to connect specifically to hostname webapp (matching DNS:webapp). The CA was trusted in /usr/local/share/ca-certificates/CA.crt, and /etc/hosts correctly resolved webapp to loopback address 127.0.10.1.


Step 6: Resolving Kernel Mandatory Access Control Blocks via AppArmor

Objective

Identify kernel-level AppArmor enforcement rules restricting file access for /usr/local/bin/webapp, update the AppArmor profile, and reload it into kernel space.

Commands Executed

sudo nano /etc/apparmor.d/usr.local.bin.webapp
sudo apparmor_parser -r /etc/apparmor.d/usr.local.bin.webapp
curl https://webapp:7000/users.html
/home/admin/agent/check.sh

Profile Modification Detail

In /etc/apparmor.d/usr.local.bin.webapp, the profile lacked rules permitting the binary to read static files under /home/webapp/static-files/. The following rules were added before the closing brace:

  /home/webapp/static-files/ r,
  /home/webapp/static-files/* r,

Command Breakdown

  • apparmor_parser: Loads, parses, and compiles AppArmor profiles into kernel space.
  • -r: Replaces and reloads an existing profile in the running kernel.

Observed Result

admin@ip-10-1-13-82:~$ curl https://webapp:7000/users.html
<html>
  <head> </head>
  <body>
    <p>From Users Page</p>
  </body>
</html>
admin@ip-10-1-13-82:~$ /home/admin/agent/check.sh 
OK

Interpretation

The persistent 403 Forbidden error was caused by AppArmor Mandatory Access Control. While POSIX permissions allowed webapp to read /home/webapp/static-files/users.html, the AppArmor kernel profile restricted the process binary from reading outside authorized paths. Reloading the profile resolved the issue completely.


Root Cause Analysis

The outage resulted from four distinct security configuration flaws across different system layers:

┌────────────────────────────────────────────────────────────────────────┐
│                        Root Cause Breakdown                            │
├────────────────────────────────────────────────────────────────────────┤
│ Layer 1: Process Startup │ Restricted POSIX permissions (0700 root:root)│
│                          │ on /home/webapp/pki blocked reading server  │
│                          │ certs, causing process start retry loops.   │
├──────────────────────────┼─────────────────────────────────────────────┤
│ Layer 2: TLS Validation  │ Client curl requests failed strict validation│
│                          │ due to untrusted private CA (CA.crt) and    │
│                          │ missing DNS SAN mapping in /etc/hosts.      │
├──────────────────────────┼─────────────────────────────────────────────┤
│ Layer 3: POSIX Access    │ Static web files owned by admin:admin with   │
│                          │ 0750 permissions blocked user webapp.       │
├──────────────────────────┼─────────────────────────────────────────────┤
│ Layer 4: Kernel MAC      │ AppArmor policy confined /usr/local/bin/    │
│                          │ webapp and denied read access to static path│
│                          │ /home/webapp/static-files/*.                │
└──────────────────────────┴─────────────────────────────────────────────┘

Why POSIX Fixes Alone Were Insufficient

Linux security modules operate after standard POSIX permission checks pass. When curl requested users.html, the system evaluated permissions as follows:

  1. Kernel DAC Check: Kernel checked if user webapp had POSIX r access to users.html and x access to parent directories. (Passed)
  2. Kernel MAC Check (AppArmor): Kernel checked if the executing profile (usr.local.bin.webapp) permitted path /home/webapp/static-files/users.html with flag r. (Failed)
  3. Result: AppArmor intercepted the system call (openat) and returned EACCES (Permission Denied) to the process, causing the Go HTTP file server to return 403 Forbidden.

Solution and Recovery

Summary of System Changes Applied

1. POSIX Permission Recovery

# Correct ownership of PKI and static files
sudo chown -R webapp:webapp /home/webapp/pki /home/webapp/static-files

# Set directory traversal permissions
sudo chmod 0755 /home/webapp /home/webapp/static-files /home/webapp/pki

# Set restrictive file permissions
sudo chmod 0600 /home/webapp/pki/server.crt /home/webapp/pki/server.pem
sudo chmod 0640 /home/webapp/static-files/*

2. System CA Trust & Hostname Resolution

# Install private CA into system store
sudo cp /home/webapp/pki/CA.crt /usr/local/share/ca-certificates/CA.crt
sudo chmod 0644 /usr/local/share/ca-certificates/CA.crt
sudo update-ca-certificates

# Update host resolution in /etc/hosts
echo "127.0.10.1 webapp" | sudo tee -a /etc/hosts

3. AppArmor Profile Update

Edit /etc/apparmor.d/usr.local.bin.webapp to include:

  /home/webapp/static-files/ r,
  /home/webapp/static-files/* r,

Reload profile into kernel space:

sudo apparmor_parser -r /etc/apparmor.d/usr.local.bin.webapp

Verification

Confirm the full resolution by validating service status, process ports, TLS handshakes, and endpoint response:

# 1. Verify service status and listening port
sudo systemctl status webapp --no-pager
sudo ss -tulpn | grep 7000

# 2. Test HTTPS endpoint without ignoring checks (no -k flag)
curl -v https://webapp:7000/users.html

# 3. Execute scenario verification test
/home/admin/agent/check.sh

Expected Output

* Connected to webapp (127.0.10.1) port 7000
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: CN=webapp
*  SSL certificate verify ok.
< HTTP/1.1 200 OK
<html>
  <head> </head>
  <body>
    <p>From Users Page</p>
  </body>
</html>

Deeper Dive: Understanding the Linux Commands

Command: journalctl

Queries systemd logs compiled by systemd-journald.

  • Syntax: journalctl [options] [matches]

  • Flags Used:

  • -u webapp: Filter by unit name.

  • -e: Jump to end of logs.

  • -f: Follow continuous output.

  • -n 10: Display only the last 10 log entries.

  • Other Useful Flags:

  • -k: Filter kernel messages only (dmesg equivalent).

  • -p err: Filter by priority level (0: emerg to 3: err).

  • --since "10 minutes ago": Filter logs within a specific time window.

Command: openssl

Command-line toolkit for TLS/SSL management and certificate parsing.

  • Syntax: openssl sub-command [options]

  • Flags Used:

  • s_client -connect host:port: Debugs TLS connections by initiating client handshakes.

  • x509 -noout -subject -ext subjectAltName: Parses X.509 certificates to print Subject and SAN information without displaying raw PEM blocks.

  • Other Useful Flags:

  • openssl x509 -in cert.crt -text: Dumps complete X.509 certificate metadata.

  • openssl verify -CAfile CA.crt server.crt: Validates if a server certificate was signed by a specific CA certificate.

Command: apparmor_parser

Loads and manages AppArmor Mandatory Access Control profiles in the Linux kernel.

  • Syntax: apparmor_parser [options] <profile_file>

  • Flags Used:

  • -r: Replaces and reloads an existing profile in kernel memory.

  • Other Useful Flags:

  • -C: Compiles profile without loading into kernel (syntax check).

  • -R: Unloads/removes a profile from kernel memory.

  • -v: Verbose output during profile parsing.

Command: update-ca-certificates

Updates system-wide trusted CA bundle under /etc/ssl/certs/ca-certificates.crt.

  • Syntax: update-ca-certificates [options]
  • Flags Used: None (default behavior scans /usr/local/share/ca-certificates/).
  • Other Useful Flags:
  • -f / --fresh: Wipes existing symlinks in /etc/ssl/certs and rebuilds the bundle completely.

Explore Alternative Troubleshooting Methods

1. Monitoring AppArmor Denials with dmesg or auditd

Instead of guessing whether AppArmor is blocking access, inspect kernel audit logs directly:

# Filter kernel audit logs for AppArmor denial events
sudo dmesg | grep -i apparmor
# Or inspect syslog audit logs
sudo grep -i "apparmor=\"DENIED\"" /var/log/syslog
  • Advantage: Instantly reveals denied paths, process names, and requested operation flags (e.g., requested_mask="r").

2. Tracing System Calls with strace

To trace exact file descriptor open calls and kernel error codes:

sudo strace -e trace=open,openat,access -p $(pgrep webapp)
  • Advantage: Shows raw kernel response codes (e.g., openat(AT_FDCWD, "/home/webapp/static-files/users.html", O_RDONLY) = -1 EACCES (Permission denied)).

3. Safely Appending to Privileged Files using tee

When appending text to root-owned files like /etc/hosts without running an editor:

echo "127.0.10.1 webapp" | sudo tee -a /etc/hosts > /dev/null
  • Advantage: Avoids Permission denied redirection errors by running tee under sudo.

Deeper Linux Concepts and Internals

POSIX Permissions (DAC) vs. AppArmor (MAC)

   Incoming Syscall: openat("/home/webapp/static-files/users.html")
                                 │
                                 ▼
                     [ Step 1: Check POSIX DAC ]
                  Evaluates User / Group / Other bits
                                 │
                        Passed   │   Failed -> EACCES (Permission Denied)
                                 ▼
                     [ Step 2: Check AppArmor MAC ]
                     Evaluates Kernel Profile Rules
                                 │
                        Passed   │   Failed -> EACCES (AppArmor Block)
                                 ▼
                       [ System Call Succeeds ]
  1. POSIX Discretionary Access Control (DAC):
  • Managed by file owners (chown, chmod).
  • Operates on User, Group, and Other permission masks.
  • Requires execute (+x) permissions on every directory in a file path to allow traversal.
  1. AppArmor Mandatory Access Control (MAC):
  • Enforced directly by the Linux Security Module (LSM) framework in the kernel.
  • Path-based enforcement restricts process binaries regardless of user privileges (even root is restricted if confined by an AppArmor profile).
  • Rules define exact file path capabilities: r (read), w (write), ux (unconfined execute), k (lock).

Common Mistakes and Troubleshooting Pitfalls

  1. Confusing DAC and MAC Denials: Assuming 403 Forbidden or Permission Denied is always fixed by chmod 777 or chown. If AppArmor is active, POSIX permission changes have no effect on MAC restrictions.
  2. Incorrect Directory Traversal Permissions: Granting read (r) access to a static file without granting execute (x) permission on parent directories (/home/webapp).
  3. Shell Redirection with Sudo: Executing sudo echo "data" >> /etc/hosts. The shell redirection >> occurs in the unprivileged parent shell before sudo is spawned. Use sudo tee -a instead.
  4. Using Insecure Flags in Verification: Testing broken HTTPS services using curl -k. This bypasses TLS validation entirely, hiding missing CA certificates, expired validity dates, or hostname SAN mismatches.

Quick Reference

Command Summary

Command Purpose Useful Flags Typical Use Case
journalctl Inspect systemd daemon logs -u <service>, -f, -e, -p err Diagnosing daemon startup failures
apparmor_parser Load/reload AppArmor profiles -r (reload), -C (check syntax) Applying MAC rule updates
update-ca-certificates Update system root CA bundle -f (fresh rebuild) Installing custom private CAs
openssl s_client Inspect TLS connections -connect host:port, -showcerts Debugging HTTPS handshakes
openssl x509 Parse X.509 certificate data -noout, -subject, -ext subjectAltName Verifying cert SAN and Common Name
sudo tee -a Append text to root files -a (append) Safe script-driven editing of /etc/hosts

Key Lessons to Remember

  • Always verify systemd logs (journalctl -u <unit>) first when ports are inactive (Connection refused).
  • Mandatory Access Control (AppArmor) rules override POSIX DAC permissions. Always check /etc/apparmor.d/ or dmesg | grep apparmor when permission errors persist despite correct file ownership.
  • Strict HTTPS validation requires three components: a valid server cert, a trusted root CA in the system store, and matching DNS SAN hostnames resolving in /etc/hosts.

Further Learning and Practice

  1. AppArmor Profile Development: Practice profiling a binary using aa-autodep and aa-logprof to automatically generate least-privilege security profiles.
  2. Custom PKI Lab: Generate a custom Root CA using OpenSSL, sign a server certificate with SAN extensions, configure an Nginx/Golang web server, and install the CA into /usr/local/share/ca-certificates/.
  3. POSIX Permission Boundary Testing: Test minimal permission sets on web server document roots (e.g., directory 0711 vs 0755, files 0640 vs 0644) to enforce least-privilege system administration.