Domain 4Running SystemsCORE

Managing Kernel

Guide to managing the Linux kernel in RHEL, inspecting kernel events, managing kernel modules with modprobe/lsmod, setting persistent module parameters, and performing kernel updates.

Domain: Operate running systems•Updated: 2026-08-30
#Kernel#Dmesg#Modprobe#Sysctl#RHCSA#RHEL

Managing Kernel

Theory Questions

1. Identify what causes a tainted kernel.

  • Command(s):
cat /proc/sys/kernel/tainted
dmesg | grep -i tainted
  • Why: A kernel becomes “tainted” when operational conditions compromise standard system stability or supportability. Common causes include loading proprietary or out-of-tree binary modules (e.g., third-party hardware drivers), forcing module loads (modprobe -f), encountering severe hardware Machine Check Exceptions (MCE), or executing staging drivers.
  • Gotchas: Loading unverified or unsigned third-party drivers on an RHCSA system can taint the kernel, causing official diagnostic utilities to flag the system as unsupported.
  • Verify:
cat /proc/sys/kernel/tainted

(Returns 0 if the kernel is untainted and clean)


2. Show kernel events since booting.

  • Command(s):
dmesg -T
# OR
journalctl -k -b
  • Why: dmesg reads the kernel ring buffer directly, while journalctl -k -b queries systemd-journald specifically for kernel messages recorded during the current boot cycle. Using -T with dmesg converts raw kernel timestamps into human-readable date strings.
  • Gotchas: Relying solely on dmesg on high-uptime systems where high log volume may cause the kernel ring buffer to overflow and overwrite early boot events.
  • Verify:
journalctl -k -b | head -n 20

3. Find the actual version of the running kernel.

  • Command(s):
uname -r
  • Why: uname -r queries the kernel release system call and outputs the active kernel build string (e.g., 5.14.0-427.el9.x86_64).
  • Gotchas: Installing a new kernel package via dnf updates on-disk files but does not change the output of uname -r until the system is rebooted.
  • Verify:
uname -r

4. Show the current version of RHEL in use.

  • Command(s):
cat /etc/redhat-release
# OR
cat /etc/os-release
  • Why: /etc/redhat-release provides a single-line release identifier specifying the distribution name and minor release number.
  • Gotchas: Confusing the RHEL distribution release version (e.g., Red Hat Enterprise Linux release 9.4) with the Linux kernel version (e.g., 5.14.0-...).
  • Verify:
cat /etc/redhat-release

5. Identify the process that helps the kernel initialize hardware devices.

  • Command(s):
systemctl status systemd-udevd
  • Why: The systemd-udevd daemon receives uevents directly from the Linux kernel whenever hardware devices are attached, enumerated, or modified, dynamically populating device nodes under /dev/ and invoking driver loading scripts.
  • Gotchas: Stopping or masking systemd-udevd.service will prevent the system from automatically discovering newly attached block storage or network adapters.
  • Verify:
systemctl is-active systemd-udevd

6. Locate the default rules for initializing new hardware devices.

  • Path / Command:
# System default rules:
ls -l /usr/lib/udev/rules.d/

# Custom local rules:
ls -l /etc/udev/rules.d/
  • Why: udev evaluates rules in priority order. Default rules provided by system packages reside under /usr/lib/udev/rules.d/, while administrator overrides and custom rules belong in /etc/udev/rules.d/.
  • Gotchas: Editing rule files directly inside /usr/lib/udev/rules.d/; package updates will overwrite manual edits in /usr/lib/. Always place custom rules in /etc/udev/rules.d/.
  • Verify:
ls -la /etc/udev/rules.d/

7. Unload a kernel module including all of its dependencies.

  • Command(s):
modprobe -r <module_name>
  • Why: Unlike rmmod, which only attempts to unload a single specified module, modprobe -r recursively unloads the specified module and any dependent modules that are no longer actively in use.
  • Gotchas: modprobe -r will fail with a “Module is in use” error if any running process or active network interface holds an open reference to the driver module.
  • Verify:
lsmod | grep <module_name>

8. Verify that appropriate kernel modules are loaded for hardware in your server.

  • Command(s):
lspci -k
# OR for USB devices:
lsusb -v
  • Why: lspci -k lists all PCI devices alongside the specific kernel module driver currently bound to each device (“Kernel driver in use:”) and available candidate modules (“Kernel modules:”).
  • Gotchas: Mistaking physical hardware component vendor names for the actual software kernel module name (e.g., an Intel Ethernet controller using the e1000e driver module).
  • Verify:
lspci -k | grep -A 3 -i "ethernet"

9. Specify a kernel module parameter persistently.

  • Command(s): Create a configuration file in /etc/modprobe.d/:
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
  • Why: modprobe parses configuration directives inside /etc/modprobe.d/*.conf whenever a module is requested automatically by udev or executed manually via command line.
  • Gotchas: Attempting to configure module parameters in /etc/sysctl.conf. sysctl manages runtime kernel parameters under /proc/sys/, whereas /etc/modprobe.d/ sets options passed to loadable kernel module drivers (.ko).
  • Verify:
cat /etc/modprobe.d/<module_name>.conf

10. Evaluate statements about updating the kernel, identifying true and false claims.

  • Command(s):
rpm -q kernel
  • Statements Evaluation:

  • True: Updating a kernel installs the new version alongside existing kernels rather than overwriting them, preserving fallback boot options in GRUB.

  • True: Switching system execution to a newly installed kernel requires a full system reboot (unless using live-patching tools such as kpatch).

  • False: Executing dnf update kernel immediately removes older working kernels from disk. (Correction: dnf retains older kernels based on the installonly_limit setting in /etc/dnf/dnf.conf, defaulting to 3 kernels.)

  • Why: Maintaining multiple kernel packages side-by-side guarantees that a system can boot into a known-good secondary kernel if the updated kernel fails.

  • Gotchas: Attempting to use rpm -Uvh kernel... historically overwrote the existing kernel; dnf update properly executes an install-only operation for kernel RPMs.

  • Verify:

rpm -q kernel

11. Show the current kernel version on your computer.

  • Command(s):
uname -r
  • Why: Prints the release version string of the currently running kernel instance.
  • Gotchas: Confusing uname -r (kernel release version) with uname -v (kernel compilation build timestamp).
  • Verify:
uname -r

12. Find current version information about your RHEL installation.

  • Command(s):
cat /etc/redhat-release
  • Why: Returns the precise distribution branding and version number of Red Hat Enterprise Linux.
  • Gotchas: Checking /etc/issue, which is frequently modified by security hardening guidelines to display login warning banners.
  • Verify:
cat /etc/redhat-release

13. List kernel modules currently loaded.

  • Command(s):
lsmod
  • Why: lsmod formats and displays the contents of /proc/modules, showing loaded module names, memory sizes, and dependent module usage counts.
  • Gotchas: lsmod output can span several screens; pipe output into grep to filter for specific driver names.
  • Verify:
lsmod | head -n 10

14. Discover kernel module parameters.

  • Command(s):
modinfo -p <module_name>
  • Why: modinfo -p inspects module binary metadata and lists all available configurable parameters alongside their data types and descriptions.
  • Gotchas: Driver drivers built statically into the main kernel binary (instead of compiled as loadable .ko modules) will not return metadata via modinfo.
  • Verify:
modinfo -p <module_name>

15. Unload a kernel module.

  • Command(s):
modprobe -r <module_name>
  • Why: Safe mechanism to unload a module and clean up unused module dependencies from kernel memory.
  • Gotchas: Running rmmod instead of modprobe -r leaves unused dependency modules loaded in memory.
  • Verify:
lsmod | grep <module_name>

16. Handle errors when trying to unload a kernel module.

  • Command(s):
# 1. Identify active dependencies and usage count:
lsmod | grep <module_name>

# 2. Identify processes accessing associated device files (if applicable):
fuser -v /dev/<device_node>
# OR
lsof /dev/<device_node>

# 3. Stop dependent services and retry unload:
modprobe -r <module_name>
  • Why: Unload errors occur when the module’s reference count is greater than zero. Terminating processes using the device node or stopping dependent services releases module references.
  • Gotchas: Using rmmod -f (force removal). Forcing module unloading on production kernels frequently destabilizes memory management and causes a kernel panic.
  • Verify:
lsmod | grep <module_name>

17. Find which kernel module parameters are supported.

  • Command(s):
modinfo -p <module_name>
  • Why: Directly reads driver symbol definitions to extract supported operational switches.
  • Gotchas: Assuming module parameter names match sysctl kernel parameter paths. Module parameters are unique to individual driver source implementations.
  • Verify:
modinfo -p <module_name>

18. Specify kernel module parameters for persistent loading.

  • Command(s):
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
  • Why: Places module options in the default configuration directory parsed by modprobe upon module insertion.
  • Gotchas: Syntax errors in /etc/modprobe.d/*.conf files can prevent module drivers from initializing correctly at system boot.
  • Verify:
cat /etc/modprobe.d/<module_name>.conf

19. Write a configuration line to automatically load a module with a specific parameter.

  • Command(s):
# 1. Configure auto-load on boot:
echo "<module_name>" > /etc/modules-load.d/<module_name>.conf

# 2. Configure module parameter:
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
  • Why: Auto-loading modules requires two steps if custom options are needed: /etc/modules-load.d/ tells systemd to load the module name at boot, and /etc/modprobe.d/ defines the parameters passed during load time.
  • Gotchas: Writing options ... inside /etc/modules-load.d/*.conf. Files in /etc/modules-load.d/ must only contain plain module names, one per line.
  • Verify:
cat /etc/modules-load.d/<module_name>.conf
cat /etc/modprobe.d/<module_name>.conf

20. Install a new version of the kernel.

  • Command(s):
dnf update -y kernel
  • Why: Downloads and installs the latest kernel RPM package, generates a matching initramfs image, updates bootloader options, and sets the new kernel as the default boot entry in GRUB.
  • Gotchas: Executing dnf reinstall kernel when an upgrade is required, or forgetting to reboot the system to run the new kernel.
  • Verify:
grubby --default-kernel

Lab Tasks

Kernel Practice

Task 1: Check whether a new kernel version is available. If so, install it and reboot so that the new kernel is used.

  • Command(s):
# 1. Check current running kernel:
uname -r

# 2. Check for kernel updates:
dnf check-update kernel

# 3. Install updated kernel package:
dnf update -y kernel

# 4. Verify default boot kernel setting:
grubby --default-kernel

# 5. Reboot system to load new kernel:
reboot
  • Why:

  • dnf check-update kernel: Queries configured repositories for newer kernel builds.

  • dnf update -y kernel: Installs the update alongside existing kernels.

  • grubby --default-kernel: Validates that GRUB has set the newly installed kernel as the default boot option.

  • Gotchas: Assuming dnf update automatically applies the kernel into runtime memory. A full system reboot is required to execute the new kernel binary.

  • Verify Post-Reboot:

uname -r

(Confirms that uname -r returns the newly updated kernel release version string)


Task 2: Show recent events logged by the kernel.

  • Command(s):
dmesg -T | tail -n 30
# OR
journalctl -k -n 30 --no-pager
  • Why: Filters output to display the 30 most recent kernel messages captured in memory or systemd-journald logs.
  • Gotchas: Forgetting --no-pager when piping journalctl inside automated verification scripts.
  • Verify:
dmesg | tail -n 5

Task 3: Locate the kernel module used by your network card. Find out whether it has configurable options.

  • Command(s):
# 1. Locate network hardware and active kernel driver:
lspci -k | grep -A 3 -i "ethernet"

# 2. Query driver parameters (example assuming module 'e1000e'):
modinfo -p e1000e
  • Why: lspci -k identifies the exact driver module name bound to the physical network card, and modinfo -p lists configurable module parameters.
  • Gotchas: Attempting to run modinfo on generic interface names (e.g., eth0 or enp1s0) instead of the driver module name (e.g., e1000e or r8169).
  • Verify:
modinfo -p $(lspci -k | grep -A 3 -i "ethernet" | grep "Kernel driver in use" | awk '{print $5}')

Task 4: Try loading one of these kernel module options manually. If successful, make the change persistent.

  • Command(s):
# 1. Unload module (using 'dummy' network module for non-disruptive testing):
modprobe -r dummy 2>/dev/null || true

# 2. Test manual module load with parameter:
modprobe dummy numdummies=2

# 3. Verify parameter in runtime sysfs:
cat /sys/module/dummy/parameters/numdummies

# 4. Make configuration persistent:
echo "dummy" > /etc/modules-load.d/dummy.conf
echo "options dummy numdummies=2" > /etc/modprobe.d/dummy.conf
  • Why:

  • modprobe dummy numdummies=2: Dynamically loads module into kernel memory with custom parameter values.

  • /etc/modules-load.d/dummy.conf: Ensures the module is automatically loaded at system startup.

  • /etc/modprobe.d/dummy.conf: Persistently passes the parameter (numdummies=2) to the module whenever loaded.

  • Gotchas: Testing parameter modifications directly on active primary network card drivers (e.g., e1000e), which can drop SSH remote access sessions. Always use isolated test modules like dummy during practice routines.

  • Verify & Reboot Persistence Test:

# Reboot system:
reboot

# Post-reboot check:
lsmod | grep dummy
cat /sys/module/dummy/parameters/numdummies

(Output must display 2, verifying persistence across system reboots)