Managing Kernel
Theory Questions
1. Identify what causes a tainted kernel.
- Command(s):
cat /proc/sys/kernel/tainted
dmesg | grep -i tainted
- Why: A kernel becomes “tainted” when operational conditions compromise standard system stability or supportability. Common causes include loading proprietary or out-of-tree binary modules (e.g., third-party hardware drivers), forcing module loads (
modprobe -f), encountering severe hardware Machine Check Exceptions (MCE), or executing staging drivers. - Gotchas: Loading unverified or unsigned third-party drivers on an RHCSA system can taint the kernel, causing official diagnostic utilities to flag the system as unsupported.
- Verify:
cat /proc/sys/kernel/tainted
(Returns 0 if the kernel is untainted and clean)
2. Show kernel events since booting.
- Command(s):
dmesg -T
# OR
journalctl -k -b
- Why:
dmesgreads the kernel ring buffer directly, whilejournalctl -k -bqueries systemd-journald specifically for kernel messages recorded during the current boot cycle. Using-Twithdmesgconverts raw kernel timestamps into human-readable date strings. - Gotchas: Relying solely on
dmesgon high-uptime systems where high log volume may cause the kernel ring buffer to overflow and overwrite early boot events. - Verify:
journalctl -k -b | head -n 20
3. Find the actual version of the running kernel.
- Command(s):
uname -r
- Why:
uname -rqueries the kernel release system call and outputs the active kernel build string (e.g.,5.14.0-427.el9.x86_64). - Gotchas: Installing a new kernel package via
dnfupdates on-disk files but does not change the output ofuname -runtil the system is rebooted. - Verify:
uname -r
4. Show the current version of RHEL in use.
- Command(s):
cat /etc/redhat-release
# OR
cat /etc/os-release
- Why:
/etc/redhat-releaseprovides a single-line release identifier specifying the distribution name and minor release number. - Gotchas: Confusing the RHEL distribution release version (e.g., Red Hat Enterprise Linux release 9.4) with the Linux kernel version (e.g.,
5.14.0-...). - Verify:
cat /etc/redhat-release
5. Identify the process that helps the kernel initialize hardware devices.
- Command(s):
systemctl status systemd-udevd
- Why: The
systemd-udevddaemon receives uevents directly from the Linux kernel whenever hardware devices are attached, enumerated, or modified, dynamically populating device nodes under/dev/and invoking driver loading scripts. - Gotchas: Stopping or masking
systemd-udevd.servicewill prevent the system from automatically discovering newly attached block storage or network adapters. - Verify:
systemctl is-active systemd-udevd
6. Locate the default rules for initializing new hardware devices.
- Path / Command:
# System default rules:
ls -l /usr/lib/udev/rules.d/
# Custom local rules:
ls -l /etc/udev/rules.d/
- Why:
udevevaluates rules in priority order. Default rules provided by system packages reside under/usr/lib/udev/rules.d/, while administrator overrides and custom rules belong in/etc/udev/rules.d/. - Gotchas: Editing rule files directly inside
/usr/lib/udev/rules.d/; package updates will overwrite manual edits in/usr/lib/. Always place custom rules in/etc/udev/rules.d/. - Verify:
ls -la /etc/udev/rules.d/
7. Unload a kernel module including all of its dependencies.
- Command(s):
modprobe -r <module_name>
- Why: Unlike
rmmod, which only attempts to unload a single specified module,modprobe -rrecursively unloads the specified module and any dependent modules that are no longer actively in use. - Gotchas:
modprobe -rwill fail with a “Module is in use” error if any running process or active network interface holds an open reference to the driver module. - Verify:
lsmod | grep <module_name>
8. Verify that appropriate kernel modules are loaded for hardware in your server.
- Command(s):
lspci -k
# OR for USB devices:
lsusb -v
- Why:
lspci -klists all PCI devices alongside the specific kernel module driver currently bound to each device (“Kernel driver in use:”) and available candidate modules (“Kernel modules:”). - Gotchas: Mistaking physical hardware component vendor names for the actual software kernel module name (e.g., an Intel Ethernet controller using the
e1000edriver module). - Verify:
lspci -k | grep -A 3 -i "ethernet"
9. Specify a kernel module parameter persistently.
- Command(s):
Create a configuration file in
/etc/modprobe.d/:
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
- Why:
modprobeparses configuration directives inside/etc/modprobe.d/*.confwhenever a module is requested automatically byudevor executed manually via command line. - Gotchas: Attempting to configure module parameters in
/etc/sysctl.conf.sysctlmanages runtime kernel parameters under/proc/sys/, whereas/etc/modprobe.d/sets options passed to loadable kernel module drivers (.ko). - Verify:
cat /etc/modprobe.d/<module_name>.conf
10. Evaluate statements about updating the kernel, identifying true and false claims.
- Command(s):
rpm -q kernel
-
Statements Evaluation:
-
True: Updating a kernel installs the new version alongside existing kernels rather than overwriting them, preserving fallback boot options in GRUB.
-
True: Switching system execution to a newly installed kernel requires a full system reboot (unless using live-patching tools such as
kpatch). -
False: Executing
dnf update kernelimmediately removes older working kernels from disk. (Correction:dnfretains older kernels based on theinstallonly_limitsetting in/etc/dnf/dnf.conf, defaulting to 3 kernels.) -
Why: Maintaining multiple kernel packages side-by-side guarantees that a system can boot into a known-good secondary kernel if the updated kernel fails.
-
Gotchas: Attempting to use
rpm -Uvh kernel...historically overwrote the existing kernel;dnf updateproperly executes an install-only operation for kernel RPMs. -
Verify:
rpm -q kernel
11. Show the current kernel version on your computer.
- Command(s):
uname -r
- Why: Prints the release version string of the currently running kernel instance.
- Gotchas: Confusing
uname -r(kernel release version) withuname -v(kernel compilation build timestamp). - Verify:
uname -r
12. Find current version information about your RHEL installation.
- Command(s):
cat /etc/redhat-release
- Why: Returns the precise distribution branding and version number of Red Hat Enterprise Linux.
- Gotchas: Checking
/etc/issue, which is frequently modified by security hardening guidelines to display login warning banners. - Verify:
cat /etc/redhat-release
13. List kernel modules currently loaded.
- Command(s):
lsmod
- Why:
lsmodformats and displays the contents of/proc/modules, showing loaded module names, memory sizes, and dependent module usage counts. - Gotchas:
lsmodoutput can span several screens; pipe output intogrepto filter for specific driver names. - Verify:
lsmod | head -n 10
14. Discover kernel module parameters.
- Command(s):
modinfo -p <module_name>
- Why:
modinfo -pinspects module binary metadata and lists all available configurable parameters alongside their data types and descriptions. - Gotchas: Driver drivers built statically into the main kernel binary (instead of compiled as loadable
.komodules) will not return metadata viamodinfo. - Verify:
modinfo -p <module_name>
15. Unload a kernel module.
- Command(s):
modprobe -r <module_name>
- Why: Safe mechanism to unload a module and clean up unused module dependencies from kernel memory.
- Gotchas: Running
rmmodinstead ofmodprobe -rleaves unused dependency modules loaded in memory. - Verify:
lsmod | grep <module_name>
16. Handle errors when trying to unload a kernel module.
- Command(s):
# 1. Identify active dependencies and usage count:
lsmod | grep <module_name>
# 2. Identify processes accessing associated device files (if applicable):
fuser -v /dev/<device_node>
# OR
lsof /dev/<device_node>
# 3. Stop dependent services and retry unload:
modprobe -r <module_name>
- Why: Unload errors occur when the module’s reference count is greater than zero. Terminating processes using the device node or stopping dependent services releases module references.
- Gotchas: Using
rmmod -f(force removal). Forcing module unloading on production kernels frequently destabilizes memory management and causes a kernel panic. - Verify:
lsmod | grep <module_name>
17. Find which kernel module parameters are supported.
- Command(s):
modinfo -p <module_name>
- Why: Directly reads driver symbol definitions to extract supported operational switches.
- Gotchas: Assuming module parameter names match
sysctlkernel parameter paths. Module parameters are unique to individual driver source implementations. - Verify:
modinfo -p <module_name>
18. Specify kernel module parameters for persistent loading.
- Command(s):
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
- Why: Places module options in the default configuration directory parsed by
modprobeupon module insertion. - Gotchas: Syntax errors in
/etc/modprobe.d/*.conffiles can prevent module drivers from initializing correctly at system boot. - Verify:
cat /etc/modprobe.d/<module_name>.conf
19. Write a configuration line to automatically load a module with a specific parameter.
- Command(s):
# 1. Configure auto-load on boot:
echo "<module_name>" > /etc/modules-load.d/<module_name>.conf
# 2. Configure module parameter:
echo "options <module_name> <parameter>=<value>" > /etc/modprobe.d/<module_name>.conf
- Why: Auto-loading modules requires two steps if custom options are needed:
/etc/modules-load.d/tells systemd to load the module name at boot, and/etc/modprobe.d/defines the parameters passed during load time. - Gotchas: Writing
options ...inside/etc/modules-load.d/*.conf. Files in/etc/modules-load.d/must only contain plain module names, one per line. - Verify:
cat /etc/modules-load.d/<module_name>.conf
cat /etc/modprobe.d/<module_name>.conf
20. Install a new version of the kernel.
- Command(s):
dnf update -y kernel
- Why: Downloads and installs the latest kernel RPM package, generates a matching
initramfsimage, updates bootloader options, and sets the new kernel as the default boot entry in GRUB. - Gotchas: Executing
dnf reinstall kernelwhen an upgrade is required, or forgetting to reboot the system to run the new kernel. - Verify:
grubby --default-kernel
Lab Tasks
Kernel Practice
Task 1: Check whether a new kernel version is available. If so, install it and reboot so that the new kernel is used.
- Command(s):
# 1. Check current running kernel:
uname -r
# 2. Check for kernel updates:
dnf check-update kernel
# 3. Install updated kernel package:
dnf update -y kernel
# 4. Verify default boot kernel setting:
grubby --default-kernel
# 5. Reboot system to load new kernel:
reboot
-
Why:
-
dnf check-update kernel: Queries configured repositories for newer kernel builds. -
dnf update -y kernel: Installs the update alongside existing kernels. -
grubby --default-kernel: Validates that GRUB has set the newly installed kernel as the default boot option. -
Gotchas: Assuming
dnf updateautomatically applies the kernel into runtime memory. A full system reboot is required to execute the new kernel binary. -
Verify Post-Reboot:
uname -r
(Confirms that uname -r returns the newly updated kernel release version string)
Task 2: Show recent events logged by the kernel.
- Command(s):
dmesg -T | tail -n 30
# OR
journalctl -k -n 30 --no-pager
- Why: Filters output to display the 30 most recent kernel messages captured in memory or systemd-journald logs.
- Gotchas: Forgetting
--no-pagerwhen piping journalctl inside automated verification scripts. - Verify:
dmesg | tail -n 5
Task 3: Locate the kernel module used by your network card. Find out whether it has configurable options.
- Command(s):
# 1. Locate network hardware and active kernel driver:
lspci -k | grep -A 3 -i "ethernet"
# 2. Query driver parameters (example assuming module 'e1000e'):
modinfo -p e1000e
- Why:
lspci -kidentifies the exact driver module name bound to the physical network card, andmodinfo -plists configurable module parameters. - Gotchas: Attempting to run
modinfoon generic interface names (e.g.,eth0orenp1s0) instead of the driver module name (e.g.,e1000eorr8169). - Verify:
modinfo -p $(lspci -k | grep -A 3 -i "ethernet" | grep "Kernel driver in use" | awk '{print $5}')
Task 4: Try loading one of these kernel module options manually. If successful, make the change persistent.
- Command(s):
# 1. Unload module (using 'dummy' network module for non-disruptive testing):
modprobe -r dummy 2>/dev/null || true
# 2. Test manual module load with parameter:
modprobe dummy numdummies=2
# 3. Verify parameter in runtime sysfs:
cat /sys/module/dummy/parameters/numdummies
# 4. Make configuration persistent:
echo "dummy" > /etc/modules-load.d/dummy.conf
echo "options dummy numdummies=2" > /etc/modprobe.d/dummy.conf
-
Why:
-
modprobe dummy numdummies=2: Dynamically loads module into kernel memory with custom parameter values. -
/etc/modules-load.d/dummy.conf: Ensures the module is automatically loaded at system startup. -
/etc/modprobe.d/dummy.conf: Persistently passes the parameter (numdummies=2) to the module whenever loaded. -
Gotchas: Testing parameter modifications directly on active primary network card drivers (e.g.,
e1000e), which can drop SSH remote access sessions. Always use isolated test modules likedummyduring practice routines. -
Verify & Reboot Persistence Test:
# Reboot system:
reboot
# Post-reboot check:
lsmod | grep dummy
cat /sys/module/dummy/parameters/numdummies
(Output must display 2, verifying persistence across system reboots)