Troubleshooting

Introduction to systemd-resolved

I’ve seen systemd-resolved cause its fair share of issues with split DNS and local hostnames, but it’s actually a powerful tool for managing DNS resolution on Linux systems. As part of the systemd ecosystem, it aims to improve the DNS resolution process and provide better integration with other systemd services.

Understanding Split DNS

Split DNS, or split-horizon DNS, is a technique used to provide different DNS responses based on the client’s location. This is commonly used in environments where internal and external DNS zones need to be separated - think of a company with an internal DNS zone for its internal network and a separate external DNS zone for its public-facing services. Don’t bother with split DNS if you don’t need it, but if you do, systemd-resolved can be configured to handle it.

Introduction to Emergency Mode

I’ve seen this go wrong when a Linux system encounters a critical issue during boot - it drops into emergency mode. This mode provides a minimal environment for troubleshooting and repair, allowing you to diagnose and fix issues that prevent the system from booting normally. In this article, we’ll explore how to troubleshoot failed mounts in emergency mode with systemd.

Understanding Emergency Mode

Emergency mode is a special boot target in systemd that provides a basic environment for troubleshooting. When a system enters emergency mode, it means that an error occurred during the boot process, and the system was unable to mount the root filesystem or other critical filesystems. The real trick is to understand that, in emergency mode, you’ll have access to a root shell, but many system services will not be started.

Introduction to Initramfs Recovery

I’ve seen this go wrong when a Linux system fails to boot due to a broken initramfs - it can be a real headache, especially if you rely on your system for daily work or critical services. The recent updates to the Linux kernel and initramfs tools may have introduced compatibility issues for some users, making it crucial to understand how to recover from such failures.

Introduction to systemd’s Restart Policy

I’ve seen this go wrong when a service is not properly configured - systemd’s ability to automatically restart services that fail or terminate unexpectedly can be a double-edged sword. On one hand, it helps maintain system stability and availability. On the other hand, if not configured correctly, it can lead to service thrashing, where a service is repeatedly restarted in a short period, potentially causing more harm than good.

Introduction to Emergency Mode

I’ve seen this go wrong when a critical system service fails or a filesystem gets corrupted - systemd boots you into emergency mode. This is a sign that something has gone wrong during the boot process, and it’s usually due to a failed mount, a critical system service failure, or even a corrupted root filesystem. When this happens, you’re presented with a minimal environment to troubleshoot and potentially repair your system.

Introduction to Rescue Shells

I’ve seen this go wrong when a Linux system fails to boot - it can be a real pain, especially if you’re not sure where to start troubleshooting. One useful tool in these situations is a rescue shell, which provides a minimal environment for repairing and recovering your system. In practice, I’ve found that a rescue shell can be a lifesaver.

Preparing for Recovery

Before you can use a rescue shell, you’ll need to boot your system using a live Linux media, such as a USB drive or CD. I usually start with a live USB, as it’s often easier to work with than a CD. You can use any Linux distribution for this purpose, but it’s often easiest to use the same distribution as your installed system. For example, if you’re running Ubuntu, you can use an Ubuntu live USB to boot your system.

Introduction to Network Manager

I’ve seen many Linux users struggle with Network Manager, a popular utility for managing network connections. It’s usually a straightforward tool, but sometimes it can be frustrating to deal with. One common issue is the “Network Manager Disabled” error, which can be tricky to resolve. In my experience, this error often occurs when Network Manager is unable to manage a network interface, and there are several reasons why this might happen.

Introduction to Initramfs Recovery

I’ve seen this go wrong when a Linux system fails to boot due to a broken initramfs - it can be a real challenge, especially for those without extensive experience in low-level system debugging. The initramfs, or initial RAM file system, is a temporary file system used during the boot process. It provides a minimal environment for the system to load the necessary modules and prepare the root file system for mounting. A corrupted or incorrectly configured initramfs can prevent the system from booting properly.

Introduction to Umask and ACLs

When working with shared directories in Linux, I’ve seen permission issues arise due to the default umask and Access Control Lists (ACLs). The real trick is understanding how these two settings interact. The umask is a 3-digit octal number that determines the default permissions for newly created files and directories, while ACLs provide a more fine-grained access control mechanism.

Understanding Umask

The umask is subtracted from the maximum possible permissions (777 for directories and 666 for files) to determine the default permissions. For example, a umask of 022 would result in default permissions of 755 for directories (777 - 022 = 755) and 644 for files (666 - 022 = 644). To view the current umask, you can use the umask command: